Learned from an email thread. This is the kind of problem you'll never think about if you are a student using R. Suppose you have a k by n matrix, say x, where k is small but n is huge (say 1,000,000), and you want to get the maximum in each column, that is, n different values.
The naive way of apply(x, 2, max) takes forever to complete.
The new trick I learned is:
# transpose x and convert to a data.frame
t.x <- as.data.frame(t(x))
# use do.call with "pmax" and the input matrix
x.max <- do.call("pmax", t.x)
Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts
Saturday, November 21, 2009
Wednesday, January 14, 2009
Interesting Proability Question
Just got an interesting probability question:
Suppose n points spread out in a 2D surface. Let's say the coordinates of those points are (x_i, y_i), i=1, 2, ..., n. Denote x_L, x_U the 25% and 75% quantile for (x_1, ..., x_n), and y_L, y_U the 25% and 75% quantile for (y_1, ..., y_n). p is the proportional of points lie in the small rectangle [x_L, y_L] *[x_U, y_U], then what is the range of p? (Assuming n is large)
It seems to me that if x and y are independent, then p is 1/4; but if x and y are perfectly correlated, say x=y, then p is 1/2. My guess is that 1/4 <= p <= 1/2, but anyone can verify this?
Suppose n points spread out in a 2D surface. Let's say the coordinates of those points are (x_i, y_i), i=1, 2, ..., n. Denote x_L, x_U the 25% and 75% quantile for (x_1, ..., x_n), and y_L, y_U the 25% and 75% quantile for (y_1, ..., y_n). p is the proportional of points lie in the small rectangle [x_L, y_L] *[x_U, y_U], then what is the range of p? (Assuming n is large)
It seems to me that if x and y are independent, then p is 1/4; but if x and y are perfectly correlated, say x=y, then p is 1/2. My guess is that 1/4 <= p <= 1/2, but anyone can verify this?
Subscribe to:
Posts (Atom)