Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts

Saturday, November 21, 2009

R trick to get column max

Learned from an email thread. This is the kind of problem you'll never think about if you are a student using R. Suppose you have a k by n matrix, say x, where k is small but n is huge (say 1,000,000), and you want to get the maximum in each column, that is, n different values.

The naive way of apply(x, 2, max) takes forever to complete.

The new trick I learned is:

# transpose x and convert to a data.frame
t.x <- as.data.frame(t(x))
# use do.call with "pmax" and the input matrix
x.max <- do.call("pmax", t.x)

Wednesday, January 14, 2009

Interesting Proability Question

Just got an interesting probability question:

Suppose n points spread out in a 2D surface. Let's say the coordinates of those points are (x_i, y_i), i=1, 2, ..., n. Denote x_L, x_U the 25% and 75% quantile for (x_1, ..., x_n), and y_L, y_U the 25% and 75% quantile for (y_1, ..., y_n). p is the proportional of points lie in the small rectangle [x_L, y_L] *[x_U, y_U], then what is the range of p? (Assuming n is large)

It seems to me that if x and y are independent, then p is 1/4; but if x and y are perfectly correlated, say x=y, then p is 1/2. My guess is that 1/4 <= p <= 1/2, but anyone can verify this?