Measuring dependencies of order statistics: An information theoretic perspective

This work considers a random sample X1,X2,...,Xn drawn independently and identically distributed from some known parent distribution PX with X₍₁₎ ≤ X₍₂₎ ≤ ... ≤ X₍n₎ being the order statistics of the sample. Under the assumption of an invertible cumulative distribution function associated with the parent distribution PX, a distribution-free property is established showing that the f-divergence between the joint distribution of order statistics and the product distribution of order statistics does not depend on PX. Moreover, it is shown that the mutual information between two subsets of order statistics also satisfies a distribution-free property; that is, it does not depend on PX. Furthermore, the decoupling rates between X₍r₎ and X₍m) (i.e., rates at which the mutual information approaches zero) are characterized for various choices of (r,m). The work also considers discrete distributions, which do not satisfy the previously-stated invertibility assumption, and it is shown that no such distribution-free property holds: the mutual information between order statistics does depend on the parent distribution PX. Upper bounds on the decoupling rates in the discrete setting are also established.