Measuring dependencies of order statistics: An information theoretic perspective

Alex Dytso, Martina Cardone, Cynthia Rush

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations


This work considers a random sample X1,X2,...,Xn drawn independently and identically distributed from some known parent distribution PX with X(1) ≤ X(2) ≤ ... ≤ X(n) being the order statistics of the sample. Under the assumption of an invertible cumulative distribution function associated with the parent distribution PX, a distribution-free property is established showing that the f-divergence between the joint distribution of order statistics and the product distribution of order statistics does not depend on PX. Moreover, it is shown that the mutual information between two subsets of order statistics also satisfies a distribution-free property; that is, it does not depend on PX. Furthermore, the decoupling rates between X(r) and X(m) (i.e., rates at which the mutual information approaches zero) are characterized for various choices of (r,m). The work also considers discrete distributions, which do not satisfy the previously-stated invertibility assumption, and it is shown that no such distribution-free property holds: the mutual information between order statistics does depend on the parent distribution PX. Upper bounds on the decoupling rates in the discrete setting are also established.

Original languageEnglish (US)
Title of host publication2020 IEEE Information Theory Workshop, ITW 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728159621
StatePublished - Apr 11 2021
Event2020 IEEE Information Theory Workshop, ITW 2020 - Virtual, Riva del Garda, Italy
Duration: Apr 11 2021Apr 15 2021

Publication series

Name2020 IEEE Information Theory Workshop, ITW 2020


Conference2020 IEEE Information Theory Workshop, ITW 2020
CityVirtual, Riva del Garda

Bibliographical note

Funding Information:
The work of M. Cardone was supported in part by the U.S. National Science Foundation under Grant CCF-1849757.

Publisher Copyright:
© 2021 IEEE.


Dive into the research topics of 'Measuring dependencies of order statistics: An information theoretic perspective'. Together they form a unique fingerprint.

Cite this