Rapid evaluation of the spectral signal detection threshold and Stieltjes transform

Research output: Contribution to journalArticlepeer-review

Abstract

Accurate detection of signal components is a frequently-encountered challenge in statistical applications with a low signal-to-noise ratio. This problem is particularly challenging in settings with heteroscedastic noise. In certain signal-plus-noise models of data, such as the classical spiked covariance model and its variants, there are closed formulas for the spectral signal detection threshold (the largest sample eigenvalue attributable solely to noise) for isotropic noise in the limit of infinitely large data matrices. However, more general noise models currently lack provably fast and accurate methods for numerically evaluating the threshold. In this work, we introduce a rapid algorithm for evaluating the spectral signal detection threshold in the limit of infinitely large data matrices. We consider noise matrices with a separable variance profile (whose variance matrix is rank 1), as these arise often in applications. The solution is based on nested applications of Newton’s method. We also devise a new algorithm for evaluating the Stieltjes transform of the spectral distribution at real values exceeding the threshold. The Stieltjes transform on this domain is known to be a key quantity in parameter estimation for spectral denoising methods. The correctness of both algorithms is proven from a detailed analysis of the master equations characterizing the Stieltjes transform, and their performance is demonstrated in numerical experiments.

Original languageEnglish (US)
Article number60
JournalAdvances in Computational Mathematics
Volume47
Issue number4
DOIs
StatePublished - Aug 2021

Bibliographical note

Funding Information:
This work was supported by NSF BIGDATA award IIS 1837992 and BSF award 2018230.

Publisher Copyright:
© 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Keywords

  • Limiting spectral distribution
  • Newton’s method
  • Separable variance profile
  • Spectral signal detection threshold
  • Stieltjes transform

Fingerprint

Dive into the research topics of 'Rapid evaluation of the spectral signal detection threshold and Stieltjes transform'. Together they form a unique fingerprint.

Cite this