Kullback-Leibler Approximation of Spectral Density Functions

Tryphon T. Georgiou, Anders Lindquist

Research output: Contribution to journalArticlepeer-review

126 Scopus citations


We introduce a Kullback-Leibler-type distance between spectral density functions of stationary stochastic processes and solve the problem of optimal approximation of a given spectral density ψ by one that is consistent with prescribed second-order statistics. In general, such statistics are expressed as the state covariance of a linear filter driven by a stochastic process whose spectral density is sought. In this context, we show i) that there is a unique spectral density φ which minimizes this Kullback-Leibler distance, ii) that this optimal approximate is of the form ψ / Q where the "correction term" Q is a rational spectral density function, and iii) that the coefficients of Q can be obtained numerically by solving a suitable convex optimization problem. In the special case where ψ = 1, the convex functional becomes quadratic and the solution is then specified by linear equations.

Original languageEnglish (US)
Pages (from-to)2910-2917
Number of pages8
JournalIEEE Transactions on Information Theory
Issue number11
StatePublished - Nov 2003

Bibliographical note

Funding Information:
Manuscript received May 2, 2002; revised January 5, 2003. This work was supported in part by grants from AFOSR, the Swedish Research Council, and the Göran Gustafsson Foundation. T. T. Georgiou is with the Department of Electrical Engineering, University of Minnesota, Minneapolis, MN 55455, USA (e-mail: tryphon@ece.umn.edu). A. Lindquist is with the Department of Mathematics, Division of Optimization and Systems Theory, Royal Institute of Technology, 100 44 Stockholm, Sweden (e-mail: alq@math.kth.se). Communicated by A. Kavcˇić, Associate Editor for Detection and Estimation. Digital Object Identifier 10.1109/TIT.2003.819324


  • Approximation of power spectra
  • Cross-entropy minimization
  • Kullback-Leibler distance
  • Mutual information
  • Optimization
  • Spectral estimation


Dive into the research topics of 'Kullback-Leibler Approximation of Spectral Density Functions'. Together they form a unique fingerprint.

Cite this