Direction-of-arrival estimation for coherent signals without knowledge of source number

A new method for direction-of-arrival (DOA) estimation of coherent signals is devised in this paper. The coherency of sources is decorrelated by employing an existing algorithm in which each row of the sample covariance matrix can be used to form a full rank Toeplitz matrix. Based on the joint diagonalization structure of the full set of Toeplitz matrices constructed from the rows, a new cost function that does not require a priori information of the source number is designed. A new spatial spectrum is then obtained where the DOAs are estimated via a 1-D search. Numerical examples are provided to demonstrate effectiveness of the proposed approach.