Anchor-free correlated topic modeling

Xiao Fu, Kejun Huang, Nicholas D. Sidiropoulos, Qingjiang Shi, Mingyi Hong

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

In topic modeling, identifiability of the topics is an essential issue. Many topic modeling approaches have been developed under the premise that each topic has a characteristic anchor word that only appears in that topic. The anchor-word assumption is fragile in practice, because words and terms have multiple uses; yet it is commonly adopted because it enables identifiability guarantees. Remedies in the literature include using three- or higher-order word co-occurence statistics to come up with tensor factorization models, but such statistics need many more samples to obtain reliable estimates, and identifiability still hinges on additional assumptions, such as consecutive words being persistently drawn from the same topic. In this work, we propose a new topic identification criterion using second order statistics of the words. The criterion is theoretically guaranteed to identify the underlying topics even when the anchor-word assumption is grossly violated. An algorithm based on alternating optimization, and an efficient primal-dual algorithm are proposed to handle the resulting identification problem. The former exhibits high performance and is completely parameter-free; the latter affords up to 200 times speedup relative to the former, but requires step-size tuning and a slight sacrifice in accuracy. A variety of real text copora are employed to showcase the effectiveness of the approach, where the proposed anchor-free method demonstrates substantial improvements compared to a number of anchor-word based approaches under various evaluation metrics.

Original languageEnglish (US)
Article number8338424
Pages (from-to)1056-1071
Number of pages16
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume41
Issue number5
DOIs
StatePublished - May 1 2019

Bibliographical note

Funding Information:
The work of X. Fu, K. Huang, and N. D. Sidiropoulos is supported in part by the US National Science Foundation (NSF) under grants ECCS-1608961 and IIS-1247632. The work of Q. Shi is supported in part by NSFC under Grant 61671411, U1709219, 61374020, by the Fundamental Research Funds for the Central Universities, and by Zhe-jiang Provincial NSF of China under Grant LR15F010002. The work of M. Hong is supported in part by US National Science Foundation under Grant CCF-1526078, CMMI-1727757, and by AFOSR under Grant 15RT0767. Xiao Fu and Kejun Huang contributed equally.

Funding Information:
The work of X. Fu, K. Huang, and N. D. Sidiropoulos is supported in part by the US National Science Foundation (NSF) under grants ECCS-1608961 and IIS-1247632. The work of Q. Shi is supported in part by NSFC under Grant 61671411, U1709219, 61374020, by the Fundamental Research Funds for the Central Universities, and by Zhejiang Provincial NSF of China under Grant LR15F010002. The work of M. Hong is supported in part by US National Science Foundation under Grant CCF-1526078, CMMI- 1727757, and by AFOSR under Grant 15RT0767. Xiao Fu and Kejun Huang contributed equally.

Publisher Copyright:
© 1979-2012 IEEE.

Keywords

  • Topic modeling
  • anchor free
  • identifiability
  • non-convex optimization
  • nonnegative matrix factorization
  • sufficiently scattered

Fingerprint

Dive into the research topics of 'Anchor-free correlated topic modeling'. Together they form a unique fingerprint.

Cite this