Anchor-free correlated topic modeling

Xiao Fu, Kejun Huang, Nicholas D. Sidiropoulos, Qingjiang Shi, Mingyi Hong

Research output: Contribution to journalArticlepeer-review

15 Scopus citations

Abstract

In topic modeling, identifiability of the topics is an essential issue. Many topic modeling approaches have been developed under the premise that each topic has a characteristic anchor word that only appears in that topic. The anchor-word assumption is fragile in practice, because words and terms have multiple uses; yet it is commonly adopted because it enables identifiability guarantees. Remedies in the literature include using three- or higher-order word co-occurence statistics to come up with tensor factorization models, but such statistics need many more samples to obtain reliable estimates, and identifiability still hinges on additional assumptions, such as consecutive words being persistently drawn from the same topic. In this work, we propose a new topic identification criterion using second order statistics of the words. The criterion is theoretically guaranteed to identify the underlying topics even when the anchor-word assumption is grossly violated. An algorithm based on alternating optimization, and an efficient primal-dual algorithm are proposed to handle the resulting identification problem. The former exhibits high performance and is completely parameter-free; the latter affords up to 200 times speedup relative to the former, but requires step-size tuning and a slight sacrifice in accuracy. A variety of real text copora are employed to showcase the effectiveness of the approach, where the proposed anchor-free method demonstrates substantial improvements compared to a number of anchor-word based approaches under various evaluation metrics.

Original languageEnglish (US)
Article number8338424
Pages (from-to)1056-1071
Number of pages16
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume41
Issue number5
DOIs
StatePublished - May 1 2019

Bibliographical note

Publisher Copyright:
© 1979-2012 IEEE.

Keywords

  • Topic modeling
  • anchor free
  • identifiability
  • non-convex optimization
  • nonnegative matrix factorization
  • sufficiently scattered

Fingerprint

Dive into the research topics of 'Anchor-free correlated topic modeling'. Together they form a unique fingerprint.

Cite this