Distributed representation of multi-sense words: A loss driven approach

Saurav Manchanda, George Karypis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Word2Vec’s Skip Gram model is the current state-of-the-art approach for estimating the distributed representation of words. However, it assumes a single vector per word, which is not well-suited for representing words that have multiple senses. This work presents LDMI, a new model for estimating distributional representations of words. LDMI relies on the idea that, if a word carries multiple senses, then having a different representation for each of its senses should lead to a lower loss associated with predicting its co-occurring words, as opposed to the case when a single vector representation is used for all the senses. After identifying the multi-sense words, LDMI clusters the occurrences of these words to assign a sense to each occurrence. Experiments on the contextual word similarity task show that LDMI leads to better performance than competing approaches.

Original languageEnglish (US)
Title of host publicationAdvances in Knowledge Discovery and Data Mining - 22nd Pacific-Asia Conference, PAKDD 2018, Proceedings
EditorsBao Ho, Dinh Phung, Geoffrey I. Webb, Vincent S. Tseng, Mohadeseh Ganji, Lida Rashidi
PublisherSpringer Verlag
Pages337-349
Number of pages13
ISBN (Print)9783319930367
DOIs
StatePublished - 2018
Event22nd Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2018 - Melbourne, Australia
Duration: Jun 3 2018Jun 6 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10938 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other22nd Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2018
CountryAustralia
CityMelbourne
Period6/3/186/6/18

Bibliographical note

Funding Information:
Acknowledgments. This work was supported in part by NSF (IIS-1247632, IIP-1414153, IIS-1447788, IIS-1704074, CNS-1757916), Army Research Office (W911NF-14-1-0316), Intel Software and Services Group, and the Digital Technology Center at the University of Minnesota. Access to research and computing facilities was provided by the Digital Technology Center and the Minnesota Supercomputing Institute.

Fingerprint Dive into the research topics of 'Distributed representation of multi-sense words: A loss driven approach'. Together they form a unique fingerprint.

Cite this