Multidimensional mining of large-scale search logs: A topic-concept cube approach

Dongyeop Kang, Daxin Jiang, Jian Pei, Zhen Liao, Xiaohui Sun, Ho Jin Choi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Scopus citations

Abstract

In addition to search queries and the corresponding clickthrough information, search engine logs record multidimensional information about user search activities, such as search time, location, vertical, and search device. Multidimensional mining of search logs can provide novel insights and useful knowledge for both search engine users and developers. In this paper, we describe our topic-concept cube project, which addresses the business need of supporting multidimensional mining of search logs effectively and efficiently. We answer two challenges. First, search queries and click-through data are well recognized sparse, and thus have to be aggregated properly for effective analysis. Second, there is often a gap between the topic hierarchies in multidimensional aggregate analysis and queries in search logs. To address those challenges, we develop a novel topicconcept model that learns a hierarchy of concepts and topics automatically from search logs. Enabled by the topicconcept model, we construct a topic-concept cube that supports online multidimensional mining of search log data. A distinct feature of our approach is that, in addition to the standard dimensions such as time and location, our topicconcept cube has a dimension of topics and concepts, which substantially facilitates the analysis of log data. To handle a huge amount of log data, we develop distributed algorithms for learning model parameters efficiently. We also devise approaches to computing a topic-concept cube. We report an empirical study verifying the effectiveness and efficiency of our approach on a real data set of 1.96 billion queries and 2.73 billion clicks.

Original languageEnglish (US)
Title of host publicationProceedings of the 4th ACM International Conference on Web Search and Data Mining, WSDM 2011
Pages385-394
Number of pages10
DOIs
StatePublished - 2011
Externally publishedYes
Event4th ACM International Conference on Web Search and Data Mining, WSDM 2011 - Hong Kong, China
Duration: Feb 9 2011Feb 12 2011

Publication series

NameProceedings of the 4th ACM International Conference on Web Search and Data Mining, WSDM 2011

Conference

Conference4th ACM International Conference on Web Search and Data Mining, WSDM 2011
Country/TerritoryChina
CityHong Kong
Period2/9/112/12/11

Keywords

  • OLAP
  • Search log
  • Topic-concept cube

Fingerprint

Dive into the research topics of 'Multidimensional mining of large-scale search logs: A topic-concept cube approach'. Together they form a unique fingerprint.

Cite this