Classifying the ideational impact of Information Systems review articles: A content-enriched deep learning approach

Julian Prester, Gerit Wagner, Guido Schryen, Nik Rushdi Hassan

Research output: Contribution to journalArticlepeer-review


Ideational impact refers to the uptake of a paper's ideas and concepts by subsequent research. It is defined in stark contrast to total citation impact, a measure predominantly used in research evaluation that assumes that all citations are equal. Understanding ideational impact is critical for evaluating research impact and understanding how scientific disciplines build a cumulative tradition. Research has only recently developed automated citation classification techniques to distinguish between different types of citations and generally does not emphasize the conceptual content of the citations and its ideational impact. To address this problem, we develop Deep Content-enriched Ideational Impact Classification (Deep-CENIC) as the first automated approach for ideational impact classification to support researchers' literature search practices. We evaluate Deep-CENIC on 1256 papers citing 24 information systems review articles from the IT business value domain. We show that Deep-CENIC significantly outperforms state-of-the-art benchmark models. We contribute to information systems research by operationalizing the concept of ideational impact, designing a recommender system for academic papers based on deep learning techniques, and empirically exploring the ideational impact in the IT business value domain.

Original languageEnglish (US)
Article number113432
JournalDecision Support Systems
StatePublished - Jan 2021

Bibliographical note

Funding Information:
We are grateful for the comments received during presentations of earlier versions of this paper at the University of Regensburg , University of New South Wales , and the International Conference on Information Systems . We also thank the editor and reviewers for their invaluable comments and suggestions. The research is supported by a grant of the German Science Foundation (DFG) for the research project "Epistemological Advances Through Qualitative Literature Reviews in Information Systems Research".


  • Academic recommender systems
  • Citation classification
  • Cumulative tradition
  • Deep learning
  • Ideational impact
  • Natural language processing

Fingerprint Dive into the research topics of 'Classifying the ideational impact of Information Systems review articles: A content-enriched deep learning approach'. Together they form a unique fingerprint.

Cite this