Towards an Experimental News User Community as Infrastructure for Recommendation Research

Joseph A. Konstan, Robin Burke, Edward C. Malthouse

Research output: Contribution to journalConference articlepeer-review


While substantial advances have been made in recommender systems - both in general and for news - using datasets, offline analyses, and one-shot experiments, longitudinal studies of real users remain the gold standard, and the only way to effectively measure the impact of recommender system designs (algorithmic and otherwise) on long-term user experience and behavior. While such infrastructure exists for studies within some individual organizations, the extensive cost and effort to build the systems, content streams, and user base make it prohibitive for most researchers to conduct such studies. We propose to develop shared research infrastructure for the research community, and have received funding to gather community input on requirements, resources, and research goals for such an infrastructure. If the full infrastructure proposal is funded, it would result in recruiting a community of thousands of users who agree to use a news delivery application within which various researchers would be install and conduct experiments. In this short paper we outline what we have heard and learned so far and present a set of questions to be directed to INRA attendees to gather their feedback at the workshop.

Original languageEnglish (US)
Pages (from-to)43-46
Number of pages4
JournalCEUR Workshop Proceedings
StatePublished - 2021
Event9th International Workshop on News Recommendation and Analytics, INRA 2021 - Amsterdam, Netherlands
Duration: Sep 27 2021 → …

Bibliographical note

Funding Information:
We have received a planning grant from the US National Science Foundation (grant CNS-2016397) to gather community input and establish feasibility and plans for this infrastructure. We have conducted initial consultations at an advertised open gathering at RecSys 2020 and through email solicitations for research questions that researchers would wish to address using the infrastructure. While NSF support is primarily focused on supporting US researchers, we recognize that successful design (and in some cases key questions) require a broader international perspective. Request for Input. We have brought this work to INRA with the goal of gathering input from both prospective users of this infrastructure and prospective partners in its construction and/or operation. While we are eager to receive input on a wide range of issues, there are five key questions that we hope will guide most of the input. We welcome input both at INRA and out-of-band (including by email to any of the authors or to the email list

Publisher Copyright:
© 2021 Copyright for this paper by its authors.


  • News Recommendation
  • Recommender Systems
  • Research Infrastructure


Dive into the research topics of 'Towards an Experimental News User Community as Infrastructure for Recommendation Research'. Together they form a unique fingerprint.

Cite this