Multi-Touch Querying on Data Physicalizations in Immersive AR

Bridger Herman, Maxwell Omdal, Stephanie Zeller, Clara A. Richter, Francesca Samsel, Greg Abram, Daniel F. Keefe

Research output: Contribution to journalArticlepeer-review

7 Scopus citations


Data physicalizations (3D printed terrain models, anatomical scans, or even abstract data) can naturally engage both the visual and haptic senses in ways that are difficult or impossible to do with traditional planar touch screens and even immersive digital displays. Yet, the rigid 3D physicalizations produced with today's most common 3D printers are fundamentally limited for data exploration and querying tasks that require dynamic input (e.g., touch sensing) and output (e.g., animation), functions that are easily handled with digital displays. We introduce a novel style of hybrid virtual + physical visualization designed specifically to support interactive data exploration tasks. Working toward a "best of both worlds"solution, our approach fuses immersive AR, physical 3D data printouts, and touch sensing through the physicalization. We demonstrate that this solution can support three of the most common spatial data querying interactions used in scientific visualization (streamline seeding, dynamic cutting places, and world-in-miniature visualization). Finally, we present quantitative performance data and describe a first application to exploratory visualization of an actively studied supercomputer climate simulation data with feedback from domain scientists.

Original languageEnglish (US)
Article number3488542
JournalProceedings of the ACM on Human-Computer Interaction
Issue numberISS
StatePublished - Nov 2021

Bibliographical note

Funding Information:
The authors wish to thank Josh Rector, Cullen Jackson, and the rest of the Sculpting Vis Collective, as well as the reviewers whose feedback helped improve this paper. This work was supported in part by the National Science Foundation (IIS-1704604 & IIS-1704904). MPAS-Ocean simulations were conducted by Mark Petersen, Phillip Wolfram, Mathew Maltrud and Xylar Asay-Davis as part of the Energy Exascale Earth System Model project, funded by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research.

Publisher Copyright:
© 2021 ACM.


  • data physicalization
  • mixed reality
  • tangible user interfaces


Dive into the research topics of 'Multi-Touch Querying on Data Physicalizations in Immersive AR'. Together they form a unique fingerprint.

Cite this