Automatic generation of dynamically relightable virtual objects with consumer-grade depth cameras

Chih Fan Chen, Evan Suma Rosenberg

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

This research demo showcases the results of novel approach for estimating the illumination and reflectance properties of virtual objects captured using consumer-grade RGB-D cameras. This method is implemented within a fully automatic content creation pipeline that generates photorealistic objects in real-time virtual reality scenes with dynamic lighting. The geometry of the target object is first reconstructed from depth images captured using a handheld camera. To get nearly drift-free texture maps of the virtual object, a set of selected images from the original color stream is used for camera pose optimization. Our approach further separates these images into diffuse (view-independent) and specular (view-dependent) components using low-rank decomposition. The lighting conditions during capture and reflectance properties of the virtual object are subsequently estimated from the specular maps. By combining these parameters with the diffuse texture, reconstructed objects are then rendered in a real-time virtual reality demo that plausibly replicates the real world illumination and showcases dynamic lighting with varying direction, intensity, and color.

Original languageEnglish (US)
Title of host publication26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1295-1296
Number of pages2
ISBN (Electronic)9781728113777
DOIs
StatePublished - Mar 2019
Event26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Osaka, Japan
Duration: Mar 23 2019Mar 27 2019

Publication series

Name26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings

Conference

Conference26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019
CountryJapan
CityOsaka
Period3/23/193/27/19

Keywords

  • Computer graphics
  • Computer graphics
  • Computing methodologies
  • Computing methodologies
  • Graphics systems and interfaces
  • Reflectance modeling
  • Rendering
  • Virtual reality

Fingerprint Dive into the research topics of 'Automatic generation of dynamically relightable virtual objects with consumer-grade depth cameras'. Together they form a unique fingerprint.

  • Cite this

    Chen, C. F., & Rosenberg, E. S. (2019). Automatic generation of dynamically relightable virtual objects with consumer-grade depth cameras. In 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings (pp. 1295-1296). [8798037] (26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/VR.2019.8798037