Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing

Yang Katie Zhao, Shang Wu, Jingqun Zhang, Sixu Li, Chaojian Li, Yingyan Celine Lin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Instant on-device Neural Radiance Fields (NeRFs) are in growing demand for unleashing the promise of immersive AR/VR experiences, but are still limited by their prohibitive training time. Our profiling analysis reveals a memory-bound inefficiency in NeRF training. To tackle this inefficiency, near-memory processing (NMP) promises to be an effective solution, but also faces challenges due to the unique workloads of NeRFs, including the random hash table lookup, random point processing sequence, and heterogeneous bottleneck steps. Therefore, we propose the first NMP framework, Instant-NeRF, dedicated to enabling instant on-device NeRF training. Experiments on eight datasets consistently validate the effectiveness of Instant-NeRF.

Original languageEnglish (US)
Title of host publication2023 60th ACM/IEEE Design Automation Conference, DAC 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350323481
DOIs
StatePublished - 2023
Externally publishedYes
Event60th ACM/IEEE Design Automation Conference, DAC 2023 - San Francisco, United States
Duration: Jul 9 2023Jul 13 2023

Publication series

NameProceedings - Design Automation Conference
Volume2023-July
ISSN (Print)0738-100X

Conference

Conference60th ACM/IEEE Design Automation Conference, DAC 2023
Country/TerritoryUnited States
CitySan Francisco
Period7/9/237/13/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

  • Algorithm-Accelerator Co-Design
  • Near-Memory Processing
  • Neural Radiance Field
  • On-Device Training

Fingerprint

Dive into the research topics of 'Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing'. Together they form a unique fingerprint.

Cite this