i-FlatCam: A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR

Yang Zhao; Ziyun Li; Yonggan Fu; Yongan Zhang; Chaojian Li; Cheng Wan; Haoran You; Shang Wu; Xu Ouyang; Vivek Boominathan; Ashok Veeraraghavan; Yingyan Lin

doi:10.1109/VLSITechnologyandCir46769.2022.9830458

i-FlatCam: A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR

Yang Zhao
, Ziyun Li
, Yonggan Fu
, Yongan Zhang
, Chaojian Li
, Cheng Wan
, Haoran You
, Shang Wu
, Xu Ouyang
, Vivek Boominathan
, Ashok Veeraraghavan
, Yingyan Lin

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Scopus citations

Abstract

We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 J/Frame, and 6.7mm×8.9mm×1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.

Original language	English (US)
Title of host publication	2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	108-109
Number of pages	2
ISBN (Electronic)	9781665497725
DOIs	https://doi.org/10.1109/VLSITechnologyandCir46769.2022.9830458
State	Published - 2022
Externally published	Yes
Event	2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022 - Honolulu, United States Duration: Jun 12 2022 → Jun 17 2022

Publication series

Name	Digest of Technical Papers - Symposium on VLSI Technology
Volume	2022-June
ISSN (Print)	0743-1562

Conference

Conference	2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022
Country/Territory	United States
City	Honolulu
Period	6/12/22 → 6/17/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Publisher link

10.1109/VLSITechnologyandCir46769.2022.9830458

Find It

Find It at U of M Libraries

Cite this

Zhao, Y., Li, Z., Fu, Y., Zhang, Y., Li, C., Wan, C., You, H., Wu, S., Ouyang, X., Boominathan, V., Veeraraghavan, A., & Lin, Y. (2022). i-FlatCam: A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR. In 2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022 (pp. 108-109). (Digest of Technical Papers - Symposium on VLSI Technology; Vol. 2022-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/VLSITechnologyandCir46769.2022.9830458

i-FlatCam: A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR. / Zhao, Yang; Li, Ziyun; Fu, Yonggan et al.
2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 108-109 (Digest of Technical Papers - Symposium on VLSI Technology; Vol. 2022-June).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Zhao, Y, Li, Z, Fu, Y, Zhang, Y, Li, C, Wan, C, You, H, Wu, S, Ouyang, X, Boominathan, V, Veeraraghavan, A & Lin, Y 2022, i-FlatCam: A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR. in 2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022. Digest of Technical Papers - Symposium on VLSI Technology, vol. 2022-June, Institute of Electrical and Electronics Engineers Inc., pp. 108-109, 2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022, Honolulu, United States, 6/12/22. https://doi.org/10.1109/VLSITechnologyandCir46769.2022.9830458

Zhao Y, Li Z, Fu Y, Zhang Y, Li C, Wan C et al. i-FlatCam: A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR. In 2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 108-109. (Digest of Technical Papers - Symposium on VLSI Technology). doi: 10.1109/VLSITechnologyandCir46769.2022.9830458

Zhao, Yang ; Li, Ziyun ; Fu, Yonggan et al. / i-FlatCam : A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR. 2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 108-109 (Digest of Technical Papers - Symposium on VLSI Technology).

@inproceedings{a143dbf47540407cac7593fc79af7293,

title = "i-FlatCam: A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR",

abstract = "We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 J/Frame, and 6.7mm×8.9mm×1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.",

author = "Yang Zhao and Ziyun Li and Yonggan Fu and Yongan Zhang and Chaojian Li and Cheng Wan and Haoran You and Shang Wu and Xu Ouyang and Vivek Boominathan and Ashok Veeraraghavan and Yingyan Lin",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022 ; Conference date: 12-06-2022 Through 17-06-2022",

year = "2022",

doi = "10.1109/VLSITechnologyandCir46769.2022.9830458",

language = "English (US)",

series = "Digest of Technical Papers - Symposium on VLSI Technology",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "108--109",

booktitle = "2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022",

}

TY - GEN

T1 - i-FlatCam

T2 - 2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022

AU - Zhao, Yang

AU - Li, Ziyun

AU - Fu, Yonggan

AU - Zhang, Yongan

AU - Li, Chaojian

AU - Wan, Cheng

AU - You, Haoran

AU - Wu, Shang

AU - Ouyang, Xu

AU - Boominathan, Vivek

AU - Veeraraghavan, Ashok

AU - Lin, Yingyan

PY - 2022

Y1 - 2022

N2 - We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 J/Frame, and 6.7mm×8.9mm×1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.

AB - We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 J/Frame, and 6.7mm×8.9mm×1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.

UR - https://www.scopus.com/pages/publications/85135211200

UR - https://www.scopus.com/pages/publications/85135211200#tab=citedBy

U2 - 10.1109/VLSITechnologyandCir46769.2022.9830458

DO - 10.1109/VLSITechnologyandCir46769.2022.9830458

M3 - Conference contribution

AN - SCOPUS:85135211200

T3 - Digest of Technical Papers - Symposium on VLSI Technology

SP - 108

EP - 109

BT - 2022 IEEE Symposium on VLSI Technology and Circuits, VLSI Technology and Circuits 2022

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 12 June 2022 through 17 June 2022

ER -

i-FlatCam: A 253 FPS, 91.49 J/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR

Abstract

Publication series

Conference

Bibliographical note

Publisher link

Find It

Other files and links

Fingerprint

Cite this