Blendshapes from commodity RGB-D sensors

Dan Casas, Oleg Alexander, Andrew W. Feng, Graham Fyffe, Ryosuke Ichikari, Paul Debevec, Rhuizhe Wang, Evan Suma, Ari Shapiro

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Creating and animating a realistic 3D human face is an important task in computer graphics. The capability of capturing the 3D face of a human subject and reanimate it quickly will find many applications in games, training simulations, and interactive 3D graphics. We demonstrate a system to capture photorealistic 3D faces and generate the blendshape models automatically using only a single commodity RGB-D sensor. Our method can rapidly generate a set of expressive facial poses from a single depth sensor, such as a Microsoft Kinect version 1, and requires no artistic expertise in order to process those scans. The system takes only a matter of seconds to capture and produce a 3D facial pose and only requires a few minutes of processing time to transform it into a blendshape-compatible model. Our main contributions include an end-to-end pipeline for capturing and generating face blendshape models automatically, and a registration method that solves dense correspondences between two face scans by utilizing facial landmarks detection and optical flows. We demonstrate the effectiveness of the proposed method by capturing different human subjects and puppeteering their 3D faces in an animation system with real-time facial performance retargeting.

Original languageEnglish (US)
Title of host publicationACM SIGGRAPH 2015 Talks, SIGGRAPH 2015
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450336369
DOIs
StatePublished - Jul 31 2015
Externally publishedYes
EventInternational Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2015 - Los Angeles, United States
Duration: Aug 9 2015Aug 13 2015

Publication series

NameACM SIGGRAPH 2015 Talks, SIGGRAPH 2015

Other

OtherInternational Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2015
CountryUnited States
CityLos Angeles
Period8/9/158/13/15

Keywords

  • Blendshapes
  • Depth sensors
  • Face animation
  • RGB-D

Fingerprint Dive into the research topics of 'Blendshapes from commodity RGB-D sensors'. Together they form a unique fingerprint.

Cite this