Facebook's Prototype VR Face Tracking Got Even Better
UploadVR1 HN recommenders · 2020–2022
70K views · demo · dated · Uploaded 2019-06-28
UploadVR covers a Facebook Reality Labs prototype that tracks facial expressions using cameras mounted in a VR headset. The updated version captures subtle micro-expressions for realistic avatars.
Pointed to in metaverse and VR threads as proof that realtime photorealistic face tracking from headset cameras is coming.
48:10Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning
Google TechTalks
3:36:39QuakeCon 2012 - John Carmack Keynote
QuakeCon Official
41:18"Voice Driven Development: Who needs a keyboard anyway?" by Emily Shea
Strange Loop Conference
30:09Andrej Karpathy - AI for Full-Self Driving at Tesla
Matroid
10:36RiftSketch: Live coding in VR with the Oculus Rift, Firefox WebVR, JavaScript and Three.js
Brian Peiris
2:35:53Tensorflow and deep learning - without a PhD by Martin Görner
Devoxx
1:49:22The Future is a Dead Mall - Decentraland and the Metaverse
Folding Ideas
15:12JPEG DCT, Discrete Cosine Transform (JPEG Pt2)- Computerphile
Computerphile
5 comments from Hacker News
It will ( [VIDEO] ), and VR already conveys MUCH more of the micro-interactions of being with people that we are unknowingly blind to in Zoom meetings, which is one reason Zoom meetings are so draining.
I don't think they don't understand it, they understand incremental improvements. Whatever you're talking about is coming: [VIDEO]
> The author mentions lip syncing and hand tracking. That goes a long way, but it doesn't recreate the video experience to me. The oculus team put together a seriously jawdropping demo of facial tracking using a few cameras in a headset, combined with a lot of AI. The prototype was from a couple years ago. I can't wait until it makes it into commercial headsets. I wonder what the hold up is? Well worth a look if you haven't seen it: [VIDEO]
There are algos that reconstruct photorealistic 3D facial expressions from headset cameras in realtime. [VIDEO]
Title should be amended to something like 'Face-to-face, rather than shoulder-to-shoulder, enhances learning, innovation.' For a high quality face-to-face remote experience, I expect we'll start to see really impressive consumer ready technology announced soon. See what Facebook have been doing with cameras mounted in VR headsets [1] or Google's Project Starline [2]. That said, when we're doing this much transformation and synthesis of the raw data to achieve high-fidelity telepresence, are we going to have issues trusting it? [1] [VIDEO] [2] [LINK]