Synthetic Data: Simulation & Visual Effects at Scale

ABSTRACT

Gil Elbaz speaks with Tadas Baltrusaitis, who recently released the seminal paper DigiFace 1M: 1 Million Digital Face Images for Face Recognition. Tadas is a true believer in synthetic data and shares his deep knowledge of the subject along with insights on the current state of the field and what CV engineers need to know. Join Gil as they discuss morphable models, multimodal learning, domain gaps, edge cases and more.

TOPICS & TIMESTAMPS

0:00 Introduction

2:06 Getting started in computer science

3:40 Inferring mental states from facial expressions

7:16 Challenges of facial expressions

8:40 Open Face

10:46 MATLAB to Python

13:17 Multimodal Machine Learning

15:52 Multimodals and Synthetic Data

16:54 Morphable Models

19:34 HoloLens

22:07 Skill Sets for CV Engineers

25:25 What is Synthetic Data?

27:07 GANs and Diffusion Models

31:24 Fake it Til You Make It

35:25 Domain Gaps

36:32 Long Tails (Edge Cases)

39:42 Training vs. Testing

41:53 Future of NeRF and Diffusion Models

48:26 Avatars and VR/AR

50:39 Advice for Next Generation CV Engineers

51:58 Season One Wrap-Up

LINKS & RESOURCES

Tadas Baltrusaitis, Principal Scientist at Microsoft, Mixed Reality and AI lab in Cambridge, UK

LinkedIn  

Github

Google Scholar

Microsoft

Fake it Til You Make It

Video 

Github

Digiface 1M

A 3D Morphable Eye Region Model for Gaze Estimation
Hololens

Multimodal Machine Learning: A Survey and Taxonomy 

3d face reconstruction with dense landmarks

Open Face

Open Face 2.0

Dr. Rana el Kaliouby

Dr. Louis-Philippe Morency

Peter Robinson

Jamie Shotton

Errol Wood

Affectiva

GUEST BIO

Tadas Baltrusaitis is a principal scientist working in the Microsoft Mixed Reality and AI lab in Cambridge UK where he leads the human synthetics team. He recently co-authored the groundbreaking paper DigiFace 1M, a data set of 1 million synthetic images for facial recognition. Tadas is also the co-author of Fake It Till You Make It: Face Analysis in the Wild Using Synthetic Data Alone, among other outstanding papers. Tadas has pushed forward the field of simulated synthetic data in a significant way throughout his career. His PhD research focused on automatic facial expression analysis in  difficult real world settings and he was a postdoctoral associate at Carnegie Mellon University where his primary research lay in automatic understanding of human behavior, expressions and mental states using computer vision.

ABOUT THE HOST

I’m Gil Elbaz, co-founder and CTO of Datagen. In this podcast, I speak with interesting computer vision thinkers and practitioners. I ask the big questions that touch on the issues and challenges that ML and CV engineers deal with every day. On the way, I hope you uncover a new subject or gain a different perspective, as well as enjoying engaging conversation. It’s about much more than the technical processes – it’s about people, journeys, and ideas. Turn up the volume, insights inside.

I’m Gil Elbaz, Co-founder and CTO of Datagen. In this podcast, I speak with interesting computer vision thinkers and practitioners. I ask the big questions that touch on the issues and challenges that ML and CV engineers deal with every day. On the way, I hope you uncover a new subject or gain a different perspective, as well as enjoying engaging conversation. It’s about much more than the technical processes – it’s about people, journeys, and ideas. Turn up the volume, insights inside.
StartupHub.ai

StartupHub.ai, the hub of the AI startup ecosystem.

Add or Claim Your Profile

Submit the form below to add or claim an existing profile. Within 24 hours, we'll review your request and connect you to your existing profile for full editing.

Pro Lite Subscription Required

Unlock all search results

With a Pro Lite subscription, you can enjoy unlimited profile viewing. Power your innovation efforts, uninterrupted.

$20/month

cancel anytime

Pro Lite includes: