The next step in our effort is enhancing avatar representation through the Codec Avatar Inclusive Dataset Research Project. Today, creating a high-quality and unique Codec Avatar requires several hours of image and audio capturing in a lab using highly-customized cameras and microphones, followed by weeks spent processing that data to generate the final avatar.
To be able to scale Codec Avatars, it will need to be relatively quick and easy to generate an avatar using your mobile phone, what we call Instant Codec Avatars. We’re working on this — but to succeed, we’ll need a vast and truly diverse data set that represents a broad range of gender expression, body types, hair styles and textures, skin tones, facial features, and more. Everything that makes up “you,” no matter who you are.
And capturing this level of diversity requires more people than we could reasonably ask to visit us in our Pittsburgh lab — so we’re taking this show on the road. While transporting this lab technology isn’t easy, we have found an innovative way to load our equipment on trucks and will begin taking it to Meta campuses, recruiting both internal and external research participants who want to contribute to the data set through this fully-voluntary research project.
As part of a session, participants will do things like mimic expressions, read sentences and converse, and make eye movements — a process that takes a few hours and is similar to what participants in our Pittsburgh lab already experience. The information received won’t be used to create Codec Avatars of each contributor, but will help us get to a future where everyone can create their own Codec Avatar easily, at home, without specialized equipment.
We’re starting with Meta campuses so we can optimize the process, but our goal is to eventually take this mobile setup to communities outside of tech hubs like the San Francisco Bay Area to enhance representation within our data set. Participants are given the option to self-identify using demographic forms, and our hope is that this Codec Avatar Diverse Dataset Research Project will help us accurately represent a more diverse group of people as a result.
Still, there’s a long way to go before people are fully empowered to generate avatars that reflect every little nuance. For instance, we don’t yet have the ability to show avatars with accessories like glasses, which is pretty fundamental. We have much more to do, but we’re excited to take this next step as we work to make Codec Avatars better represent us all.