Meet the Omnivore: Startup in3D Turns Selfies Into Talking, Dancing Avatars With NVIDIA Omniverse

Editor's note: This post is a part of our Meet the Omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.

Imagine taking a selfie and using it to get a moving, talking, customizable 3D avatar of yourself in just seconds.

A new extension for NVIDIA Omniverse, a design collaboration and world simulation platform, enables just that.

Created by developers at software startup in3D, the extension lets people instantly import 3D avatars of themselves into virtual environments using their smartphones. Omniverse Extensions are the core building blocks that let anyone create and extend functions of Omniverse Apps.

The in3D app can now bring people, in their digital forms, into Omniverse. It helps creators build engaging virtual worlds and use these avatars as heroes, actors or spectators in their stories. The app works on any phone with a camera, recreating a user's full geometry and texture based on a video selfie.

The avatars can even be added into 3D worlds with animations and a customizable wardrobe.

In3D is a member of NVIDIA Inception, a free, global program that nurtures cutting-edge startups.

Simple and Scalable Avatar Creation Creating a photorealistic 3D avatar has traditionally taken up to several months, with costs reaching up to tens of thousands of dollars. Photogrammetry, a standard approach to creating 3D references of humans from images, is extremely costly, requires a digital studio and lacks scalability.

With in3D, the process of creating 3D avatars is simple and scalable. The app understands the geometry, texture, depth and various vectors of a person via a mobile scan - and uses this information to replicate lifelike detail and create predictive animations for avatars.

Dmitry Ulyanov, CEO of in3D, which is based in Tel Aviv, Israel, said the app captures even small details with centimeter-grade accuracy and automatically fixes lighting. This allows for precise head geometry from a single selfie, as well as estimation of a user's exact body shape.

For creators building 3D worlds, in3D software can save countless hours, increase productivity and result in substantial cost savings, Ulyanov said.

Manually creating one avatar can take up to months, he added. With in3D's scanning app and software development kit, a user can scan and upload 21,000 people with a single GPU and mobile phone in the same amount of time.

Connecting to Omniverse Ulyanov said that using in3D's extension with NVIDIA Omniverse Avatar Cloud Engine (ACE) opens up many possibilities for avatar building, as users can easily customize imported avatars from in3D to engage and interact with their virtual worlds - in real time and at scale.

In3D uses Universal Scene Description (USD), an open-source, extensible file format, to seamlessly integrate its high-fidelity avatars into Omniverse. All avatar data is contained in a USD file, removing the need for complex shaders or embeddings. And bringing the avatars into Omniverse only requires a simple drag and drop.

Once imported into Omniverse via USD, the avatars can be used in apps like Omniverse Create and Audio2Face. Users have a complete toolset within Omniverse to support holistic content creation, whether animating avatars' bodies with the retargeting tool or crafting their facial expressions with Audio2Face.

To build the Omniverse Extension, in3D used Omniverse Kit and followed the development flow using the VSCode computer program. Being able to put a breakpoint anywhere in the code made VSCode an easy-to-use, convenient, out-of-the-box solution for connecting in3D to Omniverse, Ulyanov said.

The ability to centralize our SDK alongside other software for 3D developers is game changing, he said. With our Omniverse Extension now available, we're looking to expand the base of developers who use our avatars.

Having the ability to upload our SDK and connect it with all the tools that 3D developers use has made in3D a tangible solution to deploy across all 3D development environments, said Sergei Sherman, chief marketing officer at in3D. This was something we wouldn't have been able to achieve on our own in such a short amount of time.

Join In on the Creation Creators and developers across the world can download NVIDIA Omniverse for free, and enterprise teams can use the platform for their 3D projects.

Learn how to connect and create virtual worlds with Omniverse at NVIDIA GTC, the design and simulation conference for the era of AI and the metaverse, running online Sept. 19-22. Registration is free and offers access to dozens of sessions and special events.

Developers can use Omniverse Code to create their own Omniverse Extension for the inaugural #ExtendOmniverse contest by Friday, Sept. 9, at 5 p.m. PT, for a chance to win an NVIDIA RTX GPU. The winners will be announced in the NVIDIA Omniverse User Group at GTC.

Find additional documentation and tutorials in the Omniverse Resource Center, which details how developers like Ulyanov can build custom USD-based applications and extensions for the platform.

Follow NVIDIA Omniverse on Instagram, Medium, Twitter and YouTube for additional resources and inspiration. Check out the Omniverse forums, and join our Discord server and Twitch channel to chat with the community.

LINK:	https://blogs.nvidia.com/blog/2022/08/19/omniverse-developer-in3d/...
	See more stories from nvidia

Most recent headlines