Sony Pixel Power calrec Sony

How Omniverse Wove a Real CEO - and His Toy Counterpart - Together With Stunning Demos at GTC

21/12/2021

It could only happen in NVIDIA Omniverse - the company's virtual world simulation and collaboration platform for 3D workflows.

And it happened during an interview with a virtual toy model of NVIDIA's CEO, Jensen Huang.

What are the greatest one of Toy Jensen's creators asked, stumbling, then stopping before completing his scripted question.

Unfazed, the tiny Toy Jensen paused for a moment, considering the answer carefully.

The greatest are those, Toy Jensen replied, who are kind to others.

Leading-edge computer graphics, physics simulation, a live CEO, and a supporting cast of AI-powered avatars came together to make NVIDIA's GTC keynote - delivered using Omniverse - possible.

Along the way, a little soul got into the mix, too.

The AI-driven comments, added to the keynote as a stinger, provided an unexpected peek at the depth of Omniverse's technology.

Omniverse is the hub in which all the various research domains converge and align and work in unison, says Kevin Margo, a member of NVIDIA's creative team who put the presentation together. Omniverse facilitates the convergence of all of them.

Toy Jensen's ad-lib capped a presentation that seamlessly mixed a real CEO with virtual and real environments as Huang took viewers on a tour of how NVIDIA technologies are weaving AI, graphics and robotics together with humans in real and virtual worlds.

Real CEO, Digital Kitchen While the CEO viewers saw was all real, the environment around him morphed as he spoke to support the story he was telling.

Viewers saw Huang deliver a keynote that seemed to begin, like so many during the global COVID pandemic, in Huang's kitchen.

Then, with a flourish, Huang's kitchen - modeled down to the screws holding its cabinets together - slid away from sight as Huang strolled toward a virtual recreation of Endeavor's gleaming lobby.

One of our goals is to find a way to elevate our keynote events, Margo says. We're always looking for those special moments when we can do something novel and fantastical, and that showcase NVIDIA's latest technological innovations.

It was the start of a visual journey that would take Huang from that lobby to Shannon's, a gathering spot inside Endeavor, through a holodeck, and a data center with stops inside a real robotics lab and the exterior of Endeavor.

Virtual environments such as Huang's kitchen were created by a team using familiar tools supported by Omniverse such as Autodesk Maya and 3ds Max, and Adobe Substance Painter.

Omniverse served to connect them all in real-time - so each team member could see changes made by colleagues using different tools simultaneously, accelerating their work.

That was critical, Margo says.

The virtual and the real came together quickly once live filming began.

A small on-site video team recorded Huang's speech in just four days, starting October 30, in a spare pair of conference rooms at NVIDIA's Silicon Valley headquarters.

Omniverse allowed NVIDIA's team to project the dynamic virtual environments their colleagues had created on a screen behind Huang.

As a result, the light spill onto Huang changed as the scene around him changed, better integrating him into the virtual environment.

And as Huang moved through the scene, or as the camera shifted, the environment changed around Huang.

As the camera moves, the perspective and parallax of the world on the video wall responds accordingly, Mago says.

And because Huang could see the environment projected on the screens around him, he was better able to navigate each scene.

At the Speed of Omniverse All of this accelerated the work of NVIDIA's production team, which had most of what they needed in-camera after each shot rather than adding elaborate digital sets in post-production.

As a result, the video team quickly created a presentation seamlessly blending a real CEO with virtual and real-world settings.

However, Omniverse was more than just a way to speed collaboration between creatives working with real and digital elements hustling to hit a deadline. It also served as the platform that knit the string of demos featured in the keynote together.

To help developers create intelligent, interactive agents with Omniverse that can see, speak, converse on a wide range of subjects and understand naturally spoken intent, Huang announced Omniverse Avatar.

Omniverse brings together a deep stack of technologies - from ray-tracing to recommender systems - that were mixed and matched throughout the keynote to drive a series of stunning demos.

In a demo that swiftly made headlines, Huang showed how Project Tokkio for Omniverse Avatar connects Metropolis computer vision, Riva speech AI, avatar animation and graphics into a real-time conversational AI robot - the Toy Jensen Omniverse Avatar.

The conversation between three of NVIDIA's engineers and a tiny toy model of Huang was more than just a technological tour de force, demonstrating expert, natural Q&A.

It showed how photorealistic modeling of Toy Jensen and his environment - right down to the glint on Toy Jensen's glasses as he moved his head - and NVIDIA's Riva speech synthesis technology powered by the Megatron 530B large language model could support natural, fluid conversations.

To create the demo, NVIDIA's creative team created the digital model in Maya Substance, and Omniverse did the rest.

None of it was manual, you just load up the animation assets and talk to it, he said.

Huang also showed a second demo of Project Tokkio, a customer-service avatar in a restaurant kiosk that was able to see, converse with and understand two customers.

Rather than relying on Megatron, however, this model relied on a model that integrated the restaurant's menu, allowing the avatar to smoothly guide customers through their op
LINK: https://blogs.nvidia.com/blog/2021/12/21/how-omniverse-keynote/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

ASG Advances Joe Marchitto to Western Regional CTO

Share Copy link Facebook X Linkedin Bluesky Email...

01/06/2026

Scripps Stations Go Dark on DirecTV

Share Copy link Facebook X Linkedin Bluesky Email...

01/06/2026

MARSHALL ELECTRONICS POWERS SEAMLESS AV EXPERIENCES WITH...

Marshall Electronics is showcasing a comprehensive lineup of next-generation POV cameras, purpose-built to power today's connected AV environments, at InfoC...

01/06/2026

Adobe Announces Concept to Vector

Adobe Announces Concept to Vector Deepa Subramaniam June 1, 2026 0 Comments One of the biggest frustrations we hear from designers is how difficult it...

01/06/2026

Vampire Feature Night Patrol Graded with DaVinci Resolve Studio

Vampire Feature Night Patrol Graded with DaVinci Resolve Studio Brie Clayton June 1, 2026 0 Comments Colorist shapes dark, gritty tone for horror thri...

01/06/2026

U.S. Broadcasters Ready for Most Complex FIFA World Cup Ever

Share Copy link Facebook X Linkedin Bluesky Email...

01/06/2026

Broadcasters Prepare for Nation's 250th Birthday Bash

Share Copy link Facebook X Linkedin Bluesky Email...

01/06/2026

Broadcasters Reveal What Makes C-Band Alternatives Right for Them

Share Copy link Facebook X Linkedin Bluesky Email...

01/06/2026

IAMT to Offer New Educational Sessions at InfoComm 2026

Share Copy link Facebook X Linkedin Bluesky Email...

01/06/2026

NewsNation Launches New Podcasting Studio and Podcasts

Share Copy link Facebook X Linkedin Bluesky Email...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

31/05/2026

Olivia Prez-Collellmir to Premiere Original Work at Gaud Centennial in Barcelona

Olivia P rez-Collellmir to Premiere Original Work at Gaud Centennial in Barcelona The Berklee graduate and faculty member will debut her choral symphony with...

31/05/2026

Taiwan's Industry Titans Turbocharge World's AI Infrastructure Buildout With NVIDIA

Taiwan is home to more than 500 NVIDIA ecosystem partners. More than 1 million N...

31/05/2026

NVIDIA Factory Operations Blueprint Gives Factories a New AI Brain

As factories move from isolated automation to plant-wide intelligence, manufacturers need AI systems that can connect live machine signals, quality systems, wor...

31/05/2026

NVIDIA AI Cloud Ecosystem Expands Worldwide to Meet Global AI Compute Demand

The NVIDIA AI Cloud ecosystem is accelerating the global buildout of AI factory infrastructure. Partners are expanding capacity to meet growing demand from ente...

30/05/2026

NAB Asks FCC to Shift Regulatory Fee Burden to Big Tech, Broadband

Share Copy link Facebook X Linkedin Bluesky Email...

30/05/2026

NAB Announces 2026 Board Election Results

Share Copy link Facebook X Linkedin Bluesky Email...

30/05/2026

IAB Tech Lab Releases Guidance for Managing AI Crawlers and Bots

Share Copy link Facebook X Linkedin Bluesky Email...

30/05/2026

Zero in on one that says yes (and no)

Zero in on one that says yes (and no) Andy Marken May 29, 2026 0 Comments Hero image courtesy of Deposit Photos For content creators the most difficu...

29/05/2026

InfoComm 2026: NDI Demos NDI 6.3, Hands-On Presentation About Educational Integrations

With InfoComm 2026 just weeks away, NDI is giving attendees plenty of reasons to...

29/05/2026

Bell Media Inks New Long-Term Media Rights Deal for Broadcast, Streaming of the Canadian Football League

Reaffirming a partnership that has defined Canadian sports broadcasting since 19...

29/05/2026

Spring 2026 TV Survey: Vertical Live Matters. The Bigger Story Is Context

Mobile/tablet is No. 2 device for watching TV, suggesting that the sports-production industry needs to take another look at the format Ring Digital's Sprin...

29/05/2026

Germanys Berliner Ensemble Bolsters Backstage Infrastructure With Riedel Stage Systems

Berliner Ensemble, one of Berlin's five major theater companies, has expande...

29/05/2026

InfoComm 2026: Solid State Logic Spotlights TCA Tour, Live V6.2 Software, New SSL Live Trade-in Program

Solid State Logic will showcase its new compact, fly-away TCA Tour audio product...

29/05/2026

Gerald Jerry Pierce, Architect of Modern Digital Cinema, Passes Away at Age 73

Gerald (Jerry) Pierce, a pioneering technologist who helped shape the digital transformation of the motion picture industry, passed away last month on April 12 ...

29/05/2026

CBS Sports Becomes New Home of Barclays Women's Super League Thorough 2029-2030

Paramount+ will be the English-language U.S. home for Barclays Women's Super...

29/05/2026

Calrec Scales ImPulseV to Empower Broadcasters With Greater Choice in Virtualized Workflows

Further strengthening its virtualisation strategy to fully support broadcasters ...

29/05/2026

Switzerlands Canal Alpha Streamlines Playout to Delivery With Harmonic

Swiss broadcaster Canal Alpha has deployed Harmonic's award-winning, software-based XOS Advanced Media Processor to modernize playout operations across cant...

29/05/2026

InfoComm 2026: PTZOptics Showcases Intelligent Video Ecosystem

PTZOptics will showcase a new generation of intelligent video workflows at InfoComm 2026, June 17-19, Las Vegas. Visitors to booth N8227 will see how PTZOptics ...

29/05/2026

Arizona's Family Sports Debuts Direct-to-Consumer Streaming App

Arizona's Family has launched the Arizona's Family Sports (AZFS) streaming app, a new direct-to-consumer destination for live, local sports. The app is ...

29/05/2026

DAZN Brings The Canadian Football Leagues Saturday Night Football to the Masses in New Media Rights Deal

Starting in 2027, DAZN will be the exclusive home of The Canadian Football Leagu...

29/05/2026

Comcast Business Supports Advanced Technology Infrastructure at Levi's Stadium for Fan Experience, Venue Operations

Comcast Business has detailed the advanced network infrastructure it has deploye...

29/05/2026

College Sports-Production Community Gathers in Atlanta for 2026 SVG College Summit

In two-day event, leaders from academia and industry explored solutions to chall...

29/05/2026

TBT Reaches Two-Year Extension With FOX Sports for New-Look $2 Million Tournament

The Basketball Tournament (TBT), now entering their 13th year of competition, ha...

29/05/2026

Roku Expands Premium Subscriptions Experience with FOX One

Roku has launched FOX One as a Premium Subscription on The Roku Channel in the U.S. Roku customers can now subscribe to FOX One using their Roku account for liv...

29/05/2026

CBS Sports UEFA Champions League Today Studio Show Heads Budapest for Final as Transcontinental Popularity Grows

In its sixth year, the broadcaster's coverage has become a global brand and ...

29/05/2026

Ratings Roundup: NBA, NHL Playoffs and Indy 500 Photo Finish Power a Record-Setting Week

Ratings Roundup is a rundown of recent ratings news and is derived from press re...

29/05/2026

Your Summer 2026 Soundtrack? Spotify's Predictions Are In

The days are getting longer, the temperatures are rising, and playlists are filling up for the season. With summer around the corner, Spotify's global edito...

29/05/2026

Akai Pro unveil the MPC Live III Retro

New retro-inspired MPC announced There are few devices that have gained the status held by Akai Pro's MPC range, and in recent years, the company have s...

29/05/2026

Bjooks Beat Gems Kickstarter continues

Save up to 30 on acclaimed titles Following a successful launch at Superbooth 2026, Bjooks have revealed that they will be continuing the Kickstarter campa...

29/05/2026

Genelec update Aural ID

Binaural monitoring application improved Genelec have just released an update that brings some powerful new features to their HRTF-based binaural headphone ...

29/05/2026

IMSTA FESTA 2026

6 June 2026 at SAE Institute, London, UK IMSTA FESTA 2026 is almost upon us, with some of the biggest names in pro-audio set to descend upon SAE Institute i...

29/05/2026

Remembering Gerald Jerry Pierce, Architect of Modern Digital Cinema

Gerald (Jerry) Pierce, a pioneering technologist who helped shape the digital transformation of the motion picture industry, died April 12, 2026, at his home in...

29/05/2026

Roku Unveils Redesigned TV Home Screen

Share Copy link Facebook X Linkedin Bluesky Email...

29/05/2026

Sinclair and AMP Sports to Launch Style of Play' Podcast

Share Copy link Facebook X Linkedin Bluesky Email...

29/05/2026

TV Pharma Ads Are Getting Longer, VAB Says

Share Copy link Facebook X Linkedin Bluesky Email...