Sony Pixel Power calrec Sony

How Omniverse Wove a Real CEO - and His Toy Counterpart - Together With Stunning Demos at GTC

21/12/2021

It could only happen in NVIDIA Omniverse - the company's virtual world simulation and collaboration platform for 3D workflows.

And it happened during an interview with a virtual toy model of NVIDIA's CEO, Jensen Huang.

What are the greatest one of Toy Jensen's creators asked, stumbling, then stopping before completing his scripted question.

Unfazed, the tiny Toy Jensen paused for a moment, considering the answer carefully.

The greatest are those, Toy Jensen replied, who are kind to others.

Leading-edge computer graphics, physics simulation, a live CEO, and a supporting cast of AI-powered avatars came together to make NVIDIA's GTC keynote - delivered using Omniverse - possible.

Along the way, a little soul got into the mix, too.

The AI-driven comments, added to the keynote as a stinger, provided an unexpected peek at the depth of Omniverse's technology.

Omniverse is the hub in which all the various research domains converge and align and work in unison, says Kevin Margo, a member of NVIDIA's creative team who put the presentation together. Omniverse facilitates the convergence of all of them.

Toy Jensen's ad-lib capped a presentation that seamlessly mixed a real CEO with virtual and real environments as Huang took viewers on a tour of how NVIDIA technologies are weaving AI, graphics and robotics together with humans in real and virtual worlds.

Real CEO, Digital Kitchen While the CEO viewers saw was all real, the environment around him morphed as he spoke to support the story he was telling.

Viewers saw Huang deliver a keynote that seemed to begin, like so many during the global COVID pandemic, in Huang's kitchen.

Then, with a flourish, Huang's kitchen - modeled down to the screws holding its cabinets together - slid away from sight as Huang strolled toward a virtual recreation of Endeavor's gleaming lobby.

One of our goals is to find a way to elevate our keynote events, Margo says. We're always looking for those special moments when we can do something novel and fantastical, and that showcase NVIDIA's latest technological innovations.

It was the start of a visual journey that would take Huang from that lobby to Shannon's, a gathering spot inside Endeavor, through a holodeck, and a data center with stops inside a real robotics lab and the exterior of Endeavor.

Virtual environments such as Huang's kitchen were created by a team using familiar tools supported by Omniverse such as Autodesk Maya and 3ds Max, and Adobe Substance Painter.

Omniverse served to connect them all in real-time - so each team member could see changes made by colleagues using different tools simultaneously, accelerating their work.

That was critical, Margo says.

The virtual and the real came together quickly once live filming began.

A small on-site video team recorded Huang's speech in just four days, starting October 30, in a spare pair of conference rooms at NVIDIA's Silicon Valley headquarters.

Omniverse allowed NVIDIA's team to project the dynamic virtual environments their colleagues had created on a screen behind Huang.

As a result, the light spill onto Huang changed as the scene around him changed, better integrating him into the virtual environment.

And as Huang moved through the scene, or as the camera shifted, the environment changed around Huang.

As the camera moves, the perspective and parallax of the world on the video wall responds accordingly, Mago says.

And because Huang could see the environment projected on the screens around him, he was better able to navigate each scene.

At the Speed of Omniverse All of this accelerated the work of NVIDIA's production team, which had most of what they needed in-camera after each shot rather than adding elaborate digital sets in post-production.

As a result, the video team quickly created a presentation seamlessly blending a real CEO with virtual and real-world settings.

However, Omniverse was more than just a way to speed collaboration between creatives working with real and digital elements hustling to hit a deadline. It also served as the platform that knit the string of demos featured in the keynote together.

To help developers create intelligent, interactive agents with Omniverse that can see, speak, converse on a wide range of subjects and understand naturally spoken intent, Huang announced Omniverse Avatar.

Omniverse brings together a deep stack of technologies - from ray-tracing to recommender systems - that were mixed and matched throughout the keynote to drive a series of stunning demos.

In a demo that swiftly made headlines, Huang showed how Project Tokkio for Omniverse Avatar connects Metropolis computer vision, Riva speech AI, avatar animation and graphics into a real-time conversational AI robot - the Toy Jensen Omniverse Avatar.

The conversation between three of NVIDIA's engineers and a tiny toy model of Huang was more than just a technological tour de force, demonstrating expert, natural Q&A.

It showed how photorealistic modeling of Toy Jensen and his environment - right down to the glint on Toy Jensen's glasses as he moved his head - and NVIDIA's Riva speech synthesis technology powered by the Megatron 530B large language model could support natural, fluid conversations.

To create the demo, NVIDIA's creative team created the digital model in Maya Substance, and Omniverse did the rest.

None of it was manual, you just load up the animation assets and talk to it, he said.

Huang also showed a second demo of Project Tokkio, a customer-service avatar in a restaurant kiosk that was able to see, converse with and understand two customers.

Rather than relying on Megatron, however, this model relied on a model that integrated the restaurant's menu, allowing the avatar to smoothly guide customers through their op
LINK: https://blogs.nvidia.com/blog/2021/12/21/how-omniverse-keynote/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

27/01/2026

Akamai and Yospace Deliver Seamless, Personalised Ad Experiences to Tens of Millions of ABEMA Users

Click for Japanese version Tokyo, Japan - January 27, 2026 - Akamai Technolo...

27/01/2026

Alfred P. Sloan Foundation and Sundance Institute Name Winners of Science-In-Film Initiative's Feature Film Prize and Artist Grants

L-R: Jonathan Cuchacovich, Sonia Kennebeck, Alan Fischer, Daeil Kim, Andrew Sta...

27/01/2026

B-LINE Is Here to Help: Backline Launches 24/7 Music Industry Mental Health & Crisis Support Line in Partnership with Spotify

Today, Spotify is proud to support our partner Backline, an industry-leading men...

27/01/2026

Netn Vega's Genre-Bending Sound Hits a Crossover Moment on Spotify's RADAR

With nearly 29 million monthly listeners and clear momentum on Spotify, Net n Ve...

27/01/2026

25 Ontario Named One of Torontos Best New Buildings

January 14, 2026 We are proud to share that 25 Ontario, First Gulf's commercial project located just two minutes from our head office, has been recognized ...

27/01/2026

Full-Building Lease Secured at Speers Road

January 22, 2026 First Gulf is excited to share that a full-building lease has been secured at 625 Bronte Rd in Oakville, part of Bronte Station Business Park,...

27/01/2026

Another Gold for First Gulf and 351 King Street East

January 23, 2026 First Gulf continues to demonstrate its commitment to high-performance, sustainable real estate with 351 King Street East achieving BOMA BEST ...

27/01/2026

Clear-Com Empowers Reykjavik City Theatre with New Upgrades

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({ content_source:......

27/01/2026

Hubbard Broadcasting Renews Partnership With Nielsen in Albany and Albuquerque-Santa Fe Markets

Nielsen will continue to drive growth and provide local audience measurement in...

27/01/2026

Pebble Adds Paul Nagle-Smith as VP of Customer Fulfillment

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

Study: Top Reason for Cancelling Streaming Service Is Programming

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

U.K.'s Digital TV Group Taps Leslie Mackenzie as Chair

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

Amagi becomes the first cloud-native SaaS company offerin...

Amagi Media Labs Limited, the only cloud-native SaaS provider offering end-to-end solutions across live production, content preparation, distribution, and monet...

27/01/2026

Miri Technologies Launches V410 Live 4K Video Encoder-Dec...

Miri Technologies Inc. today unveiled its V410 live 4K video encoder/decoder for streaming, IP-based production workflows and AV-over-IP distribution. Designed ...

27/01/2026

UML Tsongas Center Upgrades with Ikegami UHK-X600 Cameras

The Paul E. Tsongas Center at UMass Lowell in Massachusetts has chosen Ikegami cameras for incorporation into its broadcast-quality television production facili...

27/01/2026

SipRadius provides simplicity and security for AV content...

SipRadius, specialists in secure, low latency media transport, will show the pro AV community its end-to-end solution for high quality content distribution and ...

27/01/2026

Hiltron to Showcase its Full Range of Satcom Solutions at...

Hiltron will highlight the fruits of its 45-plus years of experience in satellite communications system integration and product development at the annual GovSat...

27/01/2026

Broadcast and pro AV professionals cut costs and single-u...

Klvr will demonstrate how its Charger Pro solution is delivering cost reduction and performance benefits for live entertainment and pro AV markets at ISE 2026 (...

27/01/2026

Amino Showcases Single Player Digital Signage and Enterpr...

Amino, a global media technology leader in enterprise video and digital signage, today announced that its Integrated Systems Europe (ISE) 2026 booth (4N700) wil...

27/01/2026

ANI Expands Strategic Partnership with LiveU

ANI, India's leading news agency and a major content provider across the Asia Pacific region, today announced a significant expansion of its long standing p...

27/01/2026

VisualOn and The Channel Store Demonstrate Up to 65 Perce...

Independent analysis confirms faster encoding, lower CPU usage, and improved perceptual quality compared to CBR and CRF VisualOn, a leader in multimedia playba...

27/01/2026

EVS Officially Launches eShop

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

Miri to Showcase New V410 Video Encoder/Decoder at ISE 2026

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

FCC Settles Two Investigations, Renews Three TV Station Licenses

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

UML Tsongas Center Upgrades to Ikegami UHK-X600 Cameras

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

Australian Defence Force Secures Satellite Communications on SES IS-22

New agreement for uninterrupted UHF connectivity for Australian Defence Force through 2033, With Options Extending to 2041 Luxembourg, January 13, 2026 - Satel...

27/01/2026

SNL UK creative team expands as Pauli Lovejoy, Annie Hardinge and Kevin Fortune join series

Tuesday 27 January 2026 SNL UK creative team expands as Pauli Lovejoy, Annie Ha...

27/01/2026

When It Feels Real, Is It Still a Lie? The Art of Sarah' Premieres February 13

Back to All News When It Feels Real, Is It Still a Lie? The Art of Sarah' ...

27/01/2026

That Night premiering on Netflix March 13

Back to All News That Night premiering on Netflix March 13 Entertainment 27 January 2026 GlobalSpain Link copied to clipboard DOWNLOAD THE KEY ART DOWNLO...

27/01/2026

Netflix Revealed the Future of Mexican Stories Coming in 2026

Back to All News Netflix Revealed the Future of Mexican Stories Coming in 2026 Entertainment 27 January 2026 GlobalMexico Link copied to clipboard Among t...

27/01/2026

What's Next on Netflix from Japan: Expanding the Possibilities of Storytelling in 2026

Back to All News What's Next on Netflix from Japan: Expanding the Possibili...

27/01/2026

FOR-A to show cutting edge technology at FOMEX 2026

AI, graphics and virtual software power new production capabilities...

27/01/2026

RT is Supporting 20 Arts and Cultural Events all over Ireland this February

RT Supporting the Arts is delighted to support a dynamic programme of cultural events taking place across Ireland this February. From visual art, film, music, ...

26/01/2026

New Directed By' Series Explores the Making of Music Videos and Kicks Off With Maisie Peters

Music videos play a huge part in how fans connect with their favorite songs, but...

26/01/2026

NAIDOC launches 2026 theme 50 Years of Deadly, marks major milestone

NAIDOC launches 2026 theme 50 Years of Deadly , marks major milestone 25 January, 2026 Media releases The National NAIDOC Committee has today unveiled its...

26/01/2026

Amagi Lists on Indian Stock Exchanges

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

26/01/2026

FCC Releases Tentative Agenda for Jan. 29 Open Meeting

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

26/01/2026

Dante Turns 20

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

26/01/2026

APTS, PBS Tell FCC Not to Set a Firm Date for ATSC 1.0 Sunset

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

26/01/2026

Advanced Systems Group and Nu Studio Announce Exclusive P...

Advanced Systems Group (ASG) today announced an exclusive partnership with Nu Studio, the company behind the first modular, portable studio designed for immersi...

26/01/2026

OpenDrives Announces New Funding Led by IAG Capital Partn...

OpenDrives, the leader in high-end video data management and workflow solutions, today announced an add-on investment to its previous funding rounds, led by IAG...

26/01/2026

SmallHD Releases Major PageOS Update - Featuring Portrait...

SmallHD today announced a major update to its award-winning PageOS software, introducing Fleet Control, Portrait Mode, expanded Camera Control features, and Dow...

26/01/2026

Capturing the Chaos - Filipic Uses MSE Grip to Go Live a...

Whether rigging a PTZ on a truss, sliding a Blackmagic camera across a mezzanine railing, or capturing the mayhem from inside a drum, Filipic delivers visuals t...

26/01/2026

Riedel and Media Tailor Deliver Unified Broadcast and AV Infrastructure Aboard Royal Caribbean's Star of the Seas Cruise Ship

Wuppertal January 26, 2026 Riedel and Media Tailor Deliver Unified Broadcast a...