Sony Pixel Power calrec Sony

How Omniverse Wove a Real CEO - and His Toy Counterpart - Together With Stunning Demos at GTC

21/12/2021

It could only happen in NVIDIA Omniverse - the company's virtual world simulation and collaboration platform for 3D workflows.

And it happened during an interview with a virtual toy model of NVIDIA's CEO, Jensen Huang.

What are the greatest one of Toy Jensen's creators asked, stumbling, then stopping before completing his scripted question.

Unfazed, the tiny Toy Jensen paused for a moment, considering the answer carefully.

The greatest are those, Toy Jensen replied, who are kind to others.

Leading-edge computer graphics, physics simulation, a live CEO, and a supporting cast of AI-powered avatars came together to make NVIDIA's GTC keynote - delivered using Omniverse - possible.

Along the way, a little soul got into the mix, too.

The AI-driven comments, added to the keynote as a stinger, provided an unexpected peek at the depth of Omniverse's technology.

Omniverse is the hub in which all the various research domains converge and align and work in unison, says Kevin Margo, a member of NVIDIA's creative team who put the presentation together. Omniverse facilitates the convergence of all of them.

Toy Jensen's ad-lib capped a presentation that seamlessly mixed a real CEO with virtual and real environments as Huang took viewers on a tour of how NVIDIA technologies are weaving AI, graphics and robotics together with humans in real and virtual worlds.

Real CEO, Digital Kitchen While the CEO viewers saw was all real, the environment around him morphed as he spoke to support the story he was telling.

Viewers saw Huang deliver a keynote that seemed to begin, like so many during the global COVID pandemic, in Huang's kitchen.

Then, with a flourish, Huang's kitchen - modeled down to the screws holding its cabinets together - slid away from sight as Huang strolled toward a virtual recreation of Endeavor's gleaming lobby.

One of our goals is to find a way to elevate our keynote events, Margo says. We're always looking for those special moments when we can do something novel and fantastical, and that showcase NVIDIA's latest technological innovations.

It was the start of a visual journey that would take Huang from that lobby to Shannon's, a gathering spot inside Endeavor, through a holodeck, and a data center with stops inside a real robotics lab and the exterior of Endeavor.

Virtual environments such as Huang's kitchen were created by a team using familiar tools supported by Omniverse such as Autodesk Maya and 3ds Max, and Adobe Substance Painter.

Omniverse served to connect them all in real-time - so each team member could see changes made by colleagues using different tools simultaneously, accelerating their work.

That was critical, Margo says.

The virtual and the real came together quickly once live filming began.

A small on-site video team recorded Huang's speech in just four days, starting October 30, in a spare pair of conference rooms at NVIDIA's Silicon Valley headquarters.

Omniverse allowed NVIDIA's team to project the dynamic virtual environments their colleagues had created on a screen behind Huang.

As a result, the light spill onto Huang changed as the scene around him changed, better integrating him into the virtual environment.

And as Huang moved through the scene, or as the camera shifted, the environment changed around Huang.

As the camera moves, the perspective and parallax of the world on the video wall responds accordingly, Mago says.

And because Huang could see the environment projected on the screens around him, he was better able to navigate each scene.

At the Speed of Omniverse All of this accelerated the work of NVIDIA's production team, which had most of what they needed in-camera after each shot rather than adding elaborate digital sets in post-production.

As a result, the video team quickly created a presentation seamlessly blending a real CEO with virtual and real-world settings.

However, Omniverse was more than just a way to speed collaboration between creatives working with real and digital elements hustling to hit a deadline. It also served as the platform that knit the string of demos featured in the keynote together.

To help developers create intelligent, interactive agents with Omniverse that can see, speak, converse on a wide range of subjects and understand naturally spoken intent, Huang announced Omniverse Avatar.

Omniverse brings together a deep stack of technologies - from ray-tracing to recommender systems - that were mixed and matched throughout the keynote to drive a series of stunning demos.

In a demo that swiftly made headlines, Huang showed how Project Tokkio for Omniverse Avatar connects Metropolis computer vision, Riva speech AI, avatar animation and graphics into a real-time conversational AI robot - the Toy Jensen Omniverse Avatar.

The conversation between three of NVIDIA's engineers and a tiny toy model of Huang was more than just a technological tour de force, demonstrating expert, natural Q&A.

It showed how photorealistic modeling of Toy Jensen and his environment - right down to the glint on Toy Jensen's glasses as he moved his head - and NVIDIA's Riva speech synthesis technology powered by the Megatron 530B large language model could support natural, fluid conversations.

To create the demo, NVIDIA's creative team created the digital model in Maya Substance, and Omniverse did the rest.

None of it was manual, you just load up the animation assets and talk to it, he said.

Huang also showed a second demo of Project Tokkio, a customer-service avatar in a restaurant kiosk that was able to see, converse with and understand two customers.

Rather than relying on Megatron, however, this model relied on a model that integrated the restaurant's menu, allowing the avatar to smoothly guide customers through their op
LINK: https://blogs.nvidia.com/blog/2021/12/21/how-omniverse-keynote/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

12/05/2026

Ad-Hoc News: aconnic AG announces improved fund raising plan to support business and refinance debt

aconnic AG (ISIN: DE000A0LBKW6), Munich, has developed a modified fund raising p...

12/05/2026

CNN Pushes Into Lifestyle Products with CNN Weather' App Launch

Share Copy link Facebook X Linkedin Bluesky Email...

12/05/2026

Marshall Electronics Cameras Elevate Elite Equestrian Com...

Tyrell Corporation, specialists in high-end live sports and entertainment broadcasts, was tasked with delivering compelling broadcast coverage of premier equest...

12/05/2026

IBC2026 registration opens as international media industr...

Registration is now open for IBC2026 as the global media, entertainment and technology community prepares to converge on the RAI Amsterdam from 11 14 September ...

12/05/2026

Ross Video Showcases Unified Production Workflows at Broa...

Ross Video, a global leader in live video production technology, will present its latest innovations and integrated production workflows at BroadcastAsia 2026, ...

12/05/2026

With over 50bn euros in assets under management and recor...

500 selected leaders from around the world across start-ups, corporates, and venture capital. Over 50bn in assets under management among attending investors, a...

12/05/2026

Randy Koeman Joins CVP as Part of Continued European Expa...

CVP, one of Europe's leading suppliers of professional video and broadcast solutions, has appointed Randy Koeman as Sales Manager, Netherlands, marking anot...

12/05/2026

Taiwan Television selects PlayBox Neo Suite multi-channel...

Taiwan Television has expanded its HD content ingest workflow capabilities by investing in the PlayBox Neo Suite - a leading multi-channel multi-server UHD/HD/S...

12/05/2026

Jigsaw24 Appoints Tom Laker and Ashish Chauhan to Support...

Jigsaw24 is strengthening its media team with the appointment of Tom Laker as Client Director and Ashish Chauhan as Project Manager. These strategic hires bring...

12/05/2026

Pebble brings future-ready playout innovation to Broadcas...

Broadcast playout leader showcases hybrid IP infrastructure and workflow automation with Videoland deployment as regional benchmark...

12/05/2026

dzjinius to Showcase All-in-one Production Management Pla...

dzjinius, the all-in-one production management platform for the media and entertainment industry, will showcase its centralised approach to production workflows...

12/05/2026

G&D and CT Square Launch New Joint Venture in India

Share Copy link Facebook X Linkedin Bluesky Email...

12/05/2026

Congress Urged to Protect Live Sports on Broadcast TV

Share Copy link Facebook X Linkedin Bluesky Email...

12/05/2026

Roku to Stream Inaugural Enhanced Games in North America

Share Copy link Facebook X Linkedin Bluesky Email...

12/05/2026

Austrian Broadcaster Selects Lawo, SLG Broadcast for Studio Upgrade

Share Copy link Facebook X Linkedin Bluesky Email...

12/05/2026

NUGEN Audio Debuts Enhanced DialogCheck at MPTS 2026

NUGEN Audio will debut Version 1.1 of DialogCheck, its intelligent dialog intelligibility and compliance tool, at the 2026 Media Production & Technology Show (S...

12/05/2026

Disguise Powers Interactive LED Castle for Laura Pausini

The reactive visuals can adapt to any stage or screen setup and comprise 50 layers of Notch content Today, Disguise announced that its high performance GX 3 m...

12/05/2026

DoPchoice Intros Light-Shaping Accessories for ARRI Omnib...

Snapgrid , Snapbag & Airglow Expand Creative Control for the ARRI Modular LED BarFollowing the launch of the ARRI Omnibar, DoPchoice introduces a dedicated li...

12/05/2026

Macarena Garca and Carlos Cuevas star in 'El Acercamiento de la Mujer Cactus y el Hombre Globo'

Back to All News Macarena Garc a and Carlos Cuevas star in El Acercamiento de l...

12/05/2026

Netflix Unveils Documentary on Norway's 26-Year Journey to This Year's World Cup

Back to All News Netflix Unveils Documentary on Norway's 26-Year Journey to...

12/05/2026

The Netflix Effect

Back to All News The Netflix Effect Ted Sarandos co-CEO Social Impact 12 May 2026 Global Link copied to clipboard Download all assets Ten years ago, Ne...

12/05/2026

News Test

This is my news content...

12/05/2026

NVIDIA and SAP Bring Trust to Specialized Agents

From finance and procurement to supply chain and manufacturing, specialized AI agents are moving into the enterprise systems where business decisions are made, ...

11/05/2026

Ross Production Services 48-Foot HyperMax-1 Truck Quickly in Demand

New production unit features not only wealth of Ross gear but also Sony cameras, Canon lenses, and a Calrec Argo S audio board...

11/05/2026

Solid State Logic Launches TCA Tour Portable Audio Production System

Solid State Logic (SSL) has announced TCA Tour, a portable fly-away audio production system built from System T components for broadcast, touring, and live prod...

11/05/2026

Riedel Communications Named Official Connectivity Integration Provider for Glasgow 2026 Commonwealth Games

Riedel Communications has announced it will serve as Official Connectivity Integ...

11/05/2026

G&D and NETGEAR AV Launch KVM-over-IP Plugin for Automated Network Configuration

Guntermann and Drunck (G&D) and NETGEAR AV have announced a plugin that automates network configuration for G&D KVM-over-IP deployments on NETGEAR AV Line infra...

11/05/2026

The Famous Group and Elite Edge Announce Unified Ownership

The Famous Group (TFG) and Elite Edge, a production studio specializing in live-action production, show opens, set design, and fabrication for sports and live e...

11/05/2026

SVG All-Stars: Odair Auger, Tech Project Manager, NBCUniversal Telemundo Enterprises

The native of S o Paulo, Brazil, will play a crucial role in the broadcaster'...

11/05/2026

Cobalt Digital to Showcase IPMX/ST 2110 Ecosystem at BroadcastAsia 2026

Cobalt Digital will exhibit its IPMX/ST 2110 product lineup at BroadcastAsia 2026 (Stand 5D2-4), anchored by the COBALT blueCORE family of standalone signal pro...

11/05/2026

Interra Systems to Demonstrate QC, Monitoring, and Captioning Solutions at BroadcastAsia 2026

Interra Systems will exhibit at BroadcastAsia 2026 (Stand 5C1-10), demonstrating...

11/05/2026

T-Mobile Deploys 5G and AI Tools Across 2026 PGA Championship

T-Mobile is serving as a technology partner for the 2026 PGA Championship at Aronimink Golf Club in Newtown Square, Pennsylvania, deploying 5G connectivity and ...

11/05/2026

Vizrt Named Official Technical Supplier of Eurovision Song Contest 2026

Vizrt has announced it has been named an Official Technical Supplier of the Eurovision Song Contest 2026, in partnership with host broadcaster ORF. The 70th edi...

11/05/2026

WNBA and AWS Announce Multi-Year Partnership

The WNBA and Amazon Web Services (AWS) have announced a multi-year partnership making AWS the Official Cloud and Cloud AI Partner of the WNBA. AWS is also joini...

11/05/2026

NAB 2026: kronehit Awards Lawo and SLG Broadcast Contract for Full Studio Modernization

Austrian private broadcaster kronehit publicly awarded SLG Broadcast AG and Lawo...

11/05/2026

Behind The Mic: PGA Championship Changing Weekday Announcer Lineup for 2026 Tournament

Behind The Mic provides a roundup of recent news regarding on-air talent, includ...

11/05/2026

Eurovision at 70: The Biggest Hits, Artists, and Trends From the Iconic Song Competition

As Eurovision fans gear up to celebrate the 70th anniversary of the iconic song ...

11/05/2026

Pelo Spotify, artistas brasileiros geraram aproximadamente R$ 2 bilhes em royalties em 2025

Pela primeira vez na hist ria, o Brasil figura entre os oito maiores mercados de...

11/05/2026

Bastl Instruments showcase the Kalimba

New desktop instrument focused on tactile performance Bastl Instruments have announced the Kalimba, a new desktop instrument that combines physical modellin...

11/05/2026

1010music release the blackbox 2

Compact sampling workstation gains new features 1010music have announced the blackbox 2, a new version of their compact standalone sampler designed for DAWl...

11/05/2026

KOMA Elektronik Monoplex Eurorack sequencer

Performance-focused sequencer with microtonal control Berlin-based KOMA Elektronik have announced Monoplex, a new 42HP Eurorack sequencer designed for hands...

11/05/2026

IK Multimedia launch the Tonex One+

Boasts wireless control from mobile devices The latest iteration of IK Multimedia's compact amp and effects modelling pedal expands on the capabilities ...

11/05/2026

Transfigured Orchestra Vol 3: Cinema Brass from Sonora Cinematic

Spans traditional orchestral and experimental sounds Sonora Cinematic have recently released the third instalment in their Transfigured Orchestra series, de...

11/05/2026

SUPERBOOTH 2026 Video Show Reports

Our exhibition coverage Watch all our SUPERBOOTH 2026 video coverage in one place. Check back to this page regularly as we will be updating with more video ...

11/05/2026

You're not meant to look at an Eclipse, but you won't be able to take your eyes off Delta's Eurovision staging

You're not meant to look at an Eclipse, but you won't be able to take yo...