Sony Pixel Power calrec Sony

New NVIDIA Neural Graphics SDKs Make Metaverse Content Creation Available to All


New NVIDIA Neural Graphics SDKs Make Metaverse Content Creation Available to All

Brie Clayton August 11, 2022


A dozen tools and programs - including new releases NeuralVDB and Kaolin Wisp - enable easy, fast 3D content creation for millions of designers and creators.

The creation of 3D objects for building scenes for games, virtual worlds including the metaverse, product design or visual effects is traditionally a meticulous process, where skilled artists balance detail and photorealism against deadlines and budget pressures.

It takes a long time to make something that looks and acts as it would in the physical world. And the problem gets harder when multiple objects and characters need to interact in a virtual world. Simulating physics becomes just as important as simulating light. A robot in a virtual factory, for example, needs to have not only the same look, but also the same weight capacity and braking capability as its physical counterpart.

It's hard. But the opportunities are huge, affecting trillion-dollar industries as varied as transportation, healthcare, telecommunications and entertainment, in addition to product design. Ultimately, more content will be created in the virtual world than in the physical one.

To simplify and shorten this process, NVIDIA today released new research and a broad suite of tools that apply the power of neural graphics to the creation and animation of 3D objects and worlds.

These SDKs - including NeuralVDB, a ground-breaking update to industry standard OpenVDB,and Kaolin Wisp, a Pytorch library establishing a framework for neural fields research - ease the creative process for designers while making it easy for millions of users who aren't design professionals to create 3D content.

Neural graphics is a new field intertwining AI and graphics to create an accelerated graphics pipeline that learns from data. Integrating AI enhances results, helps automate design choices and provides new, yet to be imagined opportunities for artists and creators. Neural graphics will redefine how virtual worlds are created, simulated and experienced by users.

These SDKs and research contribute to each stage of the content creation pipeline, including:

3D Content Creation

Kaolin Wisp - an addition to Kaolin, a PyTorch library enabling faster 3D deep learning research by reducing the time needed to test and implement new techniques from weeks to days. Kaolin Wisp is a research-oriented library for neural fields, establishing a common suite of tools and a framework to accelerate new research in neural fields.

Instant Neural Graphics Primitives - a new approach to capturing the shape of real-world objects, and the inspiration behind NVIDIA Instant NeRF, an inverse rendering model that turns a collection of still images into a digital 3D scene. This technique and associated GitHub code accelerate the process by up to 1,000x.

3D MoMa - a new inverse rendering pipeline that allows users to quickly import a 2D object into a graphics engine to create a 3D object that can be modified with realistic materials, lighting and physics.

GauGAN360 - the next evolution of NVIDIA GauGAN, an AI model that turns rough doodles into photorealistic masterpieces. GauGAN360 generates 8K, 360-degree panoramas that can be ported into Omniverse scenes.

Omniverse Avatar Cloud Engine (ACE) - a new collection of cloud APIs, microservices and tools to create, customize and deploy digital human applications. ACE is built on NVIDIA's Unified Compute Framework, allowing developers to seamlessly integrate core NVIDIA AI technologies into their avatar applications.

Physics and Animation

NeuralVDB - a groundbreaking improvement on OpenVDB, the current industry standard for volumetric data storage. Using machine learning, NeuralVDB introduces compact neural representations, dramatically reducing memory footprint to allow for higher-resolution 3D data.

Omniverse Audio2Face - an AI technology that generates expressive facial animation from a single audio source. It's useful for interactive real-time applications and as a traditional facial animation authoring tool.

ASE: Animation Skills Embedding - an approach enabling physically simulated characters to act in a more responsive and life-like manner in unfamiliar situations. It uses deep learning to teach characters how to respond to new tasks and actions.

TAO Toolkit - a framework to enable users to create an accurate, high-performance pose estimation model, which can evaluate what a person might be doing in a scene using computer vision much more quickly than current methods.


Image Features Eye Tracking - a research model linking the quality of pixel rendering to a user's reaction time. By predicting the best combination of rendering quality, display properties and viewing conditions for the least latency, It will allow for better performance in fast-paced, interactive computer graphics applications such as competitive gaming.

Holographic Glasses for Virtual Reality - a collaboration with Stanford University on a new VR glasses design that delivers full-color 3D holographic images in a groundbreaking 2.5-mm-thick optical stack.

Join NVIDIA at SIGGRAPH to see more of the latest research and technology breakthroughs in graphics, AI and virtual worlds. Check out the latest innovations from NVIDIA Research, and access the full suite of NVIDIA's SDKs, tools and libraries.


Enjoying the news? Sign up for the Creative COW Newsletter! Sign up for the Creative COW newsletter and get weekly updates on industry news, forum highlights, jobs, inspirational tutorials, tips, burning questions, and more! Receive bulletins from the largest, longest-running community dedicated to supporting professionals working in film, video, and audio.

Enter your email address, a first and last name, and let u
See more stories from creativecow

North America Stories


Billie Eilish: Happier Than Ever', The World Tour Relies on URSA Broadcast G2 Cameras

Billie Eilish: Happier Than Ever', The World Tour Relies on URSA Broadcast ...


Major Broadcasters Launch NextGen TV Signals in Roanoke-Lynchburg

ROANOKE-LYNCHBURG, Va. NextGen TV broadcasts have come to the Roanoke-Lynchburg market where WSET-TV, WDBJ, WWCW, WSLS and WZBJ have launched NextGen TV, aka AT...


NAB Show NY Tech Chat to Shine Spotlight on Exhibitors

WASHINGTON NAB Show New York (opens in new tab) attendees will have the opportunity to interact directly with exhibiting companies about their brands and produc...


Interface Communications Deploys Seven Hitachi 1080p Cameras

FLUSHING, N.Y. Mobile TV production services company Interface Communications recently upgraded its camera lineup to support 1080p production of NBA coverage wi...


NBA Launches Reimagined NBA App

NEW YORK The NBA has launched what it is calling a completely reimagined global NBA App that is designed to be an all-in-one destination for NBA fans of every t...


The Price of QD-OLED TVs Could Fall Closer to Regular OLED Quicker Than Expected

If youre hoping to see big price drops in OLED TV prices over the next couple of years, it looks like youll be disappointed. According to a report from Display ...


Cinespace Studios to Develop Major New Studio in Toronto

TORONTO PortsToronto has announced that Cinespace Studios (Cinespace) has entered into a 30-year lease for Marine Terminal 52 to develop a studio facility for T...


NEC Launches Innovation Hub for Open RAN 5G

TOKYO NEC Corporation has launched a North American 5G innovation unit in New Providence, New Jersey that will work to expand its product development and cater ...


Norway's TV 2, Telenor Tap Nevion for New Sports Production System

OSLO Virtualized media production solution provider Nevion has announced that Norwegian broadcaster TV 2 and telecom service provider Telenor will be building a...


OpenAP Adds New Programmatic Capabilities to OpenID

NEW YORK In a move that makes it easier to transact programmatic ad buys across TV and digital, OpenAP has announced new programmatic capabilities for OpenIDSM,...


Making an alien planet using Video Copilot Orb

Making an alien planet using Video Copilot Orb Graham Quince September 27, 2022 0 Comments In this video, I show my preferred way of creating a texture...


Framestore Brings Kevin Bacon's Dreams to Life for EE and Apple's Newest Campaign

Framestore Brings Kevin Bacon's Dreams to Life for EE and Apple's Newest...


CineGear Expo Atlanta Ready to Roll at Trilith Studios

CineGear Expo Atlanta Ready to Roll at Trilith Studios Brie Clayton September 27, 2022 0 Comments October 7 & 8, 2022-Trilith Studios 461 Sandy Creek R...


SEKAI NO OWARI's Smash Hit Song Habit' Graded with DaVinci Resolve Studio

SEKAI NO OWARI's Smash Hit Song Habit' Graded with DaVinci Resolve Stud...


Acrovid releases Intertake 2, a stand alone powerful video processor

Acrovid releases Intertake 2, a stand alone powerful video processor Brie Clayton September 27, 2022 0 Comments Acrovid announces the release of Intert...


Video Virtuoso Sabour Amirazodi Shares AI-Powered Editing Tips This Week In the NVIDIA Studio'

Editor's note: This post is part of the weekly In the NVIDIA Studio series, ...


WRAL-TV's Basaran Shares Knowledge, Experience with Future Leaders

WRAL-TV Assistant News Director Aysu Basaran WRAL-TV Assistant News Director Aysu Basaran is known for her positive mentorship of others, and she recently prov...


September 27, 2022

Lassa virus endemic area may expand dramatically in coming decades New analysis by scientists at Scripps Research and University of Brussels finds that climate ...


What's in a name?

What's the difference between an acronym and an initialization? An acronym is when the first letter of the full name or phrase is used to create a new word...


L3Harris Teams with Austal USA to Deliver Autonomy to USNS Apalachicola

That work culminated into a series of sea trials that started in June 2022 and completed on August 29, 2022. Upon final acceptance by the Navy, USNS Apalachicol...


NEP Norway Taps Mediaproxy for ST 2110 Expansion

MELBOURNE Mediaproxy has announced that the outside broadcast and playout facilities group NEP is using Mediaproxy's LogServer software for SMPTE ST 2110 re...


Allen Media Group Adds 27 Hearst Very Local News Channels to Local Now

ATLANTA Allen Media Group has announced a new partnership with Hearst Television today, adding 27 premium news channels from Hearst's Very Local digital new...


Emerging and Computational Media Expert Ana Herruzo to Deliver Keynote at SMPTE Media Technology Summit

WHITE PLAINS, N.Y. SMPTE has announced that Ana Herruzo, an architect, interacti...


LTN Makes Key Executive Personnel Moves

COLUMBIA, Md. Integrated media technology solutions provider LTN today announced several executive personnel moves, including those affecting the chief marketin...


Steve Haro Joins NAB TV Board of Directors

WASHINGTON Television Board Chair Perry Sook has appointed Steve Haro, executive vice president for government affairs for TelevisaUnivision, to the National As...


NFL Channel Launches on LG TVs

ENGLEWOOD CLIFFS, N.J. LG Electronics USA and the National Football League (NFL) have inked a deal to launch the NFL Channel, the NFLs digital streaming channel...


Bally Sports+ Expands Distribution With Roku Launch

BALTIMORE As the Bally Sport+ direct-to-consumer app celebrated its official national rollout on Sept. 26, Sinclair's Diamond Sports Group also announced th...


NBC Sports Next Acquires Rapid Replay

MINNEAPOLIS, Minn. NBC Sports Next, a division of NBC Sports, has announced it is acquiring Rapid Replay, an innovative provider of video streaming solutions....


HRP Live Teams Up With CP Communications on CrossFit Games Coverage

ST. PETERSBURG, Fla. Production company HRP Live relied on CP Communications for RF, bonded cellular, intercom and streaming services for its coverage in Februa...


Lori Loughlin To Star in Great American Family Movie

Lori Loughlin will star in the Great American Family movie Fall Into Winter, which premieres in January. Her character is aghast when her brother sells his half...


CCIA Slams Canadian Online News-Compensation Bill

The Journalism Competition and Preservation Act (JCPA), which was favorably reported to the full Senate for a vote this week, isn't the only online-targeted...


20/20' Hits 45 Seasons Sept. 23

Newsmag 20/20 marks 45 seasons on ABC when the new season begins September 23. The season starter looks at a Florida husband serving life for murdering his wife...


Viewers Are Paying Attention to Political Advertising: TVision

Viewers might complain about the barrage of political commercials on the way as Election Day approaches, but they seem to pay attention to them....


AMCs The Walking Dead, Foxs Call Me Kat, Harlem Globetrotters Back on TV, and More in Busted Pilot

Broadcasting + Cables Michael Malone and R. Thomas Umstead chat about AMCs The W...


A New Feature to Begin Providing a More Personalized Mobile Games Experience on Netflix

Back to All News A New Feature to Begin Providing a More Personalized Mobile Ga...


Top 10 Week of September 19: DAHMER - Monster' is the Most Viewed Title This Week; Do Revenge' Slays the English Films List

Back to All News Top 10 Week of September 19: DAHMER - Monster' is the Mos...


Viral Video & The Power of Community Media

Small New England town battles back after a viral video exposed its Dunkin' dilemma. Today content creators strive to produce videos that will go viral- or...


Learn the Tips of TV Writing and Development from Director Misha Green

By Stephanie Ornelas During Sundance Collab's recent course on TV writing and development, Sundance Institute's Jandiz Estrada-Cardoso asked director ...


Jon Rambeau Named as New President of L3Harris' Integrated Missions Systems Segment

MELBOURNE, Fla. - L3Harris Technologies (NYSE:LHX) announced today Jon Rambeau j...


NAB Show NY: One Minute With TSL Products

TV Tech talked with Greg Siers, director of sales, Americas, ahead of NAB Show New York. Attendees can find the company on the show floor of the Javits Center i...


TVision: Swing Voters More Engaged with Mid-term Ads

Voters not aligned with either the Democratic or Republican parties are more likely to pay attention to political ads than the partisan counterparts leading up ...


Avid Editing Technologies Used to Produce Multiple Emmy Award Winners and Nominees

BURLINGTON, Mass. Avid is congratulating customers that worked on programming th...


Singer Christiane Karam Raises Her Voice-and Her Fists-for Lebanon

Singer Christiane Karam Raises Her Voice-and Her Fists-for Lebanon How the Voice Department faculty member's journey of healing led her from Beirut to Ber...


Survey: Gen Z Wants More Interactive Live Video from Brands

SANTA CLARA, Calif. New research is highlighting the importance of interactive live video for Gen Z consumers. More than half of respondents (51%) to a survey c...


TV5Monde Launches on Roku

LOS ANGELES TV5Monde has signed a new distribution deal that will bring its premium streaming service TV5Mondeplus to Roku users spanning several territories....


NAB Show NY: One Minute With Pliant Technologies

As NAB Show New York prepares to return to the Javits Center, TV Tech caught up with Gary Rosen, vice president of global sales. Attendees at the show can find ...


SVG Europe @ IBC 2022: The Complete Collection of Video Interviews from the Show Floor

SVG Europe @ IBC 2022: The Complete Collection of Video Interviews from the Show...


SVG, YoSy Media Team Up To Launch HBCU Bridge Initiative

SVG, YoSy Media Team Up To Launch HBCU Bridge Initiative Students at 100+ HBCUs will be connected to internships and careers in sports-TV production and broadca...


Bally Sports+ Launches Widely as In-Market Live Sports Streaming Takes Noted Step Forward

Bally Sports Launches Widely as In-Market Live Sports Streaming Takes Noted Ste...