Latest NVIDIA Graphics Research Advances Generative AI's Next Frontier
02/05/2023
Around 20 NVIDIA Research papers advancing generative AI and neural graphics - including collaborations with over a dozen universities in the U.S., Europe and Israel - are headed to SIGGRAPH 2023, the premier computer graphics conference, taking place Aug. 6-10 in Los Angeles.
The papers include generative AI models that turn text into personalized images; inverse rendering tools that transform still images into 3D objects; neural physics models that use AI to simulate complex 3D elements with stunning realism; and neural rendering models that unlock new capabilities for generating real-time, AI-powered visual details.
Innovations by NVIDIA researchers are regularly shared with developers on GitHub and incorporated into products, including the NVIDIA Omniverse platform for building and operating metaverse applications and NVIDIA Picasso, a recently announced foundry for custom generative AI models for visual design. Years of NVIDIA graphics research helped bring film-style rendering to games, like the recently released Cyberpunk 2077 Ray Tracing: Overdrive Mode, the world's first path-traced AAA title.
The research advancements presented this year at SIGGRAPH will help developers and enterprises rapidly generate synthetic data to populate virtual worlds for robotics and autonomous vehicle training. They'll also enable creators in art, architecture, graphic design, game development and film to more quickly produce high-quality visuals for storyboarding, previsualization and even production.
AI With a Personal Touch: Customized Text-to-Image Models Generative AI models that transform text into images are powerful tools to create concept art or storyboards for films, video games and 3D virtual worlds. Text-to-image AI tools can turn a prompt like children's toys into nearly infinite visuals a creator can use for inspiration - generating images of stuffed animals, blocks or puzzles.
However, artists may have a particular subject in mind. A creative director for a toy brand, for example, could be planning an ad campaign around a new teddy bear and want to visualize the toy in different situations, such as a teddy bear tea party. To enable this level of specificity in the output of a generative AI model, researchers from Tel Aviv University and NVIDIA have two SIGGRAPH papers that enable users to provide image examples that the model quickly learns from.
One paper describes a technique that needs a single example image to customize its output, accelerating the personalization process from minutes to roughly 11 seconds on a single NVIDIA A100 Tensor Core GPU, more than 60x faster than previous personalization approaches.
A second paper introduces a highly compact model called Perfusion, which takes a handful of concept images to allow users to combine multiple personalized elements - such as a specific teddy bear and teapot - into a single AI-generated visual:
Serving in 3D: Advances in Inverse Rendering and Character Creation Once a creator comes up with concept art for a virtual world, the next step is to render the environment and populate it with 3D objects and characters. NVIDIA Research is inventing AI techniques to accelerate this time-consuming process by automatically transforming 2D images and videos into 3D representations that creators can import into graphics applications for further editing.
A third paper created with researchers at the University of California, San Diego, discusses tech that can generate and render a photorealistic 3D head-and-shoulders model based on a single 2D portrait - a major breakthrough that makes 3D avatar creation and 3D video conferencing accessible with AI. The method runs in real time on a consumer desktop, and can generate a photorealistic or stylized 3D telepresence using only conventional webcams or smartphone cameras.
A fourth project, a collaboration with Stanford University, brings lifelike motion to 3D characters. The researchers created an AI system that can learn a range of tennis skills from 2D video recordings of real tennis matches and apply this motion to 3D characters. The simulated tennis players can accurately hit the ball to target positions on a virtual court, and even play extended rallies with other characters.
Beyond the test case of tennis, this SIGGRAPH paper addresses the difficult challenge of producing 3D characters that can perform diverse skills with realistic movement - without the use of expensive motion-capture data.
https://blogs.nvidia.com/wp-content/uploads/2023/05/Tennis.mp4
Not a Hair Out of Place: Neural Physics Enables Realistic Simulations Once a 3D character is generated, artists can layer in realistic details such as hair - a complex, computationally expensive challenge for animators.
Humans have an average of 100,000 hairs on their heads, with each reacting dynamically to an individual's motion and the surrounding environment. Traditionally, creators have used physics formulas to calculate hair movement, simplifying or approximating its motion based on the resources available. That's why virtual characters in a big-budget film sport much more detailed heads of hair than real-time video game avatars.
A fifth paper showcases a method that can simulate tens of thousands of hairs in high resolution and in real time using neural physics, an AI technique that teaches a neural network to predict how an object would move in the real world.
The team's novel approach for accurate simulation of full-scale hair is specifically optimized for modern GPUs. It offers significant performance leaps compared to state-of-the-art, CPU-based solvers, reducing simulation times from multiple days
LINK: | https://blogs.nvidia.com/blog/2023/05/02/graphics-research-advances-ge... |
See more stories from nvidia |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
09/05/2024
Join Calrec at MPTS 2024
Join Calrec at MPTS 2024 | May 15 -16 | Booth A40 | Olympia, London We're looking forward to meeting up with customers and partners at this year's Media...
09/05/2024
NHL Draft to be First Live Televised Event from Vegas' Sphere
The 2024 Upper Deck NHL Draft will be held at Sphere in Las Vegas on Friday, June 28 and Saturday, June 29, marking the first time the event will be held in Las...
09/05/2024
WBD Nears 100M Streaming Subs, Streaming Profits Up
NEW YORK In its Q1 2024 earnings report, Warner Bros. Discovery continued to see growth in its streaming operations, adding 2 million streaming subs in Q1 2024 ...
09/05/2024
Custom Consoles Unveils M-Desk Refinements
LEIGHTON BUZZARD, U.K. Custom Consoles has announced refinements to its M-Desk Technical HA heavy duty height-adjustable desk workstation....
09/05/2024
Veritone Rolls Out AI Consulting, Services Group
DENVER Veritone has launched a consulting and services group focused on bridging the marketplace gap it says exists for seamless artificial intelligence (AI) im...
09/05/2024
How sound (and the technology behind it) can set the tone and enhance emotions in film and TV
Sefi Carmel, CEO and founder of SphereTrax, talks about the importance of the so...
09/05/2024
Yale University's Center for Collaborative Arts and Media (CCAM) Cultivates Creativity with Blackmagic Design
Yale University's Center for Collaborative Arts and Media (CCAM) Cultivates ...
09/05/2024
Blender: Awesome Rack Focus Rig
Blender: Awesome Rack Focus Rig Simon Ubsdell May 9, 2024 0 Comments In which we build a focusing rig in the Blender compositor to enable selective fo...
09/05/2024
Peacock Orders The Office' Offshoot
Peacock has picked up a series from Greg Daniels, who developed The Office for NBC, that is set in the same universe, according to the network, as the NBC hit...
09/05/2024
Sinclair Net Income Drops To $23 Million in First Quarter
Sinclair reported lower net income in the first quarter as distribution revenues rose and ad revenue excluding political spending dipped....
09/05/2024
Vizio Reports $12.1 Million Loss for First Quarter
Vizio Holding Corp., which has agreed to be acquired by Walmart, reported a loss in the first quarter despite growth in its advertising and data businesses....
09/05/2024
Kathleen Keefe, Broadcasting Exec, Dies at 68
Kathleen Keefe, who served as VP of sales for both Hearst Television and Post-Newsweek Stations, died peacefully in her sleep at her home in Roxbury, Connecticu...
09/05/2024
Future Today Adds Movies From Warner Bros Discovery to Fawesome
Streaming company Future Today said it made a deal with Warner Bros. Discovery that will bring 46 movies to Future Today's Fawesome channel....
09/05/2024
Warner Bros. Discovery Reports $966 Million First-Quarter Loss
Despite reporting a profit for its direct-to-consumer business, Warner Bros. Discovery reported a big loss as it took huge write-downs in the first quarter....
09/05/2024
Nexstar Reports Higher Net Income in First Quarter
Nexstar Media Group reported higher first-quarter earnings and said that it cut losses at The CW by $50 million year-over-year....
09/05/2024
Fubo Grabs Viewer Attention, According to TVision Study
Fubo, the sports-focused streaming service, scored high in getting its viewers attention, according to a study conducted by measurement and analytics company TV...
09/05/2024
SailGP Expanding LiveLineFX Graphics Platform to Other Sports
LONDON SailGP, which sponsors high-speed boat racing events worldwide says it is developing its LiveLineFX sports graphics platform for use in other sports....
09/05/2024
Disney, WBD to Offer Streaming Bundle Starting this Summer
BURBANK, Calif. Starting this summer, U.S. consumers will be able to purchase a new streaming bundle offered by Disney and Warner Bros. Discovery that includes ...
09/05/2024
Sony and Apollo plan to break up Paramount'
Companies are expected to sell off Paramounts linear channels as well as close Paramount Plus By Jenny Priestley Published: May 9, 2024 Companies are expe...
09/05/2024
Paramount Global Selects Amdocs Vubiquity to Provide Mana...
Amdocs (NASDAQ:DOX), a leading provider of software and services to communications and media companies, today announced that Vubiquity, an Amdocs company, has b...
09/05/2024
CueScript Awarded Best of Show for CueFlip at NAB 2024
CueScript, the leading developer of professional teleprompting solutions, has announced that its CueFlip screen-to-screen folding field prompter has won in this...
09/05/2024
Mother's Day Ideas from CBC
Capitol Broadcasting has a myriad of ways to celebrate Mom! In honor of Mother's Day, here are a few ideas CBC divisions have to help you honor, or remember...
09/05/2024
Scripps Appoints New President and CEO of the Scripps Howard Fund
CINCINNATI Meredith Delaney has been appointed president and CEO of the Scripps Howard Fund, the philanthropic arm of The E.W. Scripps Company....
09/05/2024
Walmart Launches the Onn. 4K Streaming Device
Following leaks earlier this month of Walmart's plans to launch a new 4K streaming box, Walmart has officially unveiled the Onn. 4K Pro, a hybrid streaming ...
09/05/2024
DirecTV Adds More CW Stations
DirecTV continues to add CW affiliates to its streaming line-up, with the addition of four stations owned by Gray Television....
09/05/2024
Takeshi Kitano's film KUBI graded with DaVinci Resolve
Takeshi Kitano's film KUBI graded with DaVinci Resolve Brie Clayton May 8, 2024 0 Comments Blackmagic Design today announced that DaVinci Resolve ...
09/05/2024
zweiB updates to latest version of R&S CLIPSTER
zweiB updates to latest version of R&S CLIPSTER Brie Clayton May 8, 2024 0 Comments Specialist in deliverables for cinema, events and trade shows Ro...
09/05/2024
Simplify Your Filmmaking with Portable Lights
Simplify Your Filmmaking with Portable Lights Sean Alami May 8, 2024 0 Comments Discover the sleek, mobile Zhiyun lights that enhance cinematic scenes...
09/05/2024
The Benefits of using an Audio Interface
The Benefits of using an Audio Interface Colin Smith May 8, 2024 0 Comments This tutorial takes you through the hardware and software benefits of usin...
09/05/2024
Mother's Day Idea from CBC
Capitol Broadcasting has a myriad of ways to celebrate Mom! In honor of Mother's Day, here are a few ideas CBC divisions have to help you honor, or remember...
09/05/2024
Lachlan Murdoch Sees Progress Toward Sports Streaming Product
Fox CEO Lachlan Murdoch said his company, The Walt Disney Co. and Warner Bros. Discovery are in a two-minute drill to get their streaming sports joint venture t...
09/05/2024
Dale Earnhardt Jr. Joins Prime Video's NASCAR Team
NASCAR legend Dale Earnhardt Jr. is joining the Prime Video broadcast booth when the streaming network begins NASCAR Cup Series coverage next year. Earnhardt Jr...
09/05/2024
Nexstar Pitches Sports on The CW, Extra Reach Via Stations (Upfronts)
If upfront advertisers are looking for reach, Nexstar Media Group believes its base of local stations gives it a unique way to achieve that goal....
09/05/2024
Pluto TV Debuts NBCUniversal Local Streaming News Channels
NBCUniversal Local's 15 NBC and Telemundo local and regional streaming news channels begin launching on Pluto TV this month, starting May 8 with five channe...
09/05/2024
Don't Tell Mom the Babysitter's Dead' Debuts on BET Plus May 16
BET Plus said it has set May 16 as the premiere date for its original movie Don't Tell Mom the Babysitter's Dead....
09/05/2024
Disney's Rita Ferro Is Banking on Sports, Streaming, Tech (Upfronts)
Heading into the upfront, the advertising market is noticeably healthier than it was a year ago, Rita Ferro, president of global advertising sales at The Walt D...
09/05/2024
Exhibition Schedule 2024
It's exhibition season again, and dB will be at several major shows, eager to meet as many visitors as possible. At The Media Production & Technology Show ...
09/05/2024
Prison Journalism: Challenges of foreign citizens' imprisonment in South Africa
Prison Journalism: Challenges of foreign citizens' imprisonment in South Afr...
09/05/2024
Prison Journalism: Pray for liberation and healing
Prison Journalism: Pray for liberation and healingThabo Mthembu was incarcerated in Pollsmoor Prison from 2014 to 2019. Read Thabo's story by Thabo Mthemb...
09/05/2024
Taylor Swift's European Eras tour kicks off in Paris TONIGHT
Taylor Swift's European Eras tour kicks off in Paris TONIGHTThe highest-grossing tour in history has finally arrived in Europe as Taylor Swift kicks off her...
09/05/2024
Tonight on Scandal: MaHilda and Phakamile are forced into navigating a tense truce
Tonight on Scandal: MaHilda and Phakamile are forced into navigating a tense tru...
09/05/2024
RT Supporting the Arts: What's On this May
RT Supporting the Arts: What's On this May Watch our YouTube promo here This May RT is delighted to support Dublin Dance Festival along with 20 other ar...
09/05/2024
Honkai: Star Rail' Blasts Off on GeForce NOW
Gear up, Trailblazers - Honkai: Star Rail lands on GeForce NOW this week, along with an in-game reward for members to celebrate the title's launch in the cl...
09/05/2024
Rohde & Schwarz showcases operational efficiency at MPTS
Rohde & Schwarz showcases operational efficiency at MPTS Uncompromised quality and reliability coupled with simplicity and efficiency Rohde & Schwarz will ...
08/05/2024
Saved! and Rudo y Cursi Among Films With May Anniversaries
Diego Luna's character Beto aims to play professional soccer in Carlos Cuar n's film Rudo y Cursi....
08/05/2024
SGL Carbon: Focus on global growth trends pays off
SGL Carbon had a solid start to the first quarter of 2024. Despite the slight decline in sales of 3.9% to 272.6 million (Q1 2023: 283.7 million), adjusted EBI...
08/05/2024
The results are in from Malm Arena as Semi Final 1 participants move through to the Grand Final of this year's Eurovision Song Contest
The results are in from Malm Arena as Semi Final 1 participants move through to...
08/05/2024
Powerful NITV documentary Kindred to premiere during Reconciliation Week
Powerful NITV documentary Kindred to premiere during Reconciliation Week Media releases A journey of family, friendship and home *Watch the trailer here* ...