Sony Pixel Power calrec Sony

Seamless in Seattle: NVIDIA Research Showcases Advancements in Visual Generative AI at CVPR

17/06/2024

NVIDIA researchers are at the forefront of the rapidly advancing field of visual generative AI, developing new techniques to create and interpret images, videos and 3D environments.

More than 50 of these projects will be showcased at the Computer Vision and Pattern Recognition (CVPR) conference, taking place June 17-21 in Seattle. Two of the papers - one on the training dynamics of diffusion models and another on high-definition maps for autonomous vehicles - are finalists for CVPR's Best Paper Awards.

NVIDIA is also the winner of the CVPR Autonomous Grand Challenge's End-to-End Driving at Scale track - a significant milestone that demonstrates the company's use of generative AI for comprehensive self-driving models. The winning submission, which outperformed more than 450 entries worldwide, also received CVPR's Innovation Award.

NVIDIA's research at CVPR includes a text-to-image model that can be easily customized to depict a specific object or character, a new model for object pose estimation, a technique to edit neural radiance fields (NeRFs) and a visual language model that can understand memes. Additional papers introduce domain-specific innovations for industries including automotive, healthcare and robotics.

Collectively, the work introduces powerful AI models that could enable creators to more quickly bring their artistic visions to life, accelerate the training of autonomous robots for manufacturing, and support healthcare professionals by helping process radiology reports.

Artificial intelligence, and generative AI in particular, represents a pivotal technological advancement, said Jan Kautz, vice president of learning and perception research at NVIDIA. At CVPR, NVIDIA Research is sharing how we're pushing the boundaries of what's possible - from powerful image generation models that could supercharge professional creators to autonomous driving software that could help enable next-generation self-driving cars.

At CVPR, NVIDIA also announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

Forget Fine-Tuning: JeDi Simplifies Custom Image Generation Creators harnessing diffusion models, the most popular method for generating images based on text prompts, often have a specific character or object in mind - they may, for example, be developing a storyboard around an animated mouse or brainstorming an ad campaign for a specific toy.

Prior research has enabled these creators to personalize the output of diffusion models to focus on a specific subject using fine-tuning - where a user trains the model on a custom dataset - but the process can be time-consuming and inaccessible for general users.

JeDi, a paper by researchers from Johns Hopkins University, Toyota Technological Institute at Chicago and NVIDIA, proposes a new technique that allows users to easily personalize the output of a diffusion model within a couple of seconds using reference images. The team found that the model achieves state-of-the-art quality, significantly outperforming existing fine-tuning-based and fine-tuning-free methods.

JeDi can also be combined with retrieval-augmented generation, or RAG, to generate visuals specific to a database, such as a brand's product catalog.

https://blogs.nvidia.com/wp-content/uploads/2024/06/JeDi-cow-sculpture.mp4

New Foundation Model Perfects the Pose NVIDIA researchers at CVPR are also presenting FoundationPose, a foundation model for object pose estimation and tracking that can be instantly applied to new objects during inference, without the need for fine-tuning.

The model, which set a new record on a popular benchmark for object pose estimation, uses either a small set of reference images or a 3D representation of an object to understand its shape. It can then identify and track how that object moves and rotates in 3D across a video, even in poor lighting conditions or complex scenes with visual obstructions.

FoundationPose could be used in industrial applications to help autonomous robots identify and track the objects they interact with. It could also be used in augmented reality applications where an AI model is used to overlay visuals on a live scene.

NeRFDeformer Transforms 3D Scenes With a Single Snapshot A NeRF is an AI model that can render a 3D scene based on a series of 2D images taken from different positions in the environment. In fields like robotics, NeRFs can be used to generate immersive 3D renders of complex real-world scenes, such as a cluttered room or a construction site. However, to make any changes, developers would need to manually define how the scene has transformed - or remake the NeRF entirely.

Researchers from the University of Illinois Urbana-Champaign and NVIDIA have simplified the process with NeRFDeformer. The method, being presented at CVPR, can successfully transform an existing NeRF using a single RGB-D image, which is a combination of a normal photo and a depth map that captures how far each object in a scene is from the camera.

VILA Visual Language Model Gets the Picture A CVPR research collaboration between NVIDIA and the Massachusetts Institute of Technology is advancing the state of the art for vision language models, which are generative AI models that can process videos, images and text.

The group developed VILA, a family of open-source visual language models that outperforms prior neural networks on key benchmarks that test how well AI models answer questions about images. VILA's unique pretraining process unlocked new model capabilities, including enhanced world knowledge, stronger in-context learning and the ability to reason across multiple images.

VILA can understand memes and reason based on multiple images or video frames. The VILA model fa
LINK: https://blogs.nvidia.com/blog/visual-generative-ai-cvpr-research/...
See more stories from nvidia

North America Stories

30/04/2026

Ocean Blue Software Launches ATSC 3.0 Inspector for Smart TVs

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

Digital Azul positions Lisbon as a remote production hub...

Scalable broadcast-grade production over public internet, replacing traditional OB workflows...

30/04/2026

V Nova Brings DTV Deployment LCEVC Ecosystem Progress and...

Live demonstrations highlight LCEVC ecosystem momentum, AI-powered video pipelines, and expansion across broadcast, streaming, and social media. DTV (TV 3.0) ...

30/04/2026

Student Spotlight: Matthew Leon

Student Spotlight: Matthew Leon The dual major shares his path from community college to Berklee, and how his heritage influences his work. April 29, 2026 B...

30/04/2026

Berklee Artists to Perform at Major Global Music Festivals

Berklee Artists to Perform at Major Global Music Festivals As part of the Berklee Popular Music Institute, students will perform at Lollapalooza, Governors Ba...

30/04/2026

Study: Rokus Low-Cost, Ad-Free Howdy Streamer Hits 1 Million Subs

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

MLS Innovation Lab Selects AI Partners

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

NRB Files FCC Complaint Over Jimmy Kimmel Live!' Monologue

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

D.C. Court Denies Emergency Stay of Nexstar/Tegna Merger

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

NAB Criticizes FCC for Ordering Early Renewal of ABC-Owned Stations

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

FCC Approves Station Swaps Between Scripps and Gray Media

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

Telestream Introduces Pulse - a Software-Defined Test an...

A flexible monitoring platform designed to simplify ST 2110 operations, consolidate vendor tools, and support modern live production environments. See it at NAB...

30/04/2026

Langlev Takes Run Amok to Sundance with Zeiss Supreme Pri...

Sundance-premiering Run Amok is an expressive, unconventional take on today's teen experience, replete with musical numbers. When cinematographer Shachar ...

30/04/2026

Studio Technologies Elevates Jacksonville State Universit...

JACKSONVILLE, AL, APRIL 29, 2026 Jacksonville State University, known as Jax State and a proud NCAA Division I member of Conference USA, has transformed its a...

30/04/2026

Knowledge Network selects ThinkAnalytics to launch AI-pow...

Transforming viewer and content data into real-time intelligence to deliver relevant streaming experiences at scale ThinkAnalytics, the global leader in AI-pow...

30/04/2026

Sports Production, Delivery is Big Biz at NAB

Sports Production, Delivery is Big Biz at NAB Andy Marken April 29, 2026 0 Comments Hero image source: NAB One of the neat things about trade shows i...

30/04/2026

It's Gonna Be May: 16 Games Hit the Cloud This Month, With More NVIDIA GeForce RTX 5080 Power

It's gonna be May - and the cloud's in full festival mode. 16 games ar...

30/04/2026

April 29, 2026

Scripps Research ranks third in 2026 Cure Innovation Index April 29, 2026 LA JOLLA, CA Scripps Research ranked third in the inaugural 2026 Cure Innovation In...

29/04/2026

Churchill Downs Racetrack Lifts Curtain on New Big Board Ahead of Kentucky Derby Week

It was a delicate job in a 150-year-old venue laden with traditions. Begun at th...

29/04/2026

How the MLS Innovation Lab Enhances Everything From Athlete Performance to Production to Ticket Sales

In annual event, the league gives startup companies the opportunity to prove the...

29/04/2026

Save the Date: SVG Venues & Teams Summit Travels South to Miamis Kaseya Center on Aug. 12

Panel discussions, networking, and a facility tour will take place in the renova...

29/04/2026

The Cast and Crew of Conbody VS Everybody Have Each Other's Backs for Life

(L-R) Derek Drescher, Coss Marte, and Syretta Wright have each other's backs. (Micheal Hurcomb/Shutterstock for Sundance Film Festival) By Veronika Lee Cla...

29/04/2026

YEP Spotlight: Julissa Padilla

Some people stumble into their passion. Julissa Padilla walked straight into a film vault. For her, entertainment was never just about the movies themselves. It...

29/04/2026

Clear-Com Promotes Brian Grahn, Ben Turnwell to Expanded Roles

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

The CW Unveils Major Streaming Deals with ESPN and Roku

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

Clear-Com Announces New Roles for Brian Grahn and Ben Tur...

Clear-Com has appointed Brian Grahn as Market Outreach Manager of the Americas and Ben Turnwell as Business Development Manager for EMEA live, expanding their ...

29/04/2026

Reduce Workflow Complexity with nxtedition at MPTS 2026

nxtedition is bringing its range of consolidated production tools to MPTS 2026, with new developments spanning transcription, editing, graphics and AI-assisted ...

29/04/2026

Telxius taps Synamedia to unlock seamless global multi-CD...

Quortex Switch to boost the streaming experience for Telxius customers, reaching millions of viewers worldwide Synamedia and Telxius, the leading global connec...

29/04/2026

freispace and Projective Announce Joint Showcase of Integ...

freispace, the leading ERP-as-a-Service platform for media and entertainment production, and Projective, a leading provider of post-production collaboration tec...

29/04/2026

DHD Celebrates 30th Anniversary at 2026 NAB Show

DHD reports strong interest in its broadcast audio product range, exhibited at the April 19th-22nd NAB Show in Las Vegas. The event attracted a claimed 58,000 a...

29/04/2026

Student Spotlight: Alan Catz

Student Spotlight: Alan Catz The Argentine film and game composer talks about working on League of Legends, receiving Berklee's BMI Award, and the lifelon...

29/04/2026

Jay Jennings Builds the Worlds You Hear on Screen

Jay Jennings Builds the Worlds You Hear on Screen The supervising sound designer behind A Minecraft Movie, The Meg, Letters from Iwo Jima, and dozens of other...

29/04/2026

Paramount Will be 49.5% Foreign Owned After WBD Merger

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

NAB Show 2026: AI, Vertical and BPS Dominate Broadcasters' Discussions

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

IAB Tech Lab Announces New OpenRTB Attributes

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

'SUPERNOVA: GENESIS' Reached a Peak Audience of More Than 6.5 Million Viewers on Netflix

Back to All News SUPERNOVA: GENESIS Reached a Peak Audience of More Than 6.5 Mi...

29/04/2026

April 28, 2026

Jin-Quan Yu elected to the National Academy of Sciences Yu is recognized for his pioneering work in synthetic organic chemistry. April 28, 2026 LA JOLLA, CA S...

28/04/2026

Sound for the NFL Draft Show Balances Often Conflicting Goals

The audio team for the entertainment event must blend speech intelligibility with full-range music reproduction while considering the broadcast Last week's...

28/04/2026

Pac-12 Releases New Primary Mark Ahead of 2026-27 Season

The Pac-12 Conference has released an updated primary mark and logo as the starting point of the new league's brand identity. The mark was soft-launched acr...

28/04/2026

DP World Tour to Become First Professional Sports Organization to Use Amazon Leo Satellite Connectivity

The DP World Tour and Amazon Leo have signed an agreement making Amazon's lo...

28/04/2026

Pixellot and HELIOS Announce Automated Hockey Shift Video Integration

Pixellot and HELIOS have announced an integration that automatically converts full-game hockey video into individualized shift videos for each athlete, without ...

28/04/2026

Daktronics Installs New LED Video Display at Asheville Tourists Ballpark

Daktronics has partnered with the Asheville Tourists to manufacture and install a new LED video display. The installation was completed in late 2025 and is now ...

28/04/2026

Eutelsat and PCTV Renew Long-Term Video Distribution Partnership in Mexico

Eutelsat has announced the renewal of its partnership with PCTV, a content aggregation and distribution company in Mexico and part of Megacable Holdings, for co...

28/04/2026

Gary SouthShore RailCats Install New Daktronics LED Display at U.S. Steel Yard

Daktronics has partnered with the Gary SouthShore RailCats to install a new LED video display at U.S. Steel Yard, replacing the previous Daktronics display inst...

28/04/2026

Telos Alliance and College Radio Foundation Award Omnia.11 Processor to Wright State Universitys WWSU-FM

Telos Alliance and the College Radio Foundation have announced that WWSU-FM of W...

28/04/2026

Inside the Mix: A1s Florian Brown and James Deason on Golfs Biggest Broadcasts

Golf viewership is growing. The 2025 Ryder Cup drew five million viewers in the UK, a 45% increase over the 2023 event. The US Open was the most streamed golf e...

28/04/2026

The CW Network Acquires Exclusive Broadcast Rights to WWE NXT Premium Live Events

The CW Network and WWE, part of TKO Group Holdings (NYSE: TKO), have announced t...

28/04/2026

NAB 2026: AIMS Wins NAB Show 2026 Product of the Year Award for IPMX

The Alliance for IP Media Solutions (AIMS) has announced that the Internet Protocol Media Experience (IPMX) suite of standards and specifications has been named...

28/04/2026

NAB 2026 SportsTechBuzz in Review: A Look Back at the Big News From 150+ Companies in Vegas

The 2026 NAB Show is in the books and the show once again served up a cavalcade ...

28/04/2026

Gray Media, RAJ Sports Launch New Womens Sports RSN in Portland - Rose City SportsNet

Gray Media and RAJ Sports have announced Rose City SportsNet (RCSN), a new netwo...