Sony Pixel Power calrec Sony

Seamless in Seattle: NVIDIA Research Showcases Advancements in Visual Generative AI at CVPR

17/06/2024

NVIDIA researchers are at the forefront of the rapidly advancing field of visual generative AI, developing new techniques to create and interpret images, videos and 3D environments.

More than 50 of these projects will be showcased at the Computer Vision and Pattern Recognition (CVPR) conference, taking place June 17-21 in Seattle. Two of the papers - one on the training dynamics of diffusion models and another on high-definition maps for autonomous vehicles - are finalists for CVPR's Best Paper Awards.

NVIDIA is also the winner of the CVPR Autonomous Grand Challenge's End-to-End Driving at Scale track - a significant milestone that demonstrates the company's use of generative AI for comprehensive self-driving models. The winning submission, which outperformed more than 450 entries worldwide, also received CVPR's Innovation Award.

NVIDIA's research at CVPR includes a text-to-image model that can be easily customized to depict a specific object or character, a new model for object pose estimation, a technique to edit neural radiance fields (NeRFs) and a visual language model that can understand memes. Additional papers introduce domain-specific innovations for industries including automotive, healthcare and robotics.

Collectively, the work introduces powerful AI models that could enable creators to more quickly bring their artistic visions to life, accelerate the training of autonomous robots for manufacturing, and support healthcare professionals by helping process radiology reports.

Artificial intelligence, and generative AI in particular, represents a pivotal technological advancement, said Jan Kautz, vice president of learning and perception research at NVIDIA. At CVPR, NVIDIA Research is sharing how we're pushing the boundaries of what's possible - from powerful image generation models that could supercharge professional creators to autonomous driving software that could help enable next-generation self-driving cars.

At CVPR, NVIDIA also announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

Forget Fine-Tuning: JeDi Simplifies Custom Image Generation Creators harnessing diffusion models, the most popular method for generating images based on text prompts, often have a specific character or object in mind - they may, for example, be developing a storyboard around an animated mouse or brainstorming an ad campaign for a specific toy.

Prior research has enabled these creators to personalize the output of diffusion models to focus on a specific subject using fine-tuning - where a user trains the model on a custom dataset - but the process can be time-consuming and inaccessible for general users.

JeDi, a paper by researchers from Johns Hopkins University, Toyota Technological Institute at Chicago and NVIDIA, proposes a new technique that allows users to easily personalize the output of a diffusion model within a couple of seconds using reference images. The team found that the model achieves state-of-the-art quality, significantly outperforming existing fine-tuning-based and fine-tuning-free methods.

JeDi can also be combined with retrieval-augmented generation, or RAG, to generate visuals specific to a database, such as a brand's product catalog.

https://blogs.nvidia.com/wp-content/uploads/2024/06/JeDi-cow-sculpture.mp4

New Foundation Model Perfects the Pose NVIDIA researchers at CVPR are also presenting FoundationPose, a foundation model for object pose estimation and tracking that can be instantly applied to new objects during inference, without the need for fine-tuning.

The model, which set a new record on a popular benchmark for object pose estimation, uses either a small set of reference images or a 3D representation of an object to understand its shape. It can then identify and track how that object moves and rotates in 3D across a video, even in poor lighting conditions or complex scenes with visual obstructions.

FoundationPose could be used in industrial applications to help autonomous robots identify and track the objects they interact with. It could also be used in augmented reality applications where an AI model is used to overlay visuals on a live scene.

NeRFDeformer Transforms 3D Scenes With a Single Snapshot A NeRF is an AI model that can render a 3D scene based on a series of 2D images taken from different positions in the environment. In fields like robotics, NeRFs can be used to generate immersive 3D renders of complex real-world scenes, such as a cluttered room or a construction site. However, to make any changes, developers would need to manually define how the scene has transformed - or remake the NeRF entirely.

Researchers from the University of Illinois Urbana-Champaign and NVIDIA have simplified the process with NeRFDeformer. The method, being presented at CVPR, can successfully transform an existing NeRF using a single RGB-D image, which is a combination of a normal photo and a depth map that captures how far each object in a scene is from the camera.

VILA Visual Language Model Gets the Picture A CVPR research collaboration between NVIDIA and the Massachusetts Institute of Technology is advancing the state of the art for vision language models, which are generative AI models that can process videos, images and text.

The group developed VILA, a family of open-source visual language models that outperforms prior neural networks on key benchmarks that test how well AI models answer questions about images. VILA's unique pretraining process unlocked new model capabilities, including enhanced world knowledge, stronger in-context learning and the ability to reason across multiple images.

VILA can understand memes and reason based on multiple images or video frames. The VILA model fa
LINK: https://blogs.nvidia.com/blog/visual-generative-ai-cvpr-research/...
See more stories from nvidia

North America Stories

10/04/2026

The Invisible OPEX Killer: Is Your Server Room Dragging You Down?

The Invisible OPEX Killer: Is Your Server Room Dragging You Down? In the broadcast world, we talk a lot about uptime. We talk about talent retention, latency...

10/04/2026

NAB 2026: Imagine Communications to Showcase Expanded Multiviewer Portfolio

Imagine Communications will showcase its multiviewer portfolio at NAB Show 2026 (April 19-22, Booth N1328, Las Vegas Convention Center), including Prismon and t...

10/04/2026

NAB 2026: Chyron Releases PRIME VSAR 2.3 with Updated Unreal Engine Integration

Chyron has released PRIME VSAR 2.3, an update to its virtual set and augmented reality solution for broadcast. The release adds compatibility with Unreal Engine...

10/04/2026

NAB 2026: Techex to Showcase New tx darwin Capabilities

Techex will exhibit at NAB Show 2026 (Booth W2267, April 19-23, Las Vegas Convention Center), demonstrating new tx darwin features including consumer multiview,...

10/04/2026

NAB 2026: NDI to Showcase Ecosystem and NDI 6.3

NDI will exhibit at NAB Show 2026, demonstrating its IP video ecosystem through live partner integrations, NDI 6.3 features, AI metadata workflows, and creator ...

10/04/2026

FOR-A Acquires Tamura Corporations Information Equipment Business

FOR-A has announced the acquisition of all shares of Tamu Radiance Corporation, a new company spun off from the Information Equipment Business of Tamura Corpora...

10/04/2026

NAB 2026: InSync Technology to Unveil New Video Processing and Frame Rate Conversion Products

InSync Technology will showcase new and updated video conversion products at NAB...

10/04/2026

TNT Sports and DAZN Announce Monthly Boxing Event Series in the United States

TNT Sports and DAZN have announced a partnership to air monthly boxing events in the United States under the brand The Fight. The series will be promoted in p...

10/04/2026

Panasonic Introduces SQ3 Series 4K LCD Displays for Professional Environments

Panasonic Projector and Display has announced the SQ3 Series of 4K LCD displays as part of its MEVIX professional display portfolio. All sizes will be available...

10/04/2026

Amagi Adds Agentic Capabilities to Its Media Operations Platform

Amagi has announced the addition of Agentic Media Operations to its Amagi NOW platform, integrating AI reasoning agents across its media supply chain workflows ...

10/04/2026

LTN Announces Network Enhancements Ahead of C-Band Spectrum Auction

LTN has announced enhancements to its global IP video network targeting broadcasters transitioning from satellite distribution. The updates come ahead of US fed...

10/04/2026

Daktronics Installs New LED Displays at Yankee Stadium

Daktronics has installed new LED displays at Yankee Stadium, upgrading the main centerfield board, two flanking boards, and two ribbon displays spanning the 200...

10/04/2026

NAB 2026: Harmonic Announces AI and Cloud Updates to Hybrid Streaming Solution

Harmonic has announced updates to its hybrid streaming solution, including Model Context Protocol (MCP) connectivity for AI applications, cloud-native deploymen...

10/04/2026

NAB 2026: MultiDyne to Debut FiberSaver-10G and VF-9100

MultiDyne Video and Fiber Optic Systems will introduce two new fiber transport products at NAB Show 2026 (Booth C4425, April 19-22): the FiberSaver-10G waveleng...

10/04/2026

NAB 2026: Telos Alliance and ip-studio to Demonstrate STUDIO ZERO

Telos Alliance and ip-studio will demonstrate STUDIO ZERO, a cloud-hosted virtual studio, at NAB Show 2026. First introduced at NAB Show 2023, STUDIO ZERO integ...

10/04/2026

Pixotope and d&b Solutions Announce Strategic Partnership for XR and Virtual Studio Production

d&b solutions, a London-based audio-visual, lighting, and media integration grou...

10/04/2026

ARRI and SmallHD Announce Lens Data Monitor Overlay License for Hi-5 and Hi-5 SX

ARRI and SmallHD have announced a new expansion license for ARRI's Hi-5 and Hi-5 SX hand units that displays lens data overlays on supported SmallHD monitor...

10/04/2026

Roku to Stream Exclusive Savannah Bananas Game Package on Roku Sports Channel

Roku and the Banana Ball Championship League (BBCL) have announced an exclusive streaming partnership to bring five BBCL games to the Roku Sports Channel in 202...

10/04/2026

Ratings Roundup: More Than 18 Million Fans Tune Into 2026 NCAA Mens March Madness on TNT and CBS Sports

Ratings Roundup is a rundown of recent rating news and is derived from press rel...

10/04/2026

No Other Land, Mr. Nobody Against Putin,and More Sundance Institute-Supported Films Nominated for Peabody Awards

The Peabody Awards don't just recognize great storytelling, they spotlight t...

10/04/2026

2026 NAB Show Exhibitor Insight: Bitcentral

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Bitcentral To Feature Connected Media Workflows At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Bitcentral to Showcase Connected Media Workflows and Inte...

NEWPORT BEACH, Calif., April 10, 2026 Bitcentral, a leading provider of professional media solutions for broadcast and digital video, will showcase its latest...

10/04/2026

Ikegami to Introduce Expanded Range of Broadcast Production Solutions at NAB 2026

Ikegami to Introduce Expanded Range of Broadcast Production Solutions at NAB 202...

10/04/2026

AJA Debuts SMPTE ST 2110 and openGear Solutions Ahead of NAB 2026

AJA Debuts SMPTE ST 2110 and openGear Solutions Ahead of NAB 2026 Brie Clayton April 10, 2026 0 Comments New gear and updates address evolving hybrid ...

10/04/2026

Portland Fire+ Streaming Platform Launches

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Tod Musgrave Joins Proton as U.S. Sales & Marketing Director

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Proton Expands Minicam Portfolio With Proton Pro At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

FCC To Vote on Changes to Audible Crawl Rule

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Frequency Launches AI Platform for Streaming Television a...

Frequency, the engine behind the worlds leading streaming television channels, today launched its AI platform for Frequency Studio, powering the entire channel ...

10/04/2026

Jnger Audio Joins EBU ADM Integration Group as Founding Member to Help Advance ADM/S-ADM Integration

J nger Audio Joins EBU ADM Integration Group as Founding Member to Help Advance...

09/04/2026

NAB 2026: Zixi to Demonstrate Live Video Workflows and Satellite Replacement

Zixi will demonstrate IP-based live video workflow solutions at NAB Show 2026 (Booth W2057). The industry is moving quickly toward IP-based distribution as br...

09/04/2026

Deloitte Research: Women's Elite Sports Revenues Expected to Reach at Least $3 Billion in 2026

Global women's elite sports revenues are expected to reach at least $3 billi...

09/04/2026

Monitor Engineer Gavin Tempany Mixes Kylie Minogue's Tension Tour on Solid State Logic L550 Plus

Monitor engineer Gavin Tempany mixed Kylie Minogue s Tension Tour on a Solid Sta...

09/04/2026

NAB 2026: KOKUSAI DENKI Electric America to Debut New 4K Camera and Remote Control Panel

KOKUSAI DENKI Electric America will exhibit at NAB Show 2026 (Booth C5507), debu...

09/04/2026

NBC Sports Reviews Innovations and Milestones from Its 2025-26 NBA Regular Season

With the 2025-26 NBA regular season concluded and the playoffs beginning next we...

09/04/2026

NAB 2026: Telestream and Mimir Announce Integration for Ingest-to-Editorial Workflows

Telestream and Mimir have announced an integration connecting Telestream's V...

09/04/2026

NAB 2026: Bitmovin Expands Live Encoding and Observability Solutions for End-to-End Live Streaming Monitoring

Bitmovin has expanded its Live Encoding and Observability solutions to provide r...

09/04/2026

Nashville Predators and Scripps Sports Announce Multi-Year Broadcast Agreement

The Nashville Predators and Scripps Sports have announced a multi-year media rights agreement covering local preseason, regular season, and first-round playoff ...

09/04/2026

ASG Partners with Beam Dynamics for Asset Intelligence Platform

Advanced Systems Group, LLC has announced a partnership with Beam Dynamics to offer the Beam Asset and License Intelligence Platform to its clients. The platfor...

09/04/2026

NAB 2026: Lawo Introduces Edge One Converged Video and Audio Stagebox

Lawo has unveiled Edge One, a combined video and audio stagebox for broadcast and Pro AV workflows. The device will be on display at NAB Show (Booth C2108, Apri...

09/04/2026

NAB 2026: SMPTE to Host ST 2110 IP Media Roadshow

The Society of Motion Picture and Television Engineers (SMPTE) will host the SMPTE ST 2110 IP Media Roadshow on Tuesday, April 21, 2026, at the Las Vegas Conven...

09/04/2026

Atlanta Braves Upgrade Video Displays at Truist Park

The Atlanta Braves have completed upgrades to video displays in and around Truist Park ahead of the 2026 MLB season. The upgrades include the Delta Out-of-Town ...

09/04/2026

USC Installs Daktronics LED Displays Across Four Athletics Venues

The University of Southern California has contracted Daktronics (NASDAQ: DAKT) of Brookings, South Dakota, to manufacture and install 22 LED displays across fou...

09/04/2026

NAB 2026: Backlight to Showcase Iconik and Wildmoka Integration

Backlight, the media technology company behind Iconik and Wildmoka, will showcase its Creative Operations Platform at NAB Show 2026 (Booth N2829, April 19-22). ...

09/04/2026

MotoAmerica Superbike to Air on VICE TV for 2026 Season

MotoAmerica and V10 Entertainment have announced a partnership to broadcast MotoAmerica Superbike racing on VICE TV for the 2026 season. Coverage begins live on...

09/04/2026

Proton Camera Innovations Appoints Tod Musgrave as US Sales and Marketing Director

Proton Camera Innovations has announced the appointment of Tod Musgrave as US Sa...

09/04/2026

Former UEFA, Orange Executive Nicolas Dal Launches OVERCAST Private-Cloud Production Service

Designed specifically for live sports broadcasting, new platform features IP-nat...

09/04/2026

NEWstalgia: How the Return of the NBA on NBC Was Driven by a Bold and Ownable' Graphics Package

Blending 1990s DNA, modern motion theory, and a distinctly colorful brand identi...

09/04/2026

SVG Sit-Down: Christy Media's Amy Vacher on What It Takes To Find the Best Person for the Job

Technical capability is essential, but long-term success often depends on how we...