Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

12/06/2026

Bitmovin Partners with Simplestream and Xperi to Support...

Bitmovin, a provider of video streaming solutions, has partnered with Simplestream, a provider of OTT and broadcast solutions, and technology provider Xperi, to...

12/06/2026

Jigsaw24 Signs Deal to Resell Leostream Remote Desktop Ac...

Leostream Corporation, creator of the world-leading Leostream Remote Desktop Access Platform, today announced Jigsaw24, a leading B2B IT solutions provider wit...

12/06/2026

Study: 2026 Election Cycle to Hit Record $11.6 Billion Ad Spend

Share Copy link Facebook X Linkedin Bluesky Email...

12/06/2026

NAB Elects Leadership at June Board of Directors Meeting

Share Copy link Facebook X Linkedin Bluesky Email...

12/06/2026

FCC Believes World Cup Communication Will Score Highly

Share Copy link Facebook X Linkedin Bluesky Email...

12/06/2026

Broadcasters Back NO FAKES Act

Share Copy link Facebook X Linkedin Bluesky Email...

12/06/2026

Scripps Unveils Coverage Plans For America's 250th Anniversary

Share Copy link Facebook X Linkedin Bluesky Email...

12/06/2026

How Aussie indie games and screen are levelling up with IP

How Aussie indie games and screen are levelling up with IP 11 June 2026 Ari Harrison, Pro Jank Footy Head of Games Joey Egger and Ari Harrison of Umbrella sha...

12/06/2026

RT stays Up All Night with brand new daily 2026 FIFA World Cup Vodcast

RT stays Up All Night with brand new daily 2026 FIFA World Cup Vodcast Adding to the fun around 2026 FIFA World Cup tournament, RT has launched a brand new d...

12/06/2026

June 11, 2026

A fentanyl countermeasure that adapts to combat future black-market drugs Scripps Research scientists developed a vaccine that teaches the immune system to rapi...

11/06/2026

HBSs Johannes Franken on Digital Innovations, the Role of the Influencer at the 2026 FIFA World Cup

The immense size of the tourney and its Atlantic-spanning operation also disting...

11/06/2026

Nielsen: Soccer Fandom in North America Tops 136 Million, Up 10.9% in Five Years

Nielsen has released a new soccer fandom consumer research report, The Fans Behind The Game: FIFA World Cup 2026 Edition, examining the soccer audience in the...

11/06/2026

Telemundo Announces All-Day Opening Day Coverage for FIFA World Cup 2026 on June 11

Telemundo will launch its FIFA World Cup 2026 coverage on Thursday, June 11 with...

11/06/2026

Fubo Announces Distribution Agreement With NBCUniversal

FuboTV Inc. has announced a distribution agreement with NBCUniversal. Fubo customers can now stream Telemundo and Universo, with NBC Sports Network (NBCSN), NBC...

11/06/2026

DAZN Announces In-App Features for FIFA World Cup 2026 Coverage in Spain, Italy, and Japan

DAZN has announced its in-app features for FIFA World Cup 2026 coverage in Spain...

11/06/2026

Roblox Report: Sports Engagement on Platform Drives Real-World Fandom and Purchases

Roblox has released the 2026 Roblox Digital Expression Report: Wave 4 - Sports D...

11/06/2026

Andrea Bocelli, David Guetta, Megan Thee Stallion, and EJAE Release Official FIFA World Cup 2026 Anthem DNA'

FIFA has unveiled DNA, the Official FIFA World Cup 2026 Anthem, performed by A...

11/06/2026

ESPN Announces Extensive English- and Spanish-Language World Cup 2026 Coverage

ESPN will provide English- and Spanish-language news and information coverage of FIFA World Cup 2026 across its U.S. media platforms from June 11 through July 1...

11/06/2026

SVG Students To Watch: Teddy Batkin, Rochester Institute of Technology

The latest product of the outstanding RIT Sports Network program, this recent grad from Long Island is carving out a promising path in broadcast engineering In...

11/06/2026

DAZN and DSPORTS Announce Distribution Agreement Across Five Latin American Countries

DAZN has announced a multi-year agreement to make DSPORTS channels available to ...

11/06/2026

Resource Actors Throughout the Years at Sundance Institute's Directors Lab

Laura Dern at the 1986 Sundance Institute Directors Lab (Photo by Eric Edwards) By Lucy Spicer It takes a village to bring together the Sundance Institute lab...

11/06/2026

Introducing a New Standard for Podcast Plays and Upgraded Creator Analytics Experience

As podcast formats evolve in the streaming era, podcasting needs updated, transp...

11/06/2026

RADAR Italia Unveils 6 New Artists and a New Approach for 2026

As Spotify's global RADAR program enters its sixth year in Italy, a new class of artists is stepping into the spotlight. Today, we're announcing the six...

11/06/2026

5 Audiobooks that Amplify and Celebrate Queer Voices

Pride Month is a time for celebration, reflection, and amplifying the diverse stories and perspectives from the LGBTQIA+ community that enrich our world. To hel...

11/06/2026

VSL introduce Synchron Solo Violin 1 & Cello (sordino)

First in new line of muted string libraries VSL have just announced the launch of two new string libraries that represent the first two instalments in a new...

11/06/2026

Novation reveal the Launchkey 61 MK4 White

New colour option for 61-key Launchkey MK4 At Superbooth 2025, Novation introduced the Launchkey Mini 37 White and Launchkey 49 White, bringing an additiona...

11/06/2026

Arturia announce the MiniLab 37

Larger, but still compact! Arturia's popular compact MIDI controller keyboard is now available in a, well, slightly less compact version! The new MiniLa...

11/06/2026

Eurosatory 2026: Rohde & Schwarz shapes the new-generation battlefield

Eurosatory 2026: Rohde & Schwarz shapes the new-generation battlefield Rohde & Schwarz unveils next generation SIGINT/EW and CUAS solutions on uncrewed system...

11/06/2026

Rohde & Schwarz unveils NEMACS - Directional, ultra secure connectivity for the future battlefield

Rohde & Schwarz unveils NEMACS - Directional, ultra secure connectivity for the ...

11/06/2026

MTI FILM acquires Mango/New Edit

MTI FILM acquires Mango/New Edit Posted by MTI Film on June 10, 2026 LOS ANGELES, CA - June 2026 - MTI FILM, the multiple Emmy Award winning Hollywood post-p...

11/06/2026

Ungrounded LLM Fabricates Every Detail for Nearly 1 in 5 Movie and TV Titles Tested, New Gracenote Report Finds

Study underscores the need for authoritative content intelligence to build trust...

11/06/2026

PTZOptics, LayerJot Partner on AI-Powered PTZ at InfoComm 2026

Share Copy link Facebook X Linkedin Bluesky Email...

11/06/2026

Chyron Unveils PAINT 10.4

Share Copy link Facebook X Linkedin Bluesky Email...

11/06/2026

Maxon Brings Real-Time Architectural Visualization to AIA26 With New Redshift for Revit and Archicad Integration Beta

Maxon Brings Real-Time Architectural Visualization to AIA26 With New Redshift fo...

11/06/2026

ABC Kid's Caper Crew Shoots Australian Adventure with Blackmagic Design

ABC Kid's Caper Crew Shoots Australian Adventure with Blackmagic Design Brie Clayton June 11, 2026 0 Comments DP Judd Overton and team bring Wes A...

11/06/2026

PTZOptics and LayerJot demo Visual Reasoning at InfoComm...

PTZOptics, and LayerJot today announced live demonstrations at InfoComm 2026 showing how prompt-based AI, robotic camera control, and high-performance computing...

11/06/2026

Lightware launches GPIO Button to deliver simplified hard...

Lightware, an industry leader in signal management, announces the release of GPIO-Button-10S, a dedicated control interface enabling straightforward press-to-a...

11/06/2026

NABs LeGeyt Urges Congress to Limit NFL's Antitrust Exemption

Share Copy link Facebook X Linkedin Bluesky Email...

11/06/2026

Fubo Inks New Distribution Agreement with NBCUniversal

Share Copy link Facebook X Linkedin Bluesky Email...

11/06/2026

Kiloview to Showcase Broadcast-Grade AV-over-IP Solutions...

Kiloview, a leading innovator in AV-over-IP video solutions, will return to InfoComm 2026 (Booth# N8327) with broadcast-grade AV-over-IP solutions designed for ...

11/06/2026

Australian Games Industry Glossary of Terms

Australian Games Industry Glossary of Terms 10 June 2026 From DAU and EULA to COT and QADE, here's a list of game industry terms, industry jargon and their...

11/06/2026

Berklee's Tonya Butler Named Music Business Educator of the Year

Berklee's Tonya Butler Named Music Business Educator of the Year The Music Business Association honored Butler at its annual Bizzy Awards. June 10, 2026 ...

11/06/2026

Ann Mincieli to Receive Honorary Doctorate at Berklee NYC Graduate Commencement

Ann Mincieli to Receive Honorary Doctorate at Berklee NYC Graduate Commencement The five-time Grammy-winning engineer and producer, known for her longstanding...

11/06/2026

Daisy May Cooper rallies the nation ahead of ICC Womens T20 World Cup

Thursday 11 June 2026 Daisy May Cooper rallies the nation ahead of ICC Women's T20 World CupTurn on cookies to view this content. Go to Privacy options and...

11/06/2026

Hadewych Minis and Geert van Rampelberg to Star in New Netflix Series Directed by Paula van der Oest

Back to All News Hadewych Minis and Geert van Rampelberg to Star in New Netflix...

11/06/2026

Official Trailer for Anime Adaptation of Thunder 3' Unveiled Ahead of July 9 Premiere

Back to All News Official Trailer for Anime Adaptation of Thunder 3' Unvei...