Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

03/03/2026

Long-Awaited ATSC 3.0 Rulemaking Overshadows NAB Show Expectations

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

Audio Tech at NAB Show: Are We in the Second Wave' of IP?

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

IP's Impact on Imaging Tech on Full Display at NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

Live Production Over IP in 2026: Software-Defined Everything

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

NAB Show Leverages Revitalized LVCC To Reflect M&E Transformation

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

Home Post Production Strengthens Factual and Natural Hist...

Home Post Production has further expanded its factual, unscripted, and entertainment capabilities with the acquisition of Picture Shop Bristol, a leading post h...

03/03/2026

Full Year 2025 Results

Luxembourg, 2 March 2026 -- SES S.A. fully consolidates Intelsat from 17 July 2025 and announces financial results for the year ended 31 December 2025 FY25 Pe...

03/03/2026

Iyuno Taps Dante AV to Sync Audio and Video Content

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

HBO Max and Paramount+ Streamers to Merge

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

Survey: 70% of CTV Advertisers Plan to Boost Spending in 2026

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

XGN Global, X1 Mobile Show New 5G Broadcast Smartphone

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

Scripps Completes Sale of WFTX to Sun Broadcasting

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

HbbTV Association Formally Integrates DRM into Core Specification

Share Copy link Facebook X Linkedin Bluesky Email...

03/03/2026

RT Media Sales: Spry Finance renews Marty In The Morning lyric fm sponsorship for a fourth year

RT Media Sales has announced Spry Finance has renewed it's sponsorship of M...

02/03/2026

NBC Sports Selected Imaginary Forces to Create Cinematic Main Title Sequence for Primetime In Milan

NBC Sports selected Imaginary Forces to create the main title sequence for the M...

02/03/2026

NFL Launches Draft-Inspired Innovation Challenge to Advance Future of Fan and Event Technology

The NFL announces the launch of the NFL Draft Innovation Challenge, a new crowds...

02/03/2026

DataMiner SatOps Named Finalist for 2026 Teleport Technology of the Year

The World Teleport Association (WTA) has named Skyline Communications' DataMiner SatOps solution as a finalist for the 2026 Teleport Technology of the Year ...

02/03/2026

Ventum Tech Partners with Pixotope to Expand Virtual Production Capabilities Across the GCC

Ventum Tech announces a strategic partnership with Pixotope, a real-time graphic...

02/03/2026

Victory+ Launches NWSL Content Hub, Taps Top Creators to Redefine How Fans Connect with League

Victory , a free, ad-supported sports streaming service, announces the launch of...

02/03/2026

Global powerhouse Delta Goodrem to represent Australia at this year's Eurovision Song Contest in Vienna, Austria

Global powerhouse Delta Goodrem to represent Australia at this year's Eurovi...

02/03/2026

SBS brings together prominent voices to discuss rising social division and seek solutions for a more inclusive country

SBS brings together prominent voices to discuss rising social division and seek ...

02/03/2026

NITV Unveils Wednesday Night Sport Night

NITV Unveils Wednesday Night Sport Night Media releases A Midweek Hub for Blak Sport Flagship NRL program Over The Black Dot moves to Wednesdays at 9:30pm ...

02/03/2026

Rohde & Schwarz demonstrates FR1-FR3 carrier aggregation, advancing 6G readiness

Rohde & Schwarz demonstrates FR1-FR3 carrier aggregation, advancing 6G readiness Rohde & Schwarz and Qualcomm Technologies, Inc. have reached another pivotal ...

02/03/2026

L3Harris Appoints Ken Sharp Senior Vice President and Chief Financial Officer, Ken Bedingfield President of Missile Solutions

MELBOURNE, Fla., March 2, 2026 - L3Harris Technologies (NYSE: LHX) today announc...

02/03/2026

Nielsen reveals New Zealand's top ad spenders and categories for 2025

Harvey Norman tops New Zealand's biggest ad spenders as retail leads and telcos and beverages sure Auckland March 3, 2026 - Nielsen's New Zealand'...

02/03/2026

aconnic and Arqit join forces to present the first quantum-safe industrial OT security gateway firewall at Mobile World Congress 2026

aconnic AG (ISIN: DE000A0LBKW6), Munich, and Arqit Quantum Inc. (Nasdaq: ARQQ, A...

02/03/2026

Advanced Systems Group Promotes Jody Boatwright to Chief...

Advanced Systems Group, LLC (ASG), a technology and services provider for media creatives and content owners, has appointed Jody Boatwright as Chief Strategy Of...

02/03/2026

Lightware TPN ecosystem powers smarter networks and strea...

Lightware, an industry leader in signal management, is reinforcing its commitment to simplifying complex AV environments with the continued evolution of its TPN...

02/03/2026

ZEISS Cinema Hosts First US Aatma Lens Demo Event in LA

ZEISS Cinema opens its doors to an exclusive screening, filmmakers Q&A and hands-on showcase of the new ZEISS Aatma lens family on Tuesday, March 10, 6:00-9:00 ...

02/03/2026

DoPchoice Introduces SNAPBAG Round for the New Astera Qui...

To expand the creative potential of the compact Astera QuikBeam, DoPchoice introduces the SNAPBAG Round. The new light-shaping accessory transforms the powerfu...

02/03/2026

Douglas Dubler Master of Light - Five Decades Shooting w...

Douglas Dubler On a rooftop in New York City, veteran photographer Douglas Dubler watched the sun's angle shift across the urban landscape, his internal cl...

02/03/2026

ZEISS Expands Otus ML Lineup with the New 1435

ZEISS is expanding its Otus ML lens family with the introduction of the new ZEISS Otus ML 1.4/35. This manual-focus lens is designed for photographers who live ...

02/03/2026

Matthews Intros White FlexiMount and Phone Mount Kit by D...

Matthews Studio Equipment now offers the new Dinkum Systems White FlexiMount and Phone Mount Kit, a compact, highly adaptable setup for mobile content creation...

02/03/2026

SES and Africa Mobile Network Reach Milestone: Connecting 5.8 Million in DRC, Expanding Rural Coverage

Luxembourg, February 26, 2026 - SES and Africa Mobile Network (AMN) have expande...

02/03/2026

Ausfilm and Screen Australia launch joint UK market initiative

02 03 2026 - Media release Ausfilm and Screen Australia launch joint UK market initiative Ausfilm and Screen Australia have announced Partner with Australia (...

02/03/2026

Saturday Night Live UK announces writing team

Monday 2 March 2026 Saturday Night Live UK announces writing team L-R: Jonno Johnson; Charlie Skelton; Celya AB; Omar Badawy; Gr inne Maguire; Laura Claxton; ...

02/03/2026

Netflix Unveils the Trailer for '53 Sundays', the New Film by Cesc Gay

Back to All News Netflix Unveils the Trailer for 53 Sundays, the New Film by Cesc Gay Entertainment 02 March 2026 GlobalSpain Link copied to clipboard Dis...

28/02/2026

FOX Sports Waves Green Flag on 2026 IndyCar Season With Driver's Eye, Heads Up Display, Live Drones

With two features seen in Formula 1 coverage, the broadcaster aims to bring view...

28/02/2026

Secretary of War Pete Hegseth Visits L3Harris Solid Rocket Motor Site

Secretary of War Pete Hegseth addresses a crowd of approximately 1,500 L3Harris employees in Camden, Arkansas, as part of his Arsenal of Freedom tour....

28/02/2026

2026 NAB Show Expands Sports Summit to Four Days

Share Copy link Facebook X Linkedin Bluesky Email...

28/02/2026

Three TV5Monde FAST Channels Launch on Plex

Share Copy link Facebook X Linkedin Bluesky Email...

28/02/2026

Nielsen Introduces 200+ New Advanced Audience Segments

Share Copy link Facebook X Linkedin Bluesky Email...

28/02/2026

SBE Launches Annual Membership Drive

Share Copy link Facebook X Linkedin Bluesky Email...

28/02/2026

Gray's Fox 12 Inks Deal for Portland Fire And Portland Thorns Games

Share Copy link Facebook X Linkedin Bluesky Email...