Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

20/08/2026

NEP's Summer of Soccer Didn't End at World Cup: 1,100+ Domestic Productions Follow the Final

From host-broadcaster and rightsholder support in June and July to MLS, NWSL, US...

20/08/2026

SVG GameDay, Ep. 27: San Jose Sharks' Emily Garner - Production Tales From the Shark Tank

In-venue and creative video staffers at the professional and collegiate level ha...

20/08/2026

IBC 2026: Veritone To Showcase Enterprise AI and Media Supply Chain Automation

At IBC this year, Veritone (14.A56) is bringing two connected themes to the booth: architectural independence in enterprise AI, and the automation of the media ...

20/08/2026

Telestream and ScorePlay Partner to Connect Live Sports Ingest with Media Management

Telestream and ScorePlay have announced a partnership combining Telestream's...

20/08/2026

Grand Slam of Curling and Rock League Sign Multi-Year Deal with ESPN

The Grand Slam of Curling and Rock League have announced a three-year agreement with ESPN, bringing more than 300 hours of live curling coverage to U.S. audienc...

20/08/2026

Nielsen Announces Measurement Enhancements Ahead of Fall TV Season

Nielsen has announced a series of enhancements to its Big Data + Panel currency measurement methodology, taking effect August 31. The updates were developed in ...

20/08/2026

ORF Deploys Net Insight Nimbra Platform for Nationwide IP Contribution Network

Net Insight has been selected by service provider Big Blue Marble (formerly ORS Group) to provide its Nimbra platform as the core media transport solution for a...

20/08/2026

2026 US Open Launches Fan Activations Across New York City

The 2026 US Open will bring a series of free fan activations across New York City ahead of and during the tournament, which runs August 23-September 13. We...

20/08/2026

Bridge Technologies Adds TS Playout to VB330 and VB220 Monitoring Probes

Bridge Technologies will announce a transport stream (TS) playout feature for its VB330 and VB220 compressed monitoring probes at IBC2026 (Booth 1.A71). The fe...

20/08/2026

IBC 2026: ENCO To Showcase AI and Automation Solutions

ENCO will demonstrate solutions across captioning, translation, AI production tools, automation, and virtual production at IBC2026 (Stand 7.A34, Amsterdam RAI, ...

20/08/2026

IBC 2026: MultiDyne MRMC To Exhibit for First Time as Part of MultiDyne Organization

MultiDyne Video and Fiber Optic Systems will demonstrate camera robotics and mot...

20/08/2026

IBC 2026: Chyron Showcases Newsroom Workflow Solutions

Chyron will demonstrate news workflow solutions at IBC2026 (Stand 7.A23), marking the company's 60th anniversary. Highlights include updates to PRIME HTML, ...

20/08/2026

Spectrum SportsNet LA Now Available to Customers in Former Cox Markets

Spectrum SportsNet LA, the exclusive television home of the Los Angeles Dodgers, is now available to Spectrum customers in former Cox markets following the clos...

20/08/2026

IBC 2026: swXtch.io Launches Expanded SRT Distribution and Native Multicast Services

swXtch.io will unveil expanded Global SRT Distribution and native multicast tran...

20/08/2026

Transformation Church Deploys Riedel Artist Intercom System

Riedel Communications has announced that Transformation Church in the United States has deployed a communications system built around the Artist intercom ecosys...

20/08/2026

Apple at 50: Author and CBS Sunday Morning Correspondent David Pogue on How a Tech Giant Became a Media Powerhouse

The bestselling author and longtime Apple observer explores how product launches...

20/08/2026

Orlando Magic's David Storm Reflects on the Franchises First Year With Its New ST 2110 Control Room and Massive Center-Hung Videoboard

The Orlando Magic entered last season with a dramatically upgraded in-venue prod...

20/08/2026

Indiana Pacers Name DAZN Exclusive Streaming Home Beginning 2026-27 Season

Pacers Sports and Entertainment has announced a partnership with DAZN, making it the exclusive streaming home of Indiana Pacers basketball. A team-branded strea...

20/08/2026

Ratings Roundup: Scotty Scheffler Win Tops the Week, ESPN Posts Most-Watched WNBA Game Ever on Cable

Ratings Roundup is a rundown of recent ratings news and is derived from press re...

20/08/2026

NHL Announces Germany Expansion with FC Bayern Partnership and 2027 Global Series Games in Munich and Cologne

The NHL has announced two major developments tied to its Germany growth strategy...

20/08/2026

SoundBridges TURN server tutorial

Set up your own private remote collaboration infrastructure SoundBridge have recently put together a tutorial video that provides users with a step-by-step ...

20/08/2026

Neyrinck revive Avid D-Control surfaces

New V-Control Pro features restore basic functionality Neyrinck's V-Control software has long been keeping older Avid control surfaces compatible with n...

20/08/2026

ROSE AYLING-ELLIS BEGINS FILMING SERIES TWO OF ITVS BAFTA-WINNING THRILLER CODE OF SILENCE

TV, BritBox and Mammoth Screen have announced that filming is now underway on th...

20/08/2026

Nielsen Incorporates New Enhancements To Improve Its Data Measurement Leading Into The New Fall TV Season

New York August 19, 2026 Nielsen today shared a number of key enhancements t...

20/08/2026

FAST Sports Programming Surges at More Than Twice the Rate of Sports Channel Growth

New Gracenote Data Hub analysis finds sports program titles up 31.2% and individ...

20/08/2026

Charter Completes $34.5 Billion Cox Acquisition

Share Copy link Facebook X Linkedin Bluesky Email...

20/08/2026

SubtitleNEXT makes a significant leap in localisation eff...

At IBC, Profuz Digital will showcase at stand 8.A88 the SubtitleNEXT platform delivering innovative AI transcription integrations and stunning new features boos...

20/08/2026

Transformation Church Elevates Production Communications...

Riedel Communications today announced that Transformation Church, one of the fastest-growing and most influential digital ministries in the United States, has d...

20/08/2026

QuickLink Showcases New 4K Capabilities Across StudioEdge and StudioCall at IBC 2026

QuickLink Showcases New 4K Capabilities Across StudioEdge and StudioCall at IBC ...

20/08/2026

The Boys Season Five Graded with DaVinci Resolve Studio

The Boys Season Five Graded with DaVinci Resolve Studio Brie Clayton August 20, 2026 0 Comments Colorist Siggy Ferstl uses Depth Map to bring a three ...

20/08/2026

Adobe Firefly expands its creative AI studio: generate music, speech, and sound effects in one place

Adobe Firefly expands its creative AI studio: generate music, speech, and sound ...

20/08/2026

Broadpeak launches Multiview as a Service to accelerate m...

Multiview available via SaaS for streaming platforms and operators through broadpeak.io, with live demonstrations at IBC2026 Broadpeak , a leader in streaming ...

20/08/2026

Techex Appoints Stuart Almond as Chief Revenue Officer

Techex, a leading provider of live broadcast infrastructure solutions, has appointed Stuart Almond as Chief Revenue Officer, further strengthening its executive...

20/08/2026

MediaCity Studios upgrades with Avid

Avid today announced that MediaCity Studios has upgraded its production infrastructure with an enhanced end-to-end Avid production ecosystem, including the dep...

20/08/2026

MS Now to Launch $7.99 DTC Service

Share Copy link Facebook X Linkedin Bluesky Email...

20/08/2026

Nielsen Details Improvements to Audience Measurement Capabilities

Share Copy link Facebook X Linkedin Bluesky Email...

20/08/2026

Study: 174 Billion Illegal Streams During FIFA World Cup 2026

Share Copy link Facebook X Linkedin Bluesky Email...

20/08/2026

Prasad Opens New 8K/4K Film Preservation & Restoration Center

Share Copy link Facebook X Linkedin Bluesky Email...

20/08/2026

IPC Expands Brazilian Presence with New Channel Partner C...

IPC Expands Brazilian Presence with New Channel Partner CCATech Industry veterans Carlos Abrah o and Bruno Lopes strengthen local support and expand IPCs prese...

20/08/2026

Zixi Enters Mauritius Market Through Strategic Deployment...

Zixi Enters Mauritius Market Through Strategic Deployment with MTC Jockey Club Worlds Oldest Horse Racing Club in the Southern Hemisphere Selects Zixis Softwar...

20/08/2026

Scripps Research scientist to receive $2.76 million NIAID New Innovators Award

LA JOLLA, CA-Scripps Research assistant professor Tiantian Liu has received a five-year NIAID DP2 award from the National Institute of Allergy and Infectious Di...

20/08/2026

Audiences reveal appetite for multicultural Australia on-screen

Audiences reveal appetite for multicultural Australia on-screen 20 August 2026 Dr Georgie McClean, Dr Rebecca Huntley (L-R) By Dr Georgie McClean and Dr Rebec...

20/08/2026

How Mavis Studio Is Expanding Live Production Teaching at Sussex

How Mavis Studio Is Expanding Live Production Teaching at Sussex Brie Clayton August 19, 2026 0 Comments Hero image displays a student about to mix be...

20/08/2026

Vertical Unscripted Series Legal Queens Captured with Blackmagic Design

Vertical Unscripted Series Legal Queens Captured with Blackmagic Design Brie Clayton August 19, 2026 0 Comments Reality TV veteran Eric Gardner create...

20/08/2026

Transformation Church Elevates Production Communications With Riedel Artist and Bolero

Wuppertal August 20, 2026 Transformation Church Elevates Production Communicat...

20/08/2026

The music between the notes

The music between the notes Coming together In the middle of 2026, as conversations around artificial intelligence continue to raise questions about how musi...