Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

North America Stories

17/04/2026

IAB: Digital Ad Revenue Hit Nearly $300 Billion in 2025

Share Copy link Facebook X Linkedin Bluesky Email...

17/04/2026

PSSI Global Acquires Beagle Networks

Share Copy link Facebook X Linkedin Bluesky Email...

17/04/2026

FCC Commissioner Gomez to Speak at LPTV Day at 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

17/04/2026

swXtch.io Introduces the swXtch AI Router for Live Media Workflows

Share Copy link Facebook X Linkedin Bluesky Email...

17/04/2026

Bittree Returns to NAB Show with Diverse Patching Solutions

Share Copy link Facebook X Linkedin Bluesky Email...

17/04/2026

Weigel Broadcasting Taps OpenAP for Advanced Advertising Tools

Share Copy link Facebook X Linkedin Bluesky Email...

17/04/2026

Dubformer and Adapt Power Goalcast Expansion into Global...

Dubformer, the AI-powered dubbing platform, and Adapt, a global leader in AI-driven localization and dubbing solutions, have enabled Goalcast, the inspirational...

17/04/2026

New Mavis Studio App for iPad Redefines Mobile Live Produ...

At NAB 2026, Mavis will launch Mavis Studio, a new live production app for iPad that gives creators, broadcasters, and production teams a powerful new way to bu...

17/04/2026

Avid and Google Cloud Announce Partnership to Bring Agent...

Avid and Google Cloud today announced a multi-year strategic partnership to integrate generative and agentic AI into the media and entertainment industry's...

17/04/2026

LucidLink brings the most complete file streaming platfor...

At NAB Show, LucidLink will showcase how its platform brings content, people and workflows closer together than ever before, with newly available capabilities a...

17/04/2026

Beth Morrison and Moses Pendleton to be Honored at Boston Conservatory at Berklee's Commencement Ceremony

Beth Morrison and Moses Pendleton to be Honored at Boston Conservatory at Berkle...

17/04/2026

Jacob Collier, Jill Scott, and Vinnie Colaiuta to Receive Honorary Doctorates at Berklee College of Music's 2026 Commencement

Jacob Collier, Jill Scott, and Vinnie Colaiuta to Receive Honorary Doctorates at...

16/04/2026

NAB 2026: Appear Announces X5 General Availability, XM Management Tool, and SIx300 Module

Appear ASA (OSE: APR) will announce three additions to its X Platform at NAB Sho...

16/04/2026

CentralCast Deploys Harmonic XOS Media Processor for Public Broadcasting

Harmonic has announced that CentralCast, a centralized master control facility for U.S. public media, has deployed Harmonic's XOS Advanced Media Processor t...

16/04/2026

NAB 2026: Encompass Digital Media Integrates Interra Systems BATON into Cloud-Based Altitude Flow Platform

Interra Systems has announced that Encompass Digital Media has integrated its BA...

16/04/2026

NAB 2026: Grass Valley to Showcase Expanded Media Infrastructure Capabilities

Grass Valley will demonstrate its Media Infrastructure capabilities at NAB Show 2026 (Booth C2408, Central Hall), bringing routing, signal processing, and orche...

16/04/2026

Verizon Readies Robust Network Infrastructure Plans for FIFA World Cup 2026

As preparations ramp up for the FIFA World Cup 2026, Verizon has outlined a sweeping connectivity and infrastructure initiative that will underpin broadcast ope...

16/04/2026

Advanced Image Robotics Joins 2026 MLS Innovation Lab

Advanced Image Robotics (AIR) has announced its selection for the 2026 MLS Innovation Lab. AIR will work with MLS clubs, players, and executives on automated vi...

16/04/2026

NAB 2026: JWX Acquires True Anthem, Adding AI-Powered Social Publishing to Publisher Platform

JWX has announced the acquisition of True Anthem, an AI-powered social publishin...

16/04/2026

Jomboy Media and Fubo Launch 24/7 Creator-Led Channel

Jomboy Media and FuboTV Inc. have launched the Jomboy Media Channel, a 24/7 channel available to FuboTV base plan subscribers. The channel, timed to the start o...

16/04/2026

NAB 2026: Wave Central and EVS Announce SMPTE ST 2110 Interoperability Validation

Wave Central, a Domo Broadcast Company (Booth C2820), and EVS Broadcast Equipmen...

16/04/2026

Tata Communications and Formula 1 Release Race Before the Race, First Film in New Content Series

Tata Communications and Formula 1 have released Race Before the Race, the firs...

16/04/2026

NAB 2026: DPA Microphones N-Series Firmware Update Adds Duplex Gap and Guard Band Access for North American Users

DPA Microphones has released a firmware update for its N-Series Digital Wireless...

16/04/2026

NAB 2026: See Sony's Live Stage Presentations at NAB Show 2026

Sony's Live Stage at NAB Show 2026 is the place to hear directly from the content creators, end users, and technology experts who are pushing boundaries in ...

16/04/2026

Perfect Game and Youth Prospects Announce Broadcast Rights and Content Partnership

Perfect Game and Youth Prospects have announced a partnership covering broadcast...

16/04/2026

National Collegiate Rugby Partners with All Womens Sports Network for 2026 National 7s Championships

National Collegiate Rugby (NCR) has announced a media rights partnership with Al...

16/04/2026

NAB 2026: Audio-Technica Introduces BP350ST-UB and BP350ST-UL Mid-Side Stereo Broadcast Microphones

Audio-Technica has announced two new mid-side (MS) stereo broadcast microphones:...

16/04/2026

NAB 2026: PSSI Global Services Acquires Beagle Networks

PSSI Global Services has announced the acquisition of Beagle Networks, a provider of IT infrastructure and onsite technical support for media and enterprise cus...

16/04/2026

NAB 2026: Blackmagic Design Releases Blackmagic Camera for iOS 3.3

Blackmagic Design has released Blackmagic Camera for iOS 3.3, a free update available now from the Apple App Store. The update will be demonstrated at NAB Show ...

16/04/2026

SVG New Sponsor Spotlight: 4Wall Entertainment's Dave Caulwell on Scaling Live Event Production for Sports

As live sports production continues to expand across linear, digital, and in-ven...

16/04/2026

ESPN Enters First HDR Postseason With REMCO Support in New Bristol-Based Control Rooms

Part of an infrastructure upgrade, the recently constructed spaces accommodate n...

16/04/2026

NBC Sports Tips Off First NBA Playoffs Campaign in 23 Years With Six-Truck Fleet, Interchangeable Production Model

NBC has added NEP's Supershooter 11, which only just came online in time for...

16/04/2026

Reimagining Earth Measurement for the AI Era: L3Harris and Xoople Develop a New Spaceborne Capability

For decades, understanding the physical world from space has required a trade-of...

16/04/2026

Locality and Nielsen Announce Landmark Integration of Media Data Engine, Transforming Local TV Measurement

The integration accelerates demographic audience delivery across local markets a...

16/04/2026

Quickplay Deploys Gray Medias New Streaming Platform

Share Copy link Facebook X Linkedin Bluesky Email...

16/04/2026

Spectrum TV App Launches On Google TV and Other Android TV Devices

Share Copy link Facebook X Linkedin Bluesky Email...

16/04/2026

Telestream Enables Multi-Cloud Media Workflows with Oracl...

Telestream Cloud Services, including Vantage Cloud, UP, and SENTRY monitoring tools, are now optimized for OCI, powering flexible multi-cloud media orchestratio...

16/04/2026

Fox Corporation Names Amazon Web Services its Preferred A...

Fox Corporation (Nasdaq: FOXA, FOX; FOX or the Company ) today announced a strategic collaboration with Amazon Web Services (AWS), naming AWS as its preferre...

16/04/2026

Triveni Digital Introduces New StreamScope Analyzer to Br...

Triveni Digital, a trusted leader in ATSC 1.0 and 3.0 service delivery, data broadcasting, and quality assurance solutions, today announced an ISDB-Tb capabilit...

16/04/2026

Synamedia and SoFast team up to accelerate FAST and OTT

Synamedia and SoFast announce strategic go-to-market partnership to accelerate FAST, pay-TV, and VOD At the NAB Show 2026, Synamedia and SoFast are announcing ...

16/04/2026

Imagine Builds on a Decade of ST 2110 Leadership at 2026...

Showcases New, Open-Standard IP Solutions Across Its Portfolio, From Production to Playout Imagine Communications is marking a decade of leadership in ST 2110 ...

16/04/2026

Roku Marks 100M Milestone

Share Copy link Facebook X Linkedin Bluesky Email...

16/04/2026

Sportway and Broadcast Solutions acquire Studio Automated...

Sportway Media Group, a world leading AI-automated sports production company and Broadcast Solutions, a leading system integrator and provider of innovative sol...

16/04/2026

AJA to Acquire Video Encoding Software Company Comprimato

Share Copy link Facebook X Linkedin Bluesky Email...

16/04/2026

NAB Show 2026: Sony Announces New Cameras, Virtual Production Tools

Share Copy link Facebook X Linkedin Bluesky Email...

16/04/2026

Synamedia launches GO Shorts

Share Copy link Facebook X Linkedin Bluesky Email...

16/04/2026

Clear-Com Introduces Arcadia and Eclipse HX Updates

Share Copy link Facebook X Linkedin Bluesky Email...

16/04/2026

AJA Enters into Agreement to Acquire Video Encoding Software Company Comprimato

AJA Enters into Agreement to Acquire Video Encoding Software Company Comprimato Brie Clayton April 15, 2026 0 Comments Deal will expand AJA's video ...

16/04/2026

Deity Announces PR-4 Compact Field Recorder with Pre-Orders Launching April 14

Deity Announces PR-4 Compact Field Recorder with Pre-Orders Launching April 14 Brie Clayton April 15, 2026 0 Comments Deity Microphones today announce...

16/04/2026

Berklee's First Homecoming Brings Global Alumni to Boston for a Block Party Celebration

Berklee's First Homecoming Brings Global Alumni to Boston for a Block Party ...