Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

19/11/2026

NDI Releases Latest Version With Recording, Playback APIs

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

19/11/2026

Newsmax Asks Court to Block Foxs Acquisition of Roku

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

19/11/2026

HPA to Honor The Odyssey Director Christopher Nolan with Judges Award for Creativity and Innovation

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , er...

19/11/2026

Australian Professional Leagues Renews Partnership with B...

Five-year deployment has automated APL's media supply chain from fixture to archive, managing over 200,000 assets Blue Lucy, the workflow orchestration and...

19/11/2026

Riding the Australian wave through co-pros

Riding the Australian wave through co-pros 11 November 2026 L-R: The Surfer, We Bury the Dead Australian stories are captivating global audiences and attracti...

19/11/2026

Screen Australia and Australian Film Television and Radio School announce the First Nations Producer Fellowship Program 2027

Screen Australia and Australian Film Television and Radio School announce the Fi...

19/11/2026

AES Adds Dialogue Intelligibility to Loudness Website

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

19/11/2026

Rise Awards 2026 Celebrate Inspirational Women in Broadcast

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

19/11/2026

BPS Trial at Dominion Energy Substation Proves Successful

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

19/11/2026

Wisycom and DPA Prove Essential for Demanding and Unpredi...

iStreamU LLC, a rapidly growing production and systems integration company founded by longtime Producer/Director Ivan Gonzalez, has standardized its audio workf...

19/11/2026

Seven Scripps Research scientists named to Clarivate's 2026 list of Highly Cited Researchers

LA JOLLA, CA-Clarivate, a global information and analytics company, named seven ...

18/11/2026

William Kamau: Young Journalist of the Year finalist 2026

William Kamau is an investigative journalist from Kenya. He says witnessing the pain, displacement and killings' that followed the country's 2007 elect...

18/11/2026

Humberto Basilio: Young Journalist of the Year finalist 2026

Humberto Basilio is a freelance science, health and environmental journalist from Mexico. He says that working in one of the most dangerous countries in the wo...

18/11/2026

Aaron Atsahyel: Young Journalist of the Year finalist 2026

Aaron Atsahyel is a multimedia investigative journalist from Nigeria who draws on his own experiences as someone displaced by the Boko Haram insurgency to tell...

18/11/2026

SVG TranSPORT 2026: All Sessions Now Available To Watch on SVG PLAY

Held in New York City, the event explored the latest technologies and trends shaping the future of live-sports transmission and distribution...

18/11/2026

Samsung Takes the Gold at Your Olympic Games Dakar; Galaxy Phones Play Role in Coverage

The Youth Olympic Games Dakar 2026 opened in Senegal on October 31, giving young...

18/11/2026

Eventide launch SplitEQ 2

Dynamic bands, unmasking features & new stereo tools Launched in 2021, Eventide's SplitEQ plug-in introduced an interesting new approach to EQ, with the...

18/11/2026

IK Multimedia announce Tonex Parametric

Parametric Tone Models emulate control responses & interactions IK Multimedia have just announced the launch of Parametric Tone Models, a new technology tha...

18/11/2026

Shure launch the MV360 Spatial Audio Microphone

Captures immersive audio on phone, tablet or computer via USB-C Shure have just announced the launch of the MV360 Spatial Audio Microphone, a compact USB-C ...

18/11/2026

SBS announces key appointments across programming and commissioning

SBS announces key appointments across programming and commissioning 18 November, 2026 Media releases SBS has announced a series of appointments and role up...

18/11/2026

The National Film and Video Foundation (NFVF) Call for Festival and Market Hosting Funding Applications is Open

The National Film and Video Foundation (NFVF), an agency of the Department of Sp...

18/11/2026

DIRECTV teams with Gracenote to enhance the sports programming experiences that fans want

Strategic renewal builds on existing engagement, supports DIRECTV's objectiv...

18/11/2026

SMPTE Publishes Engineering Report on Content Authenticity

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Senate Commerce Committee Approves Thumann Severs for FCC Commissioner

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

wTVision Qualifies NEOiD PTZ Cameras for Virtual Sets, AR

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Spanish Media Group Deploys Lawo IP Audio Infrastructure

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Library of American Broadcasting Foundation Honors 2026 Giants of Broadcasting & Electronic Arts

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , er...

18/11/2026

Monthly Subscription Streaming Service Prices Up Just 8% Since 2022

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Spectrum News Streamlines Workflow With Amagi

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Media Links Joins Panasonic Kairos Partner Program

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Signiant Acquires Burnlab's Ceivo AI-Driven Media Platform

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Democrats Urge Delay of Thumann Severs FCC Confirmation

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Hisong Unveils AirStudio Muse Audio Capture System

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Spectrum Reach Taps Debbie Presser to Lead Sales at AdLink New York

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

Nationwide EAS Test Runs Without Apparent Hiccups

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

18/11/2026

SMPTE Identifies Key Requirements for Content Provenance...

SMPTE, the home for media professionals, technologists and engineers, today announced the publication of an engineering report with the summarized results of an...

18/11/2026

Live from London, its Saturday night! Saturday Night Live UK returns 16 January on Sky and NOW

Wednesday 18 November 2026 Live from London, it's Saturday night! Saturday ...

18/11/2026

Sky Kids Investigates: Being a Boy

BAFTA award-winning series sees 16-year-old reporter explore topics including toxic digital influencers and disrespectful attitudes to girls in a groundbreaking...

18/11/2026

Choosing a display for TMxCore

TMxCore has no built-in display. If you use TMxCore with a network display, you may not need a dedicated display at all. If you prefer a directly connected scre...

18/11/2026

TeleData Accelerates XGS-PON Service Rollout in Germany and Unifies its Network with Harmonic's cOS Broadband Platform

Key Takeaways TeleData is building a unified DOCSIS and fiber network with Ha...

18/11/2026

Beeble launches SwitchX 3.0 with dynamic environments and greater creative control

Beeble recently launched SwitchX 3.0, a major update to its AI-powered video-to-...

18/11/2026

Film London to lead new industry PhD program in partnership with Brunel University

Film London will lead and manage a long-term industry research program in partne...

18/11/2026

Hexagon launches MANTIS to capture 3D scene data as cameras roll

Hexagon, the global leader in measurement technologies, recently launched MANTIS by Leica Geosystems, a camera-mounted 3D capture and tracking system to give fi...

18/11/2026

Successfully achieved BSI C5 attestation

Arvato Systems Expands BSI-C5 Testing Greater Confidence in Cloud Security and Regulatory Resilience Successful BSI-C5 Retest Certification in accordance wi...

18/11/2026

RT to premiere new three-part documentary series Able

Brendan O'Connor joins six disabled job seekers as they navigate the barriers of finding work and challenge perceptions of disability in the workplace Begi...

18/11/2026

When writing becomes a weapon : New true-crime podcast series to reveal the impact of anonymous poison-pen letter case

Poison Ink begins Friday 20 November 2026 on RT Listen and RT Radio 1 Listen ...

18/11/2026

RT Choice Music Prize 2027 Key Dates Announcement

RT Choice Music Prize In association with IMRO and IRMA 2027 KEY DATES Irish Album of the Year 2026 Shortlist 18th January Irish Artist of the Year 2026 ...

18/11/2026

NVIDIA RTX Spark Compact Desktops Out Now

New NVIDIA RTX Spark compact desktops from ASUS, Dell and MSI are available today, bringing local AI agents, image and video generation, and model fine-tuning t...

17/11/2026

Fabric Insights and Fonn Group Partner to Connect Newsroom Planning and Media Operations

Fabric Insights and Fonn Group have announced a partnership to connect editorial...