Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

18/06/2026

Ratings Roundup: Knicks-Spurs NBA Finals Is Most Watched Since Jordan Bulls Era; FIFA World Cup Opens Big

Ratings Roundup is a rundown of recent rating news and is derived from press rel...

18/06/2026

InfoComm 2026: PTZOptics Debuts Healthcare Visual Reasoning Integration with LayerJot

PTZOptics has unveiled new Visual Reasoning demonstrations at InfoComm 2026 (Boo...

18/06/2026

IBC 2026: Announces New Future Tech Ignite Initiative, Expanded Exhibitor Participation

IBC2026 will take place at the RAI Amsterdam from September 11-14, bringing toge...

18/06/2026

InfoComm 2026 Holds Inaugural Media Day with Product Announcements from 19 Exhibitors

InfoComm 2026 held its first-ever Media Day on June 17, providing journalists an...

18/06/2026

Info Comm 2026: FOR-A America Announces TAA-Compliant LED Display Package

FOR-A America has announced a Trade Agreements Act (TAA)-compliant LED display solution combining Alfalite's Litepix LED displays and Brompton Technology...

18/06/2026

SVG GameDay, Ep. 20: Cleveland Browns Kyle Millen - Run of Shows & Rock n Roll

In-venue and creative video staffers at the professional and collegiate level have one major thing in common: the intensity and attention to detail ramps up dur...

18/06/2026

International Federation of American Football, TMRW Sports Partner on Global Growth of Flag Football

The International Federation of American Football (IFAF) and TMRW Sports have an...

18/06/2026

AJA Debuts Io Xpand Thunderbolt 5 Expansion Chassis for KONA and Corvid PCIe I/O Cards

AJA Video Systems has unveiled Io Xpand, a Thunderbolt 5-enabled PCIe expansion ...

18/06/2026

ESPN Marks 30th Anniversary of WNBA's Inaugural Game With Liberty-Sparks Broadcast

ESPN has announced its coverage plans for the 30th anniversary of the WNBA's...

18/06/2026

FOX Sports' Big Noon Kickoff Heads to London for Union Jack Classic From Wembley Stadium

FOX Sports' Big Noon Kickoff will broadcast live from Wembley Stadium in Lon...

18/06/2026

InfoComm 2026: Show Opens With Microsoft Keynote, Media Day, and New Industry Initiatives

InfoComm 2026 opened on Wednesday at the Las Vegas Convention Center, bringing t...

18/06/2026

SVG New Sponsor Spotlight: Akta's Matt Smith on Building AI Into the Foundation of the Video Workflow

As media companies look to deliver more live, VOD, and snackable sports content ...

18/06/2026

2026 Sundance Film Festival: Local Lens

Top L-R: Take Me Home, The Lake Bottom L-R: TheyDream, Union County Free Summer Screening Series Announced Screenings for the Local Utah Community at...

18/06/2026

iamReverb gets an update

Improvements & new IR content iamReverb Audio have just launched a free update that kits their convolution reverb plug-in out with some new features and int...

18/06/2026

Two notes unveil Genome 2.0

Modelling suite gains improved captures, iOS support & more Two notes Audio Engineering have just announced the launch of Genome 2.0, a significant update t...

18/06/2026

VSL update Vienna Ensemble Pro 8

New AI assistance feature, video overhaul & more VSL have just announced the launch of Vienna Ensemble Pro 8.1 and 8.1V, a pair of major updates to their ev...

18/06/2026

The Gauge: Poland | May 2026

The average daily TV screen time in May was 3 hours and 36 minutes, marking a clear decrease of 15 minutes compared to April. This trend proved to be much stron...

18/06/2026

Gracenote and PubMatic Bring Curated Live Sports and Content-Level Deals to Programmatic CTV

Integration embeds Gracenote content intelligence, including contextual segments...

18/06/2026

AJA Debuts Io Xpand Thunderbolt 5 Expansion Chassis for...

AJA Video Systems unveiled Io Xpand, a high-performance Thunderbolt 5-enabled PCIe expansion chassis for AJA KONA and Corvid video and audio I/O cards. As dema...

18/06/2026

Providius Announces Providius Direct for Faster AV-over-I...

New NVRT-driven workflow enables on-demand traffic mirroring and guided troubleshooting for AV teams, integrators, and support organizations ahead of InfoComm 2...

18/06/2026

Mavis Studio Brings New Muscle to Mobile Production on iP...

At InfoComm 2026, Mavis announced a major update to Mavis Studio, its live production app for the iPad, with new features designed to make professional AV produ...

18/06/2026

Harmonic Completes Divestiture of Video Business to Media...

Transaction Positions Harmonic as a Pure-Play Broadband Company Harmonic Inc. (NASDAQ: HLIT), the worldwide leader in virtualized broadband solutions, today a...

18/06/2026

ACT Entertainment and NETGEAR Partner to Power the Futur...

ACT Entertainment and NETGEAR have announced a strategic partnership that establishes verified interoperability between NETGEAR Switches and key technologies fr...

18/06/2026

IBC2026 creates new pathways from innovation to real-worl...

IBC2026 is set to bring the global media, entertainment and technology community together at the RAI Amsterdam from 11 14 September, enabling industry players f...

18/06/2026

PLIANT TECHNOLOGIES DEBUTS CREWCOM FLEX THE WORLDS FIRST...

Pliant Technologies is proud to introduce CrewCom Flex , the world's first frameless matrix intercom system and the only complete matrix IP-based intercom s...

18/06/2026

XR Sports Alliance Strengthens its Capabilities with New...

The XR Sports Alliance (XRSA) has announced that a new cohort of members has joined the strategic initiative: ActionStreamer, Antigravity, Creative Artists Agen...

18/06/2026

Hollywood Filmmakers Launch AI Production Platform Cascade

Share Copy link Facebook X Linkedin Bluesky Email...

18/06/2026

ATVA Blasts Deltavision Media for Demanding 'Egregious' Retrans Fees

Share Copy link Facebook X Linkedin Bluesky Email...

18/06/2026

MediaKind Completes Merger with Harmonic's Video Business

Share Copy link Facebook X Linkedin Bluesky Email...

18/06/2026

AJA Unveils Io Xpand Thunderbolt 5 Expansion Chassis At InfoComm 2026

Share Copy link Facebook X Linkedin Bluesky Email...

18/06/2026

Linda May Han Oh Receives Guggenheim Fellowship

Linda May Han Oh Receives Guggenheim Fellowship The Berklee professor, bassist, and composer will use the fellowship to debut Dreams of Knowing, an interdisci...

18/06/2026

Techtel delivers Haivision live streaming system for TAFE NSW St Leonards, enabling real-world broadcast training

Techtel delivers Haivision live streaming system for TAFE NSW St Leonards, enabl...

18/06/2026

The Great Roaming Rinse

New study reveals 10 hidden data drainers costing Brits hundreds abroad - and the holiday hotspots where you could get rinsed the mostThursday 18 June 2026 The...

18/06/2026

How to watch the 2026/27 Scottish Premiership season on Sky Sports

Thursday 18 June 2026 How to watch the 2026/27 Scottish Premiership season on Sky Sports Which matches are Sky Sports showing on the 2026/27 Scottish Premiers...

18/06/2026

Sky Sale: Latest deals now on, with discounts on iPhone Air & 2.5Gbps speeds

Thursday 18 June 2026 Sky Sale: Latest deals now on, with discounts on iPhone Air & 2.5Gbps speeds The latest deals have dropped from Sky Mobile, the award-wi...

18/06/2026

FOX Advertising and Toonstar Team to Create New Opportunities for Brands in Digital-First Animation

FOX Advertising and Toonstar Team to Create New Opportunities for Brands in Digi...

18/06/2026

Arqiva secures WTA Tier-4 accreditation

Arqiva's Crawley Court and Chalfont Grove teleports re-certified at highest World Teleport Association standard 18 June 2026, Winchester, UK - Arqiva, the ...

18/06/2026

Apple announces changes to iOS in Brazil - UPDATE - Posted on 18 June 2026

Apple today announced changes impacting iOS apps in Brazil that reflect a recent agreement with Brazil's competition regulator, the Conselho Administrativo ...

18/06/2026

Sync and Stream: GeForce NOW Connects to Members' Game Libraries Across Devices

Play favorite titles from popular game libraries, keep progress synced and jump ...

18/06/2026

At Cannes Lions, NVIDIA Partners Reshape Advertising and Marketing With AI

The digital era gave the advertising and marketing industry speed; the AI era is giving it autonomous operations. For companies building next-generation techn...

17/06/2026

EVS Achieves EcoVadis Gold Medal, Ranking in Top 5% of Companies Globally

EVS has announced it has received the EcoVadis Gold Medal for sustainability performance, ranking among the top 5% of companies globally in the Technology/Mid-S...

17/06/2026

Chyron Releases Weather 2.4 with Updated DataFlow Module

Chyron has released Weather 2.4, an update to its weather suite for broadcasters and meteorologists. The release focuses on enhancements to the DataFlow module,...

17/06/2026

VSiN Launches Best Bets TV, a 24/7 FAST Channel

VSiN, The Sports Betting Network, has announced the launch of Best Bets TV, a free ad-supported streaming TV (FAST) channel. The 24/7 channel is currently avail...

17/06/2026

Snapchat and Team Whistle Launch World Cup Creator Program Across Miami, New York, and Los Angeles

DAZN's Team Whistle and Snap Inc. have announced a creator program centered ...

17/06/2026

LiveU Supporting Broadcast and Public Safety Operations for Summer of Soccer

LiveU is providing video transmission technology for broadcasters, production companies, and public safety agencies across North America's busy Summer of So...

17/06/2026

NATAS Announces Two New Board Members and COO Promotion

The National Academy of Television Arts and Sciences (NATAS) has announced that Laurens Grant and Jacob Ullman have joined its Board of Directors. Chief of Staf...