Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

North America Stories

18/06/2026

Linda May Han Oh Receives Guggenheim Fellowship

Linda May Han Oh Receives Guggenheim Fellowship The Berklee professor, bassist, and composer will use the fellowship to debut Dreams of Knowing, an interdisci...

17/06/2026

EVS Achieves EcoVadis Gold Medal, Ranking in Top 5% of Companies Globally

EVS has announced it has received the EcoVadis Gold Medal for sustainability performance, ranking among the top 5% of companies globally in the Technology/Mid-S...

17/06/2026

Chyron Releases Weather 2.4 with Updated DataFlow Module

Chyron has released Weather 2.4, an update to its weather suite for broadcasters and meteorologists. The release focuses on enhancements to the DataFlow module,...

17/06/2026

VSiN Launches Best Bets TV, a 24/7 FAST Channel

VSiN, The Sports Betting Network, has announced the launch of Best Bets TV, a free ad-supported streaming TV (FAST) channel. The 24/7 channel is currently avail...

17/06/2026

Snapchat and Team Whistle Launch World Cup Creator Program Across Miami, New York, and Los Angeles

DAZN's Team Whistle and Snap Inc. have announced a creator program centered ...

17/06/2026

LiveU Supporting Broadcast and Public Safety Operations for Summer of Soccer

LiveU is providing video transmission technology for broadcasters, production companies, and public safety agencies across North America's busy Summer of So...

17/06/2026

NATAS Announces Two New Board Members and COO Promotion

The National Academy of Television Arts and Sciences (NATAS) has announced that Laurens Grant and Jacob Ullman have joined its Board of Directors. Chief of Staf...

17/06/2026

Akta Brings AI-First Cloud Video Workflows to Oracle Cloud Infrastructure

Akta, the AI-First SaaS video platform for modern broadcast and streaming operations, today announced that its video platform is now generally available on Orac...

17/06/2026

SVG Students to Watch: Nicholas Stafford, Texas Southern University

This recent graduate from Houston found inspiration in technical directing and now eyes a future career in sports production...

17/06/2026

InfoComm 2026: Audio-Technica Brings New Wireless, Networked Audio, and Certified Conferencing Solutions

Audio-Technica (booth C7959) arrives at InfoComm 2026 in Las Vegas with a slate ...

17/06/2026

SNS Positions AI Suite as On-Premise Answer to Video Indexing and Search Demands

SNS has published a guide addressing growing demand for AI-powered video indexing, transcription, facial recognition, and searchable metadata across media libra...

17/06/2026

Providius Announces Providius Direct for Network Troubleshooting on NETGEAR Pro AV Switches

Providius has announced Providius Direct, a workflow for investigating network i...

17/06/2026

NEP Launches NEP Platform Software Orchestration System, Selects Bridge Technologies VB440

NEP Group has announced the commercial availability of NEP Platform, a software ...

17/06/2026

BBright White Paper: SRT vs. RIST for Professional Video Contribution and Distribution

A new white paper examining Secure Reliable Transport (SRT) and Reliable Interne...

17/06/2026

Bango Research: 51% of Gen Z Say Highlights Are Replacing Live Games

New research from subscription bundling platform Bango finds that younger sports fans are increasingly consuming sport through highlights, clips, and social med...

17/06/2026

SMPTE Opens Entire Standards Library to Public at No Cost

SMPTE has announced that its complete Standards catalog is now freely available to the global media technology community, including all published SMPTE Standard...

17/06/2026

MediaKind Completes Merger with Harmonic's Video Business, Creating Independent Video Infrastructure Heavyweight

Harmonic has completed the sale of its Video Business to MediaKind for $145 mill...

17/06/2026

Omaha Productions to Produce 2026 World Series of Poker Main Event Coverage for ESPN

Omaha Productions will produce the 2026 World Series of Poker (WSOP) in Las Vega...

17/06/2026

NABLF Graduates Its 2026 Broadcasting Leadership Training Class

Share Copy link Facebook X Linkedin Bluesky Email...

17/06/2026

ABC and ESPN Score Most-Watched NBA Finals Since 1998

Share Copy link Facebook X Linkedin Bluesky Email...

17/06/2026

Smartly, Roku Bring Social Performance Tools to CTV

Share Copy link Facebook X Linkedin Bluesky Email...

17/06/2026

Pliant Technologies Debuts Crewcom Flex at InfoComm 2026

Share Copy link Facebook X Linkedin Bluesky Email...

17/06/2026

The Immersive Supervisor Emerges as Hollywood's Next Production Role

The Immersive Supervisor Emerges as Hollywood's Next Production Role Brie Clayton June 17, 2026 0 Comments Above image: On set live immersive revi...

17/06/2026

Vertical Musical Playback Shot with Blackmagic PYXIS 6K

Vertical Musical Playback Shot with Blackmagic PYXIS 6K Brie Clayton June 17, 2026 0 Comments Large format sensor and DaVinci Resolve workflow used fo...

17/06/2026

DAZ 3D Launches New Game-Ready Character Assets Built for Modern Engines and Production Workflows

DAZ 3D Launches New Game-Ready Character Assets Built for Modern Engines and Pro...

17/06/2026

Spectrum Awards $1.1 Million in Digital Education Grants

Share Copy link Facebook X Linkedin Bluesky Email...

17/06/2026

XR Sports Alliance Adds New Members

Share Copy link Facebook X Linkedin Bluesky Email...

17/06/2026

AIMS Launches Free Online IPMX Training Series

Share Copy link Facebook X Linkedin Bluesky Email...

17/06/2026

Kiloview Partners with SFM to Expand AV-over-IP Solutions...

Montr al, Quebec, June 11, 2026 Kiloview, a leading provider of AV-over-IP and NDI -based video transmission solutions, today announced a distribution partner...

17/06/2026

Kiloview Launches U4 IP Video Dock Bringing Professional...

Changsha, China, June 15, 2026 Kiloview officially announced the launch of U4 IP Video Dock, a compact IP video decoder and output dock designed to bring prof...

17/06/2026

Half Year, Half Off! Ivory II Summer Sale is Here

Save 50% on All Ivory II Pianos and Collections Were halfway through the year, and we're cutting prices in half for the Ivory II Summer Sale! For a limite...

17/06/2026

Visibility builds credibility - the tools you use every day...

Visibility builds credibility - the tools you use every day, now visible on your LinkedIn profile Published on Jun 17, 2026 Categories: Company News, Product ...

17/06/2026

June 16, 2026

Calibr-Skaggs awarded $5.1M by NIH to develop long-acting hepatitis B virus therapy A new program aims to replace a daily HBV drug with once-monthly or even qua...

16/06/2026

Neumann MT 48 Receives Major Firmware 2.0 Update

Neumann.Berlin has released firmware version 2.0 for the MT 48 audio interface, adding plugin compatibility, expanded Dante networking options, broadcast encode...

16/06/2026

TVNewsCheck Opens Nominations for 2027 Women in Technology Awards

TVNewsCheck has announced that nominations are now open for its 2027 Women in Technology Awards, to be presented at NAB Show 2027 on Tuesday, April 6 in the Med...

16/06/2026

Clear-Com Introduces Avalon IP Intercom Platform

Clear-Com has announced Avalon, a 1RU IP intercom platform for broadcast, live events, and production environments. Designed for IP-only workflows, Avalon suppo...

16/06/2026

SNS EVO Enables Remote and Distributed Video Editing Workflows

SNS has published a guide to remote video editing workflows using its EVO shared storage platform and companion tools, covering use cases ranging from home edit...

16/06/2026

Richmond Flying Squirrels Deploy Grass Valley LDX 110 Cameras at CarMax Park

Grass Valley has announced that the Richmond Flying Squirrels, a Minor League Baseball affiliate of the San Francisco Giants, have deployed five Grass Valley LD...

16/06/2026

AIMS Launches Free Official IPMX Training Series Online

The Alliance for IP Media Solutions (AIMS) has announced the launch of the Official IPMX Training Series, a free online program covering the design, configurati...

16/06/2026

Swerve Womens Sports Announces Distribution Deals with Fubo, Plex, Amazon Fire TV, and Anoki AI

Swerve TV has announced distribution agreements with Fubo, Plex, Amazon Fire TV,...

16/06/2026

ATP and TikTok Expand Global Content Partnership

ATP and TikTok have announced an expansion of their global content partnership, extending the ATP's TikTok hub powered by TikTok GamePlan to cover all nine ...

16/06/2026

FOX Sports Turns Los Angeles Pico Lot Into Its FIFA World Cup Production Nerve Center

Network's LA facility serves as the heart of a sprawling operation built to ...

16/06/2026

300+ Records a Day, 150 TB Daily, and a Relentless Content Avalanche: Inside FOX Sports' World Cup Media Engine

At Pico, the network's media-management team is supporting a flood of HBS fe...

16/06/2026

NHL Games Leaving CBC in Canada as Sublicense With Rogers Sportsnet Ends

The NHL will no longer air on CBC after the pulic broadcasters and national rights-holder Rogers Sportsnet were unable to come to agreement. After a successfu...

16/06/2026

SVG New Sponsor Spotlight: Virtual Eye's Ben Taylor on Making Live Sports More Valuable and Entertaining Through Data-Driven Graphics

As live sports broadcasters continue to seek new ways to make complex action mor...

16/06/2026

Thats BRISK, Baby! FOX Sports' Broadcast Remote IP Studio Kits Bring World Cup Fan Energy Back to Pico

Built with the 2026 FIFA World Cup in mind, these small but mighty IP-based tran...

16/06/2026

Chyron Unveils Chyron Weather 2.4

Share Copy link Facebook X Linkedin Bluesky Email...

16/06/2026

Historic Zhuque-3 Reusable Rocket Test Mission Captured with URSA Cine Immersive

Historic Zhuque-3 Reusable Rocket Test Mission Captured with URSA Cine Immersive Brie Clayton June 16, 2026 0 Comments Apple Immersive Video puts view...

16/06/2026

SMPTE Plans ST 2110 Education Summer Programs

Share Copy link Facebook X Linkedin Bluesky Email...