Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

30/07/2026

CP Communications Provides Broadcast Support for 2026 MLB All-Star Week in Philadelphia

CP Communications provided RF audio, RF video, communications, RF coordination, ...

30/07/2026

OpenDrives Named to 2026 CRN Storage 100 List

OpenDrives has been included in CRN's 2026 Storage 100 list in the Software-Defined Storage category. The annual list, selected by the CRN editorial team, r...

30/07/2026

SES Selected by LATAM Airlines for Multi-Orbit Satellite Connectivity

LATAM Airlines has selected SES to provide multi-orbit inflight connectivity to its fleet of Airbus and Embraer aircraft. More than 60 aircraft - including Airb...

30/07/2026

Tagboard Launches Producer API with Elgato Stream Deck Integration

Tagboard has launched the Producer API, an open interface that allows key commands in Tagboard's live production environment to be triggered from external d...

30/07/2026

FOX Sports Adds Del Mar to Horse Racing Coverage in Multi-Year Deal

FOX Sports and the New York Racing Association (NYRA) have announced a multi-year agreement with Del Mar Thoroughbred Club that will make FOX Sports the exclusi...

30/07/2026

ABC Commercial Launches Four FAST Channels on LG Smart TVs

ABC Commercial has launched four free ad-supported streaming television (FAST) channels on LG Smart TVs across North America, Great Britain, and select countrie...

30/07/2026

Six Kings Slam Returns to Netflix in October with Sinner, Alcaraz, Djokovic

The Six Kings Slam exhibition tennis tournament will return to Riyadh on October 21, 22, and 24, streaming live on Netflix at no additional cost to subscribers....

30/07/2026

Central Christian Church Uses Clear-Coms Gen-IC to Connect Remote Production Teams

Central Christian Church in Mt. Vernon, Illinois has deployed Clear-Com's LQ...

30/07/2026

Blackmagic Design Releases UltraStudio Express 3G Capture and Playback Devices

Blackmagic Design has released the UltraStudio Express 3G family, a pair of USB4 capture and playback devices compatible with Mac, Windows, and Linux computers ...

30/07/2026

ESPN Secures Media Rights to Womens Pro Baseball League

ESPN has reached a media rights agreement with the Women's Pro Baseball League (WPBL), making ESPN the national streaming home of the league's 2026 seas...

30/07/2026

Most Valuable Promotions and Professional Fighters League Merge

Most Valuable Promotions (MVP) and the Professional Fighters League (PFL) have announced a merger that will operate under the MVP banner. PFL CEO John Martin wi...

30/07/2026

Columbus Blue Jackets To Simulcast TV and Radio Broadcasts Beginning 2026-27

The Columbus Blue Jackets will simulcast all game broadcasts across television and radio beginning with the 2026-27 NHL season. Under the new format, the televi...

30/07/2026

TMRW Sports Selects Populous To Design Professional Flag Football Stadium

TMRW Sports has selected Populous as architect for a purpose-built stadium for its professional flag football league, being developed in partnership with the NF...

30/07/2026

SiriusXM Launches Sports Pass Subscription at $5 per Month

SiriusXM has announced SiriusXM Sports Pass, a new subscription plan launching September 1 that bundles the company's sports audio programming into a single...

30/07/2026

SVG Cloud & Content Workflows Summit Focuses on Live Production, Content Management, AI

Leagues, broadcasters, and technologists gather to tackle cloud economics, distr...

30/07/2026

AWSs Jason Dvorkin on the Power of AWS Elemental Inference, Agentic AI in Live Sports Production

A major player in the cloud-based solutions, Amazon Web Services (AWS) has conti...

30/07/2026

Minnesota Timberwolves Ink Local-Media-Rights Deal With DAZN

The NBA will provide production for all local game broadcasts, including pregame and postgame coverage...

30/07/2026

Cosm Detroit To Open September 10

Cosm will open its fourth immersive sports and entertainment venue in downtown Detroit on September 10. Located at 25 Cadillac Square, adjacent to Campus Martiu...

30/07/2026

NFL Network Added to ESPN App Beginning July 30

NFL Network will be available on the ESPN App beginning July 30 for Unlimited plan subscribers. The addition brings NFL Network's live games, studio program...

30/07/2026

NEP Platform Adds Riedels SimplyLive Production Suite

NEP Group has added Riedel's SimplyLive Production Suite to NEP Platform, giving customers access to the live production software through NEP's orchestr...

30/07/2026

SUNDANCE INSTITUTE NAMES TRISTA SCHROEDER GENERAL COUNSEL & CHIEF OPERATING OFFICER

LOS ANGELES, CA, July 30, 2026 - The nonprofit Sundance Institute today announce...

30/07/2026

Waves release Atlas Reverb

A new take on classic algorithmic reverbs Waves have just introduced a new flagship algorithmic reverb plug-in that offers the company's take on the ico...

30/07/2026

Arturia's Summer Sale now live

Save up to 30% on virtual instruments & plug-ins Arturia have recently launched their annual Summer Sale, which sees discounts of up to 30% applied across a...

30/07/2026

SSL unveil the Odyssey console

ActiveAnalogue technology powers new flagship modular console A year on from the launch of the Oracle, SSL have revealed a new flagship console which they s...

30/07/2026

DT 275 Pro from beyerdynamic

Up to 22dBA of passive attenuation The latest arrival to the beyerdynamic range introduces a new compact monitoring headphone that's been designed to ta...

30/07/2026

Apprenticeship success at SGL Carbon in Meitingen

As part of its traditional graduation ceremony, SGL Carbon honored the achievements of a total of twelve young professionals who successfully completed their ap...

30/07/2026

Premiere date revealed for SBS Original drama The Airport Chaplain, starring Hugo Weaving and Shabana Azeez

Premiere date revealed for SBS Original drama The Airport Chaplain, starring Hug...

30/07/2026

TVNZ Works with Evergent to Successfully Debut New Stream...

TVNZ and Evergent are celebrating a major TVNZ milestone - the evolution from a free-to-view, ad-supported business into a hybrid model, offering premium sport...

30/07/2026

WebMiere Adds Native VP9/AV1 WebM/MKV Import to Adobe Premiere Pro

WebMiere Adds Native VP9/AV1 WebM/MKV Import to Adobe Premiere Pro Brie Clayton July 29, 2026 0 Comments Brought to you by KawaiiEngine, WebMiere is a...

30/07/2026

Comments on FCC License Renewals for ABC Stations Top 150,000

Share Copy link Facebook X Linkedin Bluesky Email...

30/07/2026

CW Sports Live Streaming to Launch Aug. 4 on the ESPN App

Share Copy link Facebook X Linkedin Bluesky Email...

30/07/2026

Fox Advertising, iSpot Expand Measurement Deal

Share Copy link Facebook X Linkedin Bluesky Email...

30/07/2026

GlobalM Expands Software-Defined Media Platform for Cloud...

At IBC2026, GlobalM will showcase the latest evolution of its software-defined media platform, enabling broadcasters, telecom operators and media organisations ...

30/07/2026

Digital Alert Systems at 2026 TAB Show

At the 2026 TAB Show, Digital Alert Systems will highlight and demo the new DAS-Link AIR, an over-the-air common alert protocol (CAP) dissemination system for ...

30/07/2026

OpenDrives Named to the 2026 CRN Storage 100 List

OpenDrives Named to the 2026 CRN Storage 100 List Company Recognized Among the 50 Coolest Software-Defined Storage Vendors for Unlimited Capacity, S3 Hybrid M...

30/07/2026

VIDA Brings AI Scene-Level Search to Media Archives Endin...

New Scene Search capability in VIDA Content OS makes video content itself searchable, letting broadcasters and streamers find key moments across millions of ass...

30/07/2026

Tribeca Films Announces Summer 2026 Acquisitions Including Loves Company,' A Very Merry Breakup,' And Magic Hour'

July 30th, 2026 TRIBECA FILMS ANNOUNCES SUMMER 2026 ACQUISITIONS INCLUDING L...

30/07/2026

Which WSL matches are Sky Sports showing on the 2026/27 opening weekend?

Thursday 30 July 2026 Which WSL matches are Sky Sports showing on the 2026/27 opening weekend? The season will kick off on Friday 4 September with Manchester ...

30/07/2026

Enjoy discounts on the iPhone Air, plus 5Gb speeds with Ultimate TV for 2.10 a day

Thursday 30 July 2026 Enjoy discounts on the iPhone Air, plus 5Gb speeds with U...

30/07/2026

Creedon's Musical Atlas of Ireland returns for another season of stories and songs

Broadcaster John Creedon explores Ireland's rich musical heritage Sunday 2 A...

30/07/2026

Best in Class: Stream PC Games and Study on the Same Laptop With GeForce NOW

Back to school means balancing assignments, deadlines and downtime. GeForce NOW makes it easy to have it all. With cloud gaming, everyday laptops used for clas...

29/07/2026

Grass Valley Op-Ed Series: Seven Paradoxes Shaping the Next Era of Media Production

In this series, Grass Valley explores the seven paradoxes shaping the next era o...

29/07/2026

SVG Students to Watch: A Look Back at a Year of Spotlighting the Emerging Talent in Live Sports Production

Thats a wrap on Season 1 of SVG Students to Watch! Across the country, these stu...

29/07/2026

DAZN to Become Exclusive DTC In-Market Streaming Home of YES Network, MSG Networks; Gotham App to Shut Down

The agreement will move Gotham Sports App subscribers to DAZN during the 2026-27...

29/07/2026

Big South Promotes Emily Dassel to Associate Commissioner for Brand and Creative Strategy

Longtime digital-media leader expands role overseeing conference branding, video...

29/07/2026

Give Me the Backstory: Get to Know Gregg Araki, the Director of I Want Your Sex

By Bailey Pennick One of the most exciting things about the Sundance Film Festival is having a front-row seat for the bright future of independent filmmaking. ...