Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

North America Stories

25/03/2026

Samsung Launches Shoppable TV Experience with Amazon

Share Copy link Facebook X Linkedin Bluesky Email...

25/03/2026

Gray Media to Simulcast 2026 Atlanta Braves to Home Opener

Share Copy link Facebook X Linkedin Bluesky Email...

25/03/2026

Starfish highlights flexible TS Splicer releases and new Monitoring Dashboard at NAB Show

Starfish highlights flexible TS Splicer releases and new Monitoring Dashboard at...

25/03/2026

XLNTSOUND Elevates Video Production with Blackmagic Design

XLNTSOUND Elevates Video Production with Blackmagic Design Brie Clayton March 24, 2026 0 Comments Audio company upgrades workflow with Blackmagic Cine...

25/03/2026

First Time Exhibitor IPC Introduces One Link Keypanels for One Connect Intercom Platform at NAB 2026

First Time Exhibitor IPC Introduces One Link Keypanels for One Connect Intercom ...

24/03/2026

Utah Scientific to Showcase a Variety of Hybrid SDI/IP Innovations at the 2026 NAB Show

Utah Scientific to Showcase a Variety of Hybrid SDI/IP Innovations at the 2026 N...

24/03/2026

NAB 2026: Mediaproxy Highlights Enhancements to its LogServer suite at NAB Show 2026

Mediaproxy will showcase an updated version of its LogPlayer interface at NAB Sh...

24/03/2026

Canada's Cgep de Jonquire and Calrec Take Five With a TV Studio Upgrade

C gep de Jonqui re, a public college in Qu bec, has upgraded its TV studio complex with five Calrec Brio consoles and a Dante IP network. The installation was c...

24/03/2026

American Association of Professional Baseball Expands Broadcast Partnerships with RYZ, Bally Sports Live, NLSE, FTF Sports, and SLVR

The American Association of Professional Baseball (AAPB) has announced new and r...

24/03/2026

Ateme KYRION Enabled NT's Nationwide Contribution Network During Southeast Asian Games 2025

National Telecom Public Company Limited (NT) used Ateme's KYRION platform fo...

24/03/2026

FDJ United Upgrades Lottery Production with Grass Valley UHD Cameras

Grass Valley has expanded its relationship with ImSoPROD, the TV production entity of FDJ United, with the deployment of LDX C98 compact cameras across its nati...

24/03/2026

Scripps Sports Launches Free Streaming Channel Scripps Sports Network on March 24

The E.W. Scripps Company is launching Scripps Sports Network (SSN), a free, ad-s...

24/03/2026

SVG Sit-Down: NEP Platform VP Dan Murphy on Software-Driven Production and the Future of Live Sports Workflows

NEP Group has launched NEP Platform, a new software orchestration system that un...

24/03/2026

Daktronics Installs New Video Display and Ribbon Boards at Chase Field

Daktronics has partnered with the Arizona Diamondbacks to design, manufacture and install a new LED video display and three ribbon boards at Chase Field in Phoe...

24/03/2026

Grass Valley Upgrades University of Pittsburgh Athletics to ST 2110 IP Production Infrastructure

Grass Valley has completed an upgrade of University of Pittsburgh Athletics'...

24/03/2026

Daktronics to Upgrade LED Displays at Wrigley Field

Daktronics is upgrading eight LED displays at Wrigley Field in Chicago, totaling more than 8,300 square feet, using the Daktronics Renew product line. The upgra...

24/03/2026

ASG Appoints Kevin Poole as Senior Project and Support Manager

Advanced Systems Group has appointed Kevin Poole as Senior Project and Support Manager within the ASG Workflow and Tools practice. The position is newly created...

24/03/2026

SVG New Sponsor Spotlight: Verimatrix's Juan Martinez on Safeguarding Live Sports Content

In an increasingly streaming-centric world, protecting high-value live sports co...

24/03/2026

The Countdown Is On! Prep for Vegas with SVG's SportsTech@NABShow Blog

Mediaproxy will showcase an updated version of its LogPlayer interface at NAB Show, April 19-22 in Las Vegas, booth W1423. The update includes AI tools, native ...

24/03/2026

The Creator Economy's Growing Impact: Building the Modern Production Studio With Second Rodeo Productions' CEO Scott Brown

The production studio has worked with content juggernauts such as MrBeast, The R...

24/03/2026

MLB Local Media To Keynote Regional Sports Production Summit, June 29-30, Denver

The event will also feature a State of the Industry address from sports-media veteran and strategic advisor Patrick Crakes....

24/03/2026

Victory+ Launches NWSL Sunday Night Soccer Franchise

Victory , a free sports streaming service owned by A Parent Media Co. (APMC), has launched its NWSL Sunday Night Soccer franchise. The platform says the launch ...

24/03/2026

FAST Sports: Contrary to Some Data, Sports Does Have a Future on FAST

Sports content in free streaming is still relatively scant, but it's growing, right along with its monetization capabilities Streaming-tech company Wurl re...

24/03/2026

New Doc Sports Heaven: The Birth of ESPN' Produced by ESPN Historian Mike Soltys Arriving April 6

Directed by Greg DeHart, the doc will feature Sports Broadcasting Hall of Famers...

24/03/2026

Tubi Unveils Plans for Creator-Driven Formula 1 Altcasts in Partnership With Apple TV

For the first time ever, Tubi - Fox's free streaming service - is working w...

24/03/2026

L3Harris Ramps Up Production for Counter-Unmanned Systems

L3Harris ramps up production of its VAMPIRE C-UxS system....

24/03/2026

Delivering Victory Through Fuzing Precision and Scale

For more than 70 years, L3Harris has delivered essential fuzes and safe-and-arm solutions that bring the might to multi-domain battlefields across the globe, ...

24/03/2026

FDJ United Leaves Nothing to Chance with UHD Lottery Prod...

Grass Valley has strengthened its long-standing relationship with ImSoPROD, the TV production entity of the FDJ United, with the deployment of Grass Valley LDX ...

24/03/2026

Utah Scientific to Showcase a Variety of Hybrid SDI IP In...

At the 2026 NAB Show in Las Vegas, Utah Scientific will spotlight its expanding portfolio of hybrid SDI/IP routing, conversion, control, and signal management s...

24/03/2026

Miri V410 Live 4K Encoder-Decoder to Make North American...

Fresh off its global reveal at ISE 2026 in Barcelona, the new V410 live 4K encoder/decoder from Miri Technologies Inc. will make its North American debut at the...

24/03/2026

Cinegy delivers software defined television for the real...

Cinegy GmbH, premier provider of software-defined television technology, will show how its solutions answer the real challenges facing content creators and broa...

24/03/2026

ArtSound FM Futureproofs with DHD Audio Production Consol...

Canberra-based radio and online broadcast network ArtSound FM (https://artsound.fm/) has invested in DHD audio mixing consoles for two new studios at its headqu...

24/03/2026

IBC unveils 2026 Accelerator projects tackling industry c...

IBC today announced the nine projects selected for its 2026 Accelerator Media Innovation Programme, bringing together organisations from across broadcast, strea...

24/03/2026

Mediaproxy Highlights Significant Enhancements to its Log...

Mediaproxy, the global standard for software-based IP compliance monitoring and multiviewing solutions, will showcase the next generation of its industry-leadin...

24/03/2026

Cegep de Jonquiere and Calrec take five with a substantia...

Underscoring its global reputation as one of Canada's most renowned colleges for television production, the C gep de Jonqui re public college in Qu bec has ...

24/03/2026

Synamedia launches industry-first edge watermarking solut...

Leading video software provider Synamedia today announced the launch of the industry's first edge watermarking solution that enables faster disruption of pi...

24/03/2026

Experience Commerce Wins Social Media Mandate for Fintech...

Experience Commerce, a leading full-service digital marketing agency and part of the Cheil SWA Group, has secured the annual social media mandate for Fiat Payme...

24/03/2026

FCC Approves TV Station License Transfers to Gray

Share Copy link Facebook X Linkedin Bluesky Email...

24/03/2026

Study: Globally HBO Max Is Most Widely Bundled Streaming Service

Share Copy link Facebook X Linkedin Bluesky Email...

24/03/2026

Charter Communications Names John Lee Head Of Intelligence Ventures

Share Copy link Facebook X Linkedin Bluesky Email...

24/03/2026

Intinor introduces enhanced SRT monitoring, HDR transport and NDI Advanced support at NAB Show 2026

Intinor introduces enhanced SRT monitoring, HDR transport and NDI Advanced suppo...

24/03/2026

Miri V410 Live 4K Encoder/Decoder to Make North American Debut at 2026 NAB Show

Miri V410 Live 4K Encoder/Decoder to Make North American Debut at 2026 NAB Show Brie Clayton March 23, 2026 0 Comments Powerful new solution for strea...

24/03/2026

Tribeca Festival And Vulture Partner To Bring Vulture Festival Back To New York For Tribeca's 25th Anniversary

March 24th, 2026 TRIBECA FESTIVAL AND VULTURE PARTNER TO BRING VULTURE FESTI...

24/03/2026

Tribeca X Announces Preliminary 2026 Speaker Lineup

March 24th, 2026 TRIBECA X ANNOUNCES PRELIMINARY 2026 SPEAKER LINEUP Cedric the Entertainer, Ted Danson, Brooklyn Peltz Beckham, Dhar Mann, K-Pop Demon ...

24/03/2026

McDonald's and Netflix Call on Fans to Pick a Side With Two New 'KPop Demon Hunters'-Inspired Meals

Back to All News McDonalds and Netflix Call on Fans to Pick a Side With Two New...

24/03/2026

BTS Commands the Global Stage as 'BTS THE COMEBACK LIVE | ARIRANG' Draws 18.4 Millions Global Viewers

Back to All News BTS Commands the Global Stage as BTS THE COMEBACK LIVE | ARIRA...

24/03/2026

Advancing Open Source AI, NVIDIA Donates Dynamic Resource Allocation Driver for GPUs to Kubernetes Community

Artificial intelligence has rapidly emerged as one of the most critical workload...

23/03/2026

Professional Fighters League & U-Next Renew Multi-year Partnership to Broadcast PFL Events Live in Japan

The Professional Fighters League (PFL) has renewed its multi-year partnership wi...

23/03/2026

The Snow League Announces Google Cloud as Official Cloud and AI Partner to Power Personalized Fan Experiences Worldwide

The Snow League has named Google Cloud as its Official Cloud and AI Partner. The...

23/03/2026

Chyron Appoints Eric Wolff as Director of Venues Sales, North America

Chyron has appointed Eric Wolff as Director of Venues Sales, North America. Wolff previously served as Director of Broadcast Operations & Media Production for T...