Sony Pixel Power calrec Sony

NVIDIA Research Wins CVPR Autonomous Grand Challenge for End-to-End Driving

17/06/2024

Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognition (CVPR) conference, running this week in Seattle.

Building on last year's win in 3D Occupancy Prediction, NVIDIA Research topped the leaderboard this year in the End-to-End Driving at Scale category with its Hydra-MDP model, outperforming more than 400 entries worldwide.

This milestone shows the importance of generative AI in building applications for physical AI deployments in autonomous vehicle (AV) development. The technology can also be applied to industrial environments, healthcare, robotics and other areas.

The winning submission received CVPR's Innovation Award as well, recognizing NVIDIA's approach to improving any end-to-end driving model using learned open-loop proxy metrics.

In addition, NVIDIA announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines of every kind.

How End-to-End Driving Works The race to develop self-driving cars isn't a sprint but more a never-ending triathlon, with three distinct yet crucial parts operating simultaneously: AI training, simulation and autonomous driving. Each requires its own accelerated computing platform, and together, the full-stack systems purpose-built for these steps form a powerful triad that enables continuous development cycles, always improving in performance and safety.

To accomplish this, a model is first trained on an AI supercomputer such as NVIDIA DGX. It's then tested and validated in simulation - using the NVIDIA Omniverse platform and running on an NVIDIA OVX system - before entering the vehicle, where, lastly, the NVIDIA DRIVE AGX platform processes sensor data through the model in real time.

Building an autonomous system to navigate safely in the complex physical world is extremely challenging. The system needs to perceive and understand its surrounding environment holistically, then make correct, safe decisions in a fraction of a second. This requires human-like situational awareness to handle potentially dangerous or rare scenarios.

AV software development has traditionally been based on a modular approach, with separate components for object detection and tracking, trajectory prediction, and path planning and control.

End-to-end autonomous driving systems streamline this process using a unified model to take in sensor input and produce vehicle trajectories, helping avoid overcomplicated pipelines and providing a more holistic, data-driven approach to handle real-world scenarios.

Watch a video about the Hydra-MDP model, winner of the CVPR Autonomous Grand Challenge for End-to-End Driving:

Navigating the Grand Challenge This year's CVPR challenge asked participants to develop an end-to-end AV model, trained using the nuPlan dataset, to generate driving trajectory based on sensor data.

The models were submitted for testing inside the open-source NAVSIM simulator and were tasked with navigating thousands of scenarios they hadn't experienced yet. Model performance was scored based on metrics for safety, passenger comfort and deviation from the original recorded trajectory.

NVIDIA Research's winning end-to-end model ingests camera and lidar data, as well as the vehicle's trajectory history, to generate a safe, optimal vehicle path for five seconds post-sensor input.

The workflow NVIDIA researchers used to win the competition can be replicated in high-fidelity simulated environments with NVIDIA Omniverse. This means AV simulation developers can recreate the workflow in a physically accurate environment before testing their AVs in the real world. NVIDIA Omniverse Cloud Sensor RTX microservices will be available later this year. Sign up for early access.

In addition, NVIDIA ranked second for its submission to the CVPR Autonomous Grand Challenge for Driving with Language. NVIDIA's approach connects vision language models and autonomous driving systems, integrating the power of large language models to help make decisions and achieve generalizable, explainable driving behavior.

Learn More at CVPR More than 50 NVIDIA papers were accepted to this year's CVPR, on topics spanning automotive, healthcare, robotics and more. Over a dozen papers will cover NVIDIA automotive-related research, including:

Hydra-MDP: End-to-End Multimodal Planning With Multi-Target Hydra-Distillation

Winner of CVPR's End-to-End Driving at Scale challenge

Read the NVIDIA technical blog

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR best paper award finalist

Driving Everywhere With Large Language Model Policy Adaptation

See DRIVE Labs: LLM-Based Road Rules Guide Simplifies Driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

Improving Distant 3D Object Detection Using 2D Box Supervision

Dynamic LiDAR Resimulation Using Compositional Neural Fields

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving

Sanja Fidler, vice president of AI research at NVIDIA, will speak on vision language models at the CVPR Workshop on Autonomous Driving.

Learn more about NVIDIA Research, a global team of hundreds of scientists and engineers focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/auto-research-cvpr-2024/...
See more stories from nvidia

North America Stories

13/07/2024

VideoAmp Integrates Snap Inc. Ad Inventory

NEW YORK VideoAmp has announced a wide-ranging collaboration with Snap Inc. that will see the social technology platform integrate its first-party data, video, ...

13/07/2024

RAI Moves up Launch of DVB-T2 HEVC Broadcasts

The Italian public broadcaster RAI will start broadcasting on Multiplex B in the DVB-T2 HEVC standard on August 28, according to Marco Rossignoli, president of ...

13/07/2024

Mile-High AI: NVIDIA Research to Present Advancements in Simulation and Gen AI at SIGGRAPH

Mile-High AI: NVIDIA Research to Present Advancements in Simulation and Gen AI a...

13/07/2024

Future Today Adds Studio Content to Fawesome

Future Today said it has signed licencing deals with Sony Pictures Entertainment, Samuel Goldwyn Films and Gravitas Ventures that will add fresh content to its ...

13/07/2024

VideoAmp To Integrate Data, Inventory From Snap Into Ad-Planning Tools

VideoAmp said it made a deal enabling the measurement company to integrate Snap's first-party data, video and augmented-reality ad inventory into VideoAmp&#...

13/07/2024

ABC Renews Bachelor in Paradise' for Season 10

ABC has ordered a tenth season of Bachelor in Paradise. The season will air in 2025 on the network and stream the next day on Hulu....

13/07/2024

Bill Belichick Joins Inside the NFL' for Season 2 on The CW

Bill Belichick, the legendary head coach who led the New England Patriots to eight Super Bowl wins, is joining Inside the NFL, which starts its second season on...

13/07/2024

Allen Media Group's HBCU Go Set To Kick Off College Football Season

Allen Media Group's HBCU Go said it will televise 26 college football games played by Historically Black Colleges and Universities....

13/07/2024

Backed By Top Producers, Exploding Kittens' Card Game Adapted to Netflix Series

Exploding Kittens, an animated series based on the card game, debuts on Netflix ...

13/07/2024

ABC Owned Stations, Ava DuVernay Partner on Our America' Special

ABC Owned Television Stations premieres Our America: Hidden Stories with Ava DuVernay starting Sunday, July 14. The one-hour special sheds light on the interco...

13/07/2024

Shania Twain To Host People's Choice Country Awards' on NBC

Shania Twain will host the People's Choice Country Awards, which happens Thursday, September 26 on NBC and Peacock. The two-hour special will be live at the...

12/07/2024

Fox Sports to Create Virtual Reality Experience for Copa Amrica Final

CAMPBELL, Calif. Fox Sports and YBVR have announced that they will be creating an immersive virtual reality experience for the Copa Am rica Final on Meta Quest,...

12/07/2024

Gray TV Stations in Alaska to Air Seattle Kraken Games

TYSONS, Va. Tegna Inc., the Seattle Kraken and Gray Media have announced that Gray's KAUU in Anchorage and KYEX in Juneau will expand the Kraken broadcast n...

12/07/2024

Hallmark To Launch New Subscription Streaming Service

PASADENA, Calif. Hallmark has unveiled Hallmark+, a subscription streaming television service that will combine the company's brand of lifestyle experience ...

12/07/2024

What the Clearhaven Deal Means for Zixi's Future

In recent years, as TV companies and producers pushed to launch streaming services and deploy IP infrastructures, Zixi has been at the center of that tectonic t...

12/07/2024

Ateme To Focus on Maximizing Output, Minimizing Costs At IBC 2024

PARIS Ateme will feature technologies pushing the boundaries of transformation, monetization and experience at IBC 2024, Sept. 13-16, at the RAI Convention Cent...

12/07/2024

DNEG Group Completes Prime Focus Technologies Acquisition

The DNEG Group has completed acquisition of Prime Focus Technologies....

12/07/2024

Wimbledon and Hawk-Eye Innovations Extend Tech Agreement

LONDON The All England Lawn Tennis Club and Sony's Hawk-Eye Innovations have announced a new multi-year agreement that will see Hawk-Eye continue to provide...

12/07/2024

Vision2 Marketing to Rep FOR-A as Part of Acceleration in...

As part of a more expansive push into the U.S. market, FOR-A announces its partnership with Vision2 Marketing, a manufacturers' rep for pro audio, video, li...

12/07/2024

KTRK Optimizes Look of Live Streams and Broadcasts with S...

Houston broadcaster KTRK-TV has been on the scene for 70 years and is known for community outreach as well as investigative journalism. Journalist alumni includ...

12/07/2024

QuickLink Expands its US Presence With Two New Seasoned T...

QuickLink, the leading global provider of multi-camera video productions and remote contributions, announces a significant growth strategy with two high-level p...

12/07/2024

Imagine Communications Names Don Durand SVP Global Sales...

Imagine Communications has appointed international media industry executive Don Durand to the position of senior vice president of global sales for its ad tech ...

12/07/2024

BH Telecom selects Appear to deliver Moja TV service to s...

Appear, the global leader in live production technology, has been selected by BH Telecom to enhance the live channel processing software for its Moja TV service...

12/07/2024

Ikegami Announces New Additions to its Range of Broadcast...

Ikegami announces two new additions to its range of broadcast quality picture monitors: the 24-inch HLM-2460WA and 18.5 inch HLM-1860WR. These will make their E...

12/07/2024

Vizrt focuses on BroadcastAV market with PTZ3 PLUS camera...

Vizrt, the leader in real-time graphics and live production solutions for content creators, brings the BroadcastAV market sharply into focus with today's la...

12/07/2024

Czech TV Live Streams Prague Marathon with Resilient Deje...

Local production company Livecast counts on Dejero Smart Blending Technology for connectivity on the move Dejero recently provided Czech production company Li...

12/07/2024

LTN playout solution powers European Football Championshi...

LTN, the industry leader in transformative media technology and video transport solutions, announces the successful delivery of live and pre-recorded coverage t...

12/07/2024

Ateme to Showcase Cutting-Edge Solutions to Maximise Outp...

Ateme's solutions deliver unprecedented video experiences and content monetisation while optimizsng workflows through hybrid on-premises and cloud systems. ...

12/07/2024

Lightware Announces New Leadership in the Enterprise Team

Lightware Visual Engerineering, a leading manufacturer of connectivity solutions for the professional integrated systems market and a pioneer in signal manageme...

12/07/2024

Anton Bauer Salt-E Dog Wins Innovation Impact Award at Ro...

Anton/Bauer has announced that its pioneering sustainable power product, Salt-E Dog, has been awarded the prestigious Innovation Impact Award at the Royal Telev...

12/07/2024

Teatro TV Browser by iWedia Surpasses 40 million Devices...

iWedia, a leading innovator in software solutions for connected TV devices, is thrilled to announce a significant milestone for its flagship product, the Teatro...

12/07/2024

Tim Belcher joins Light Iron as new Managing Director

Light Iron, the post-production creative-services division of Panavision, has named industry veteran Tim Belcher the company's new Managing Director. Belche...

12/07/2024

COW Job Listing: Need a Drone Shot in Boardman, Oregon

COW Job Listing: Need a Drone Shot in Boardman, Oregon Brie Clayton July 11, 2024 0 Comments Need a Drone shot July 9, 2024COW Job Listing: Freelanc...

12/07/2024

American Underground Celebrates Pride & Juneteenth with Free, Public Learning Events

American Underground, Capitol Broadcasting's co-working community division, ...

12/07/2024

5 Session-Saving Tips for Studio One

By Craig Anderton I admit it: the following tips are based on personally embarrassing experiences. I like to work fast to keep the creative juices flowing, but...

12/07/2024

Fox Sports Heads to Berlin for Euro Final; Wraps Up Massive Month of International Soccer Coverage

Fox Sports Heads to Berlin for Euro Final; Wraps Up Massive Month of Internation...

12/07/2024

Euro 2024: UEFA Gets Set for Spain v England Final

Euro 2024: UEFA gets set for Spain v England final By George Bevir Friday, July 12, 2024 - 11:19 Print This Story When England take on Spain on Sunday in ...

12/07/2024

Euro 2024: Inside UEFA's International Broadcast Centre

Euro 2024: Inside UEFA's International Broadcast Centre By George Bevir Friday, July 12, 2024 - 08:00 Print This Story The IBC in Leipzig, Germany (Ph...

12/07/2024

NBC Sports Bay Area & California's Rusty West on Vendor Partnerships Bringing Broadcast Elements to Life

NBC Sports Bay Area & California's Rusty West on Vendor Partnerships Bringin...

12/07/2024

Long-time PSSI Broadcast/Mechanical Engineer Paul Blomstrom Dead at 44

Long-time PSSI Broadcast/Mechanical Engineer Paul Blomstrom Dead at 44 By Ken Kerschbaumer, Editorial Director Friday, July 12, 2024 - 12:55 pm Print This ...

12/07/2024

SVG Sit-Down: FOX Sports' Michael Bucklin Explores Successful Live Social Video During Euros, Copa Amrica

SVG Sit-Down: FOX Sports' Michael Bucklin Explores Successful Live Social Vi...

12/07/2024

Mile-High AI: NVIDIA Research to Present Advancements in Simulation and Gen AI at SIGGRAPH

NVIDIA is taking an array of advancements in rendering, simulation and generativ...

11/07/2024

Utah Scientific Bounces Back From Pandemic With Price Reduction

Utah Scientific Bounces Back From Pandemic With Price Reduction Buyers Could Save as Much as 30% on UDS Compact Series and UHD 12G Video Routing Systems SAL...

11/07/2024

Little Miss Sunshine, Boyhood, and More Celebrate Birthdays in July

The Hoover family embarks on an eventful road trip to California for a junior beauty pageant in Little Miss Sunshine....

11/07/2024

Grass Valley and Diversified sign multimillion-dollar strategic partnership agreement

MONTREAL, CANADA - July 11, 2024 - Grass Valley, a leader in Live Production Sol...

11/07/2024

Media Excel To Showcase New DIVA AI-Powered Encoding Technology at 2024IBC

AUSTIN, Texas Media Excel will showcase its new artificial intelligence-based DIVA (Dynamic Intelligent Video Adaptive) encoding technology at the 2024IBC, Sept...

11/07/2024

German Broadcaster ZDF Adopts Avid MediaCentral | Cloud UX

BURLINGTON, Mass. German public broadcaster ZDF is migrating its existing asset management systems onto Avid's MediaCentral | Cloud UX software platform, th...

11/07/2024

ESPN Taps DP Bryan TWZ Brousseau, hsc.tv to Light Wimbledon Studio

WIMBLEDON, U.K. Video production company hsc.tv worked with Director of Photography Bryan TWZ Brousseau to create a unique lighting and set design for ESPN'...

11/07/2024

Fourier Audio Embeds iLok USB Hardware in transform.engine for Advanced Security & Protection

Fourier Audio Embeds iLok USB Hardware in transform.engine for Advanced Security...

11/07/2024

Create with Maxon: Product Visualization from Brief to Render - Week 3

Create with Maxon: Product Visualization from Brief to Render - Week 3 Brie Clayton July 11, 2024 0 Comments Spicing it up with Materials (3/5) - Crea...