Perceiving with Confidence: How AI Improves Radar Perception for Autonomous Vehicles
28/04/2021
Autonomous vehicles don't just need to detect the moving traffic that surrounds them - they must also be able to tell what isn't in motion.
At first glance, camera-based perception may seem sufficient to make these determinations. However, low lighting, inclement weather or conditions where objects are heavily occluded can affect cameras' vision. This means diverse and redundant sensors, such as radar, must also be capable of performing this task. However, additional radar sensors that leverage only traditional processing may not be enough.
In this DRIVE Labs video, we show how AI can address the shortcomings of traditional radar signal processing in distinguishing moving and stationary objects to bolster autonomous vehicle perception.
Traditional radar processing bounces radar signals off of objects in the environment and analyzes the strength and density of reflections that come back. If a sufficiently strong and dense cluster of reflections comes back, classical radar processing can determine this is likely some kind of large object. If that cluster also happens to be moving over time, then that object is probably a car.
While this approach can work well for inferring a moving vehicle, the same may not be true for a stationary one. In this case, the object produces a dense cluster of reflections, but doesn't move. According to classical radar processing, this means the object could be a railing, a broken down car, a highway overpass or some other object. The approach often has no way of distinguishing which.
Introducing Radar DNN One way to overcome the limitations of this approach is with AI in the form of a deep neural network (DNN).
Specifically, we trained a DNN to detect moving and stationary objects, as well as accurately distinguish between different types of stationary obstacles, using data from radar sensors.
Training the DNN first required overcoming radar data sparsity problems. Since radar reflections can be quite sparse, it's practically infeasible for humans to visually identify and label vehicles from radar data alone.
Figure 1. Example of propagating bounding box labels for cars from the lidar data domain into the radar data domain. Lidar, however, can create a 3D image of surrounding objects using laser pulses. Thus, ground truth data for the DNN was created by propagating bounding box labels from the corresponding lidar dataset onto the radar data as shown in Figure 1. In this way, the ability of a human labeler to visually identify and label cars from lidar data is effectively transferred into the radar domain.
Moreover, through this process, the radar DNN not only learns to detect cars, but also their 3D shape, dimensions and orientation, which classical methods cannot easily do.
With this additional information, the radar DNN is able to distinguish between different types of obstacles - even if they're stationary - increase confidence of true positive detections, and reduce false positive detections.
The higher confidence 3D perception results from the radar DNN in turn enables AV prediction, planning and control software to make better driving decisions, particularly in challenging scenarios. For radar, classically difficult problems like accurate shape and orientation estimation, detecting stationary vehicles as well as vehicles under highway overpasses become feasible with far fewer failures.
The radar DNN output is integrated smoothly with classical radar processing. Together, these two components form the basis of our radar obstacle perception software stack.
This stack is designed to both offer full redundancy to camera-based obstacle perception and enable radar-only input to planning and control, as well as enable fusion with camera- or lidar-perception software.
With such comprehensive radar perception capabilities, autonomous vehicles can perceive their surroundings with confidence.
To learn more about the software functionality we're building, check out the rest of our DRIVE Labs series.
LINK: | https://blogs.nvidia.com/blog/2021/04/28/drive-labs-ai-radar-perceptio... |
See more stories from nvidia |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
01/05/2024
Release Rundown: What to Watch in May, From I Saw the TV Glow to Power
Owen (Justice Smith) and Maddy (Brigette Lundy-Paine) develop an eerie connection to a supernatural TV show in Jane Schoebrun's I Saw the TV Glow....
01/05/2024
Celebrate May the 4th With Special Star Wars' Playlists, Audiobooks, and More
Spotify's Star Wars celebrations are in full Force starting today, and this ...
01/05/2024
Spotify's New Hardcore Gardening Playlist and Brain Dead Collab Will Rattle Greenhouses Everywhere
All around the world, millions of Spotify fans love listening to hardcore punk m...
01/05/2024
THE CALL FOR THE FIFTH PRESIDENTIAL EMPLOYMENT STIMULUS PROGRAMME (PESP5) FOCUSED ON THE AUDIO-VISUAL INDUSTRY, IS OPEN.
THE CALL FOR THE FIFTH PRESIDENTIAL EMPLOYMENT STIMULUS PROGRAMME (PESP5) FOCUSE...
01/05/2024
QTAKE Pioneers Live 3D Stream to Apple Vision Pro
QTAKE, the leading video assist and on-set collaboration software trusted by professional filmmakers worldwide, launched a new cutting-edge feature at NAB 2024....
01/05/2024
LA County Opens Entertainment Business Interruption Fund Grant Program for Small Business, LAEDC Offers Updates
In HPA's ongoing focus on providing input and insight from city and county g...
01/05/2024
Trailblazing AI Storytelling by Empowering Storytellers: Curious Refuge
HPA Newsline had the opportunity to speak with Caleb and Shelby Ward, founders of Curious Refuge. Caleb and Shelby were participants in the 2024 HPA Tech Retrea...
01/05/2024
Supporting Global Air Traffic Modernization through Managed Services Infrastructure
L3Harris works with air navigation service providers who efficiently manage air ...
01/05/2024
Q&A with Scott Alexander, Aerojet Rocketdyne's new Missile Solutions President
Scott Alexander, Missile Solutions President, Aerojet Rocketdyne...
01/05/2024
Comcast: More Than Half of Viewers Frustrated About Finding Content
NEW YORK Just over half of U.S. viewers are frustrated over their ability to be able to find content to watch on TV according to Comcast....
01/05/2024
Bally RSNs Go Dark On Comcast Systems
Bally Sports Regional Networks were taken off Comcast's systems on April 30 when their existing distribution agreement expired and Diamond Sports Group was ...
01/05/2024
NXTGENbps powers Molinare's Notorious DIT
NXTGENbps battery solutions are designed to replace diesel generators and offer silent, emission-free power for various applications in film, TV, outside broadc...
01/05/2024
Elodie Ichter Joins Light Iron as Senior Colorist
Light Iron, the post-production creative-services division of Panavision, is excited to announce that lodie Ichter has joined the company's artist roster a...
01/05/2024
CJP presents live mo-cap demos at MPTS
CJP Broadcast Service Solutions, systems integration, production and content digitisation specialist, will feature live motion capture demonstrations at the 202...
01/05/2024
Media Links Partners with Sunteq for Representation in S...
Media Links, manufacturer and pioneer in Media over IP transport technology, has partnered with Sunteq spol s.r.o., a leading reseller located in Slovakia to st...
01/05/2024
Dot Group Showcases Enhanced Data Management and Sustaina...
Dot Group, a leader in cutting-edge data management solutions and the premier European reseller of critical Broadcast supporting IBM technologies, will demonstr...
01/05/2024
Glensound shows the latest in IP audio at MPTS
Glensound, industry leader in high quality audio systems, is showcasing the latest additions to its extensive portfolio of networked audio products at MPTS. Aud...
01/05/2024
Hitomi Broadcast to demonstrate ST2110 MatchBox at MPTS
Hitomi Broadcast, a leader in audio/video alignment and latency solutions, will be demonstrating advancements in SMPTE ST2110 workflows ensuring broadcasters ha...
01/05/2024
Leading UK vendors join in the CABSAT 30th anniversary ce...
Taking a leading role in the 30th anniversary CABSAT exposition (Dubai World Trade Centre, 21 23 May), the GREAT Britain and Northern Ireland pavilion will fe...
01/05/2024
NXTGENbps Powers Notorious DITs Green Energy Initiative w...
NXTGENbps, a pioneer in green energy solutions, proudly announces its partnership with Notorious DIT, a leading provider of digital imaging technology services ...
01/05/2024
Videosys Broadcast Brings Pan and Tilt To Coverage Of The...
As Cambridge celebrated its double win over Oxford in the 2024 Boat Race, behind the scenes celebrations were also taking place at Videosys Broadcast to mark th...
01/05/2024
Live After 5' summer concert series returns to downtown Raleigh
A popular - and free - summer concert series is coming back to downtown Raleigh! WRAL is proudly partnering with Downtown Raleigh Alliance for its Live After ...
01/05/2024
Cadent Launches Performance TV To Boost Omnichannel Ad Results
Cadent said it launched Performance TV, designed to increase the effectiveness of campaign by combining the reach of traditional linear TV with the targeting of...
01/05/2024
Eric Johnson, KOMO Seattle Anchor, Sets Retirement
Eric Johnson, anchor at KOMO Seattle, will retire as of June 6. Johnson anchors the 5 and 6 p.m. weekday newscasts and writes/produces/hosts Eric's Heroes,...
01/05/2024
T-Mobile Dials Deeper Into Ad Business With Retail Network (NewFronts)
T-Mobile said it is expanding its advertising business, creating an in-store retail media network and adding Plex, the video streaming platform for its footprin...
01/05/2024
Samsung Ads Expands AI Tools To Target Streaming Viewers (NewFronts)
Samsung Ads said it was introducing new opportunities for advertisers to use data about its viewers to target viewers across the full marketing funnel at its Ne...
01/05/2024
Samsung TV Plus Swings for the Fences With Streaming Sports Channels (NewFronts)
Samsung's ad-supported streaming platform Samsung TV Plus is putting more sports in its lineup with free channels about baseball, golf, hockey, auto racing ...
01/05/2024
Dan Rather Documentary Offers Open-Book' Look at Newsman
Rather, a documentary about famed newsman Dan Rather, premieres on Netflix May 1. The film looks at his rise to prominence in TV news, including his work coveri...
01/05/2024
Ken Wayne, KRON San Francisco Anchor, Retires Next Month
Ken Wayne, evening anchor at KRON San Francisco, is retiring after 33 years in Bay Area television. He was born in Marin County, started his journalism career a...
01/05/2024
Jeff Blaszak Promoted To Senior VP at ShowSeeker
ShowSeeker, the company behind the Pilot cloud-based order management system, said it promoted Jeff Blaszak to senior VP of business operations and strategy....
01/05/2024
Great American Media Picks VideoAmp As Ad Currency
Great American Media said it reached an agreement with VideoAmp that will enable clients to use VideoAmp's ad buying measurement and data capabilities when ...
01/05/2024
NXTGENbps powers Molinaire's Notorious DIT
NXTGENbps battery solutions are designed to replace diesel generators and offer silent, emission-free power for various applications in film, TV, outside broadc...
01/05/2024
Amazon CEO: AWS has multi-billion dollar revenue run rate in AI already
Sales rose to $25bn during the first three months of 2024, up 17 per cent year-on-year By Matthew Corrigan Published: May 1, 2024 Sales rose to $25bn duri...
01/05/2024
Roku to Offer NBC Olympics Zone
NEW YORK NBCUniversal and Roku are teaming up to offer the NBC Olympic Zone on Roku, a new destination on the Roku Home Screen Menu. The dedicated channel wil...
01/05/2024
U.K.'s Freely Officially Launches
Freely the new streaming service backed by Britain's leading broadcasters BBC, ITV, Channel 4 and Channel 5 is now available through the next generation of ...
01/05/2024
Study: FAST Channel Growth Accelerates
NEW YORK A new report on free, ad-supported streaming channels indicates that their growth continues to accelerate, with the number of channels hours views and ...
01/05/2024
VuWall Unveils New Capabilities For VuStream 150 Video Encoder
MONTREAL VuWall has enhanced its VuStream 150 H.264 video encoder with new features and capabilities, including support for HDMI signals at resolutions up to 38...
01/05/2024
Samsung TV Plus Announces New High-Profile FAST Channels for Sports
NEW YORK At the IAB NewFronts 2024, Samsung Ads announced a new lineup of premium sports, music, family and entertainment content on its leading FAST service Sa...
01/05/2024
WAPA TV Celebrates 70th Anniversary As #1 Station in Puerto Rico
SAN JUAN, PUERTO RICO WAPA TV has announced that it will be celebrating its 70th anniversary on May 1 as the number 1 broadcasters on the island....
01/05/2024
Diamond Sports Group Renews Its Distribution Deal with Cox
SOUTHPORT, Conn. & ATLANTA Diamond Sports Group and Cox Communications have reached a multi-year renewal of their distribution agreement for the continued carri...
01/05/2024
Warner Bros. Discovery Networks Dropped by Fubo
As the NBA playoffs heat up on TNT, Warner Bros. Discovery and FuboTV have been unable to reach a new distribution deal. That impasse forced the vMVPD to drop T...
01/05/2024
Time for NAB Reality Check
Time for NAB Reality Check Andy Marken April 30, 2024 0 Comments Above image source - Allegiant, Summit Entertainment Great leaders don't seek...
01/05/2024
Dream Street Productions Switches Live Corporate Events with ATEM Television Studio HD8 ISO
Dream Street Productions Switches Live Corporate Events with ATEM Television Stu...
01/05/2024
lodie Ichter Joins Light Iron as Senior Colorist
lodie Ichter Joins Light Iron as Senior Colorist Brie Clayton April 30, 2024 0 Comments Acclaimed colorist brings global expertise on high-profile pro...
01/05/2024
SVG Digital Engagement Forum: Sessions Now Available to Watch on SVG PLAY
SVG Digital Engagement Forum: Sessions Now Available to Watch on SVG PLAY Catch up on the latest in streaming, interactivity, engagement and more from this inau...
01/05/2024
David Beckham Scores on the Cover of 'Queue' Issue 16
Back to All News David Beckham Scores on the Cover of Queue Issue 16 David Beckham on the cover of Queue. Krista Smith Director, Queue Entertainment 01 Ma...
01/05/2024
Haivision Celebrates 20 Years of Leadership and Innovation in Live Video
MONTREAL, CANADA - May 1, 2024 - Haivision Systems Inc. ( Haivision ) (TSX: HAI), a leading global provider of mission-critical, real-time video networking and ...
01/05/2024
MPTS. London. May 15-16, 2024
Wednesday, 15 May 11:35 The Gallows Pole - Creating a Workflow to Support the Creative Vision FilmLight sits down with the award-winning Residence Pictures t...