
Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.
Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.
NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.
Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.
In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.
A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.
DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.
On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.
The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.
In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.
It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.
A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.
The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.
And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.
These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.
The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
Most recent headlines
08/03/2021
Abu Dhabi-UAE: 20 February 2021 Khalifa University of Science and Technology, ...
08/03/2021
AMD and Xilinx Special Meetings of Stockholders to be Held on April 7, 2021 Recommend Stockholders Vote For the Acquisition and Other Proposals Set Forth in t...
08/03/2021
Amsterdam, 8 March 2020 - VEON Ltd. (NASDAQ and Euronext Amsterdam: VEON), a leading global provider of connectivity and internet services, announces the appoin...
08/03/2021
The media and entertainment sector has experienced disruption on a level no one could have previously imagined since the COVID-19 pandemic began. It has forced ...
08/03/2021
WASHINGTON-The FCC Media Bureau has reminded low power television and TV translator stations of the approaching July 13 deadline to complete their digital trans...
08/03/2021
WASHINGTON-As part of the $1.9 trillion American Rescue Plan for COVID relief approved by the Senate over the weekend, $175 million has been earmarked as emerge...
08/03/2021
LONDON & LIEGE, Belgium-Blackbird and EVS have announced a partnership that will have the two companies working together on launch deployments for international...
08/03/2021
NEW YORK-Nearly 7.2 million traditional multichannel (MVPD) subscribers opted to cancel their subscriptions in 2020, according to a recent report from Kagan, an...
08/03/2021
ATLANTA-The NCAA Men's Basketball Tournament, better known as March Madness, is set to return after not taking place in 2020 because of the COVID-19 pandemi...
08/03/2021
WASHINGTON-The NAB has positioned itself against another proposal for unlicensed operations in the 6 GHz, this one submitted to the FCC by Facebook, Broadcom, C...
08/03/2021
LONDON-Rise, an advocacy group for gender diversity in the broadcast technology media sector, has announced the details of its 2021 mentoring programs, which fo...
08/03/2021
TORONTO-As part of the Toronto Raptors' celebration of Women's History Month this March, the team has announced that its March 24 game against the Denve...
08/03/2021
Rebuilding Sudan's shattered economy after the 2019 uprising requires the country's journalists to understand and explain the key economic issues to the...
08/03/2021
Reading Time: 3 minutes
Update 8 March 2021: In the time since Telenor Group p...
08/03/2021
Wiring the 2002 Winter Olympics With Tom Sahara: A Sports Broadcasting Hall of F...
08/03/2021
University of Florida Ushers in New Era of Baseball Excellence With Florida Ball...
08/03/2021
Swedish empire: SVT's sa Edlund J nsson talks ambition, adrenaline and live...
08/03/2021
Positive pressure of directing live: The adrenalin-fuelled world of winter sport...
08/03/2021
Straight from school to head of production: The rise of BBC Sport's Debbie Dubois International Womens Day 2021 focus By Heather McLean, Editor
Thursday,...
08/03/2021
Game, set and match: Eurosport's Aurelia Mounier on the fascination and dram...
08/03/2021
Boxing clever: Sky Sports' Jennie Blackmore on balancing being a parent with...
08/03/2021
Freelance lifestyle: Martiene Bruggink on life in editorial and production for E...
08/03/2021
Spotify is always dedicated to empowering women creators and giving them a place to share their content with the world. Today, International Women's Day, we...
08/03/2021
- Pre-order another of Sony's revolutionary TVs - the BRAVIA XR A80J OLED, powered by Cognitive Processor XR. Also ready for pre-ordering soon are the entry...
08/03/2021
Sky Sports has today launched a new partnership with the Women's Sport Collective that will see the broadcaster support the network's ambition of provid...
08/03/2021
Sky, WarnerMedia and ViacomCBS Networks UK have partnered to improve the literac...
08/03/2021
Published On: March 8th, 2021
--
--
--
--
width=1920 height=1080 class=wp-image-0 lazyload data-orig-src=https://www.sgo.es/wp-content/uploads/Stitchi...
08/03/2021
Bl thnaid Treacy will present Today alongside D ith 'S for the remainder of March as Sin ad Kennedy begins her maternity leave.
Further announcements to...
08/03/2021
RT Statement
RT confirms that Eoghan McDermott will not be returning to RT 2FM. His contract is due to expire shortly.
RT is developing a new direction f...
08/03/2021
Rohde & Schwarz delivers new radio systems for Evenes airbase Rohde & Schwarz air traffic control radios upscale Royal Norwegian Air Force base for safe and s...
08/03/2021
Industrial Radio Lab (IRL) Dresden and Rohde & Schwarz cooperate to research wir...
08/03/2021
Like a traveler who overpacks a suitcase with a closet's worth of clothes, m...
08/03/2021
Artists, this is your chance to push past creative limits - and win great prizes - while exploring NVIDIA Omniverse through a new design contest.
Called Creat...
08/03/2021
Nokia to supply Cibicom with mission critical LTE network across Denmark
Press Release
Nokia to supply Cibicom with mission-critical LTE network across De...
08/03/2021
The NFVF funding applications to attend Markets and Festivals taking place in Ap...
08/03/2021
Back to All News
NEW JAPANESE FILM MY DEAREST, LIKE A CHERRY BLOSSOM (WORKING TITLE)
Entertainment
07 March 2021
GlobalJapan
Netflix confirmed the producti...
08/03/2021
Back to All News
Liza Soberano And Shay Mitchell Are Alexandra Trese
Entertainment
07 March 2021
Global
LIZA SOBERANO AND SHAY MITCHELL ARE ALEXANDRA TRES...
08/03/2021
Back to All News
We Are Just A Story AwayMs. Netflix
Business
08 March 2021
GlobalIndia
DOWNLOAD IMAGES
Cop, banker, stand-up comic, homemaker, boss, pilo...
08/03/2021
Back to All News
Netflix ISP Speed Index for February 2021
Innovation
08 March 2021
Global
For the month of February, 15 countries and regions were in the ...
08/03/2021
Back to All News
Kurt Sutter To Make Feature Directorial Debut With This Beast
Business
08 March 2021
GlobalUnited States
Kurt Sutter (Sons of Anarchy, Sou...
08/03/2021
By Cack Wilhelm and Jake Storm
The push to remote work and distributed teams has been underway for years, as companies and hiring managers realize that one can...
08/03/2021
McDonald & Dodds launch watched by 6.4 million viewers
The launch of the new series of ITVs mismatched detectives, McDonald & Dodds, was viewed by 6.4 million ...
08/03/2021
March 8th 2021
Deborah Turness will be joining ITN as its Chief Executive Officer from NBC News International where she is president.
Deborah became the first...
08/03/2021
Posted on 08 Mar 2021
The first step of Englands lockdown easing has begun as part of the Governments four-stage road map.
Whether youre a retail store, leisu...
08/03/2021
A multi-layered plan to ensure a safe environment for everyone
MWC21 takes place 28 June - 1 July 2021 in Barcelona
8 March 2021, London: The GSMA has shared...
08/03/2021
Journalism and society simply cannot afford silence...
08/03/2021
PDF Version
<...
08/03/2021
TV Moving to All-IP - Dream or Reality? Deutsche Telekom, RDK, Broadpeak, and Wy...
08/03/2021
CFO Optimistic About Berklees Future Challenges remain but Berklee is on track to rebound from the largest financial crisis in its history, says Mac Hisey, seni...
08/03/2021
Grammy-Nominated Songwriter Kara DioGuardi to Host Virtual Talent Competition DioGuardis competition, Sing It Forward, provides artists with mentoring and caree...