Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

08/03/2021

Khalifa University's Second CubeSat DhabiSat' Lifts Off Aboard Cygnus Spacecraft to Inter...

Abu Dhabi-UAE: 20 February 2021 Khalifa University of Science and Technology, ...

08/03/2021

AMD and Xilinx Special Meetings of Stockholders to be Held on April 7, 2021

AMD and Xilinx Special Meetings of Stockholders to be Held on April 7, 2021 Recommend Stockholders Vote For the Acquisition and Other Proposals Set Forth in t...

08/03/2021

VEON appoints Artashes Davtyan as CEO of Beeline Uzbekistan

Amsterdam, 8 March 2020 - VEON Ltd. (NASDAQ and Euronext Amsterdam: VEON), a leading global provider of connectivity and internet services, announces the appoin...

08/03/2021

Keeping Connected at a Distance

The media and entertainment sector has experienced disruption on a level no one could have previously imagined since the COVID-19 pandemic began. It has forced ...

08/03/2021

LPTV Digital Transition Deadline Approaches, Reminds Media Bureau

WASHINGTON-The FCC Media Bureau has reminded low power television and TV translator stations of the approaching July 13 deadline to complete their digital trans...

08/03/2021

COVID Relief Bill Provides $175M to Public Broadcasters

WASHINGTON-As part of the $1.9 trillion American Rescue Plan for COVID relief approved by the Senate over the weekend, $175 million has been earmarked as emerge...

08/03/2021

Blackbird Teams Up With EVS on International Sporting Events

LONDON & LIEGE, Belgium-Blackbird and EVS have announced a partnership that will have the two companies working together on launch deployments for international...

08/03/2021

Kagan: 7.2M MVPD Subscriptions Lost in 2020

NEW YORK-Nearly 7.2 million traditional multichannel (MVPD) subscribers opted to cancel their subscriptions in 2020, according to a recent report from Kagan, an...

08/03/2021

Turner Sports Updates NCAA March Madness Live for Tourney's Return

ATLANTA-The NCAA Men's Basketball Tournament, better known as March Madness, is set to return after not taking place in 2020 because of the COVID-19 pandemi...

08/03/2021

NAB Counters Facebook Proposal for 6 GHz VLP Operation

WASHINGTON-The NAB has positioned itself against another proposal for unlicensed operations in the 6 GHz, this one submitted to the FCC by Facebook, Broadcom, C...

08/03/2021

Rise Launching Mentoring Program in North America

LONDON-Rise, an advocacy group for gender diversity in the broadcast technology media sector, has announced the details of its 2021 mentoring programs, which fo...

08/03/2021

All-Women Broadcast Team Will Call Toronto Raptors Game

TORONTO-As part of the Toronto Raptors' celebration of Women's History Month this March, the team has announced that its March 24 game against the Denve...

08/03/2021

Giving Sudanese a voice in rebuilding their economy

Rebuilding Sudan's shattered economy after the 2019 uprising requires the country's journalists to understand and explain the key economic issues to the...

08/03/2021

Telenor Group's response to proposed Myanmar Cyber Security Bill (Updated 8.3.2021)

Reading Time: 3 minutes Update 8 March 2021: In the time since Telenor Group p...

08/03/2021

Wiring the 2002 Winter Olympics With Tom Sahara: A Sports Broadcasting Hall of Fame Sit-Down

Wiring the 2002 Winter Olympics With Tom Sahara: A Sports Broadcasting Hall of F...

08/03/2021

University of Florida Ushers in New Era of Baseball Excellence With Florida Ballpark

University of Florida Ushers in New Era of Baseball Excellence With Florida Ball...

08/03/2021

International Women's Day 2021: SVT's sa Edlund Jnsson Talks Ambition, Adrenaline, and Live Sports Addiction

Swedish empire: SVT's sa Edlund J nsson talks ambition, adrenaline and live...

08/03/2021

International Women's Day 2021: The Adrenaline-Fueled World of Winter Sports Junkie Aurelie Gonin

Positive pressure of directing live: The adrenalin-fuelled world of winter sport...

08/03/2021

International Women's Day 2021: The Rise of BBC Sport's Debbie Dubois

Straight from school to head of production: The rise of BBC Sport's Debbie Dubois International Womens Day 2021 focus By Heather McLean, Editor Thursday,...

08/03/2021

International Women's Day 2021: Eurosport's Aurelia Mounier on the Fascination and Drama of Producing Live Tennis

Game, set and match: Eurosport's Aurelia Mounier on the fascination and dram...

08/03/2021

International Women's Day 2021: Sky Sports' Jennie Blackmore on Balancing Being a Parent with the Thrill of Live Sports Production

Boxing clever: Sky Sports' Jennie Blackmore on balancing being a parent with...

08/03/2021

International Women's Day 2021: Martiene Bruggink on Life in Editorial and Production for ESPN, UEFA, and the KNVB

Freelance lifestyle: Martiene Bruggink on life in editorial and production for E...

08/03/2021

Celebrating International Women's Day With Our New Campaign: EQUAL

Spotify is always dedicated to empowering women creators and giving them a place to share their content with the world. Today, International Women's Day, we...

08/03/2021

Sony launches BRAVIA XR A80J OLED TV with cognitive intelligence in Europe

- Pre-order another of Sony's revolutionary TVs - the BRAVIA XR A80J OLED, powered by Cognitive Processor XR. Also ready for pre-ordering soon are the entry...

08/03/2021

Sky Sports partner with the Women's Sport Collective

Sky Sports has today launched a new partnership with the Women's Sport Collective that will see the broadcaster support the network's ambition of provid...

08/03/2021

Sky, WarnerMedia and ViacomCBS Turn on the Subtitles' for more than 500 episodes of their most popular kids' content

Sky, WarnerMedia and ViacomCBS Networks UK have partnered to improve the literac...

08/03/2021

RAF creates its first 360 Immersive Experience using Mistika VR

Published On: March 8th, 2021 -- -- -- -- width=1920 height=1080 class=wp-image-0 lazyload data-orig-src=https://www.sgo.es/wp-content/uploads/Stitchi...

08/03/2021

Blthnaid Treacy will present Today alongside Dith 'S for March

Bl thnaid Treacy will present Today alongside D ith 'S for the remainder of March as Sin ad Kennedy begins her maternity leave. Further announcements to...

08/03/2021

RT Statement: Eoghan McDermott

RT Statement RT confirms that Eoghan McDermott will not be returning to RT 2FM. His contract is due to expire shortly. RT is developing a new direction f...

08/03/2021

Rohde & Schwarz delivers new radio systems for Evenes airbase

Rohde & Schwarz delivers new radio systems for Evenes airbase Rohde & Schwarz air traffic control radios upscale Royal Norwegian Air Force base for safe and s...

08/03/2021

Industrial Radio Lab (IRL) Dresden and Rohde & Schwarz cooperate to research wireless technologies for industry 4.0

Industrial Radio Lab (IRL) Dresden and Rohde & Schwarz cooperate to research wir...

08/03/2021

In Genomics Breakthrough, Harvard, NVIDIA Researchers Use AI to Spot Active Areas in Cell DNA

Like a traveler who overpacks a suitcase with a closet's worth of clothes, m...

08/03/2021

Artists: Unleash Your Marble Arts in NVIDIA Omniverse Design Challenge

Artists, this is your chance to push past creative limits - and win great prizes - while exploring NVIDIA Omniverse through a new design contest. Called Creat...

08/03/2021

Nokia to supply Cibicom with mission critical LTE network across Denmark

Nokia to supply Cibicom with mission critical LTE network across Denmark Press Release Nokia to supply Cibicom with mission-critical LTE network across De...

08/03/2021

The NFVF funding applications to attend Markets and Festivals taking place in April and May are open

The NFVF funding applications to attend Markets and Festivals taking place in Ap...

08/03/2021

NEW JAPANESE FILM MY DEAREST, LIKE A CHERRY BLOSSOM (WORKING TITLE)

Back to All News NEW JAPANESE FILM MY DEAREST, LIKE A CHERRY BLOSSOM (WORKING TITLE) Entertainment 07 March 2021 GlobalJapan Netflix confirmed the producti...

08/03/2021

Liza Soberano And Shay Mitchell Are Alexandra Trese

Back to All News Liza Soberano And Shay Mitchell Are Alexandra Trese Entertainment 07 March 2021 Global LIZA SOBERANO AND SHAY MITCHELL ARE ALEXANDRA TRES...

08/03/2021

We Are Just A Story Away

Back to All News We Are Just A Story AwayMs. Netflix Business 08 March 2021 GlobalIndia DOWNLOAD IMAGES Cop, banker, stand-up comic, homemaker, boss, pilo...

08/03/2021

Netflix ISP Speed Index for February 2021

Back to All News Netflix ISP Speed Index for February 2021 Innovation 08 March 2021 Global For the month of February, 15 countries and regions were in the ...

08/03/2021

Kurt Sutter To Make Feature Directorial Debut With This Beast

Back to All News Kurt Sutter To Make Feature Directorial Debut With This Beast Business 08 March 2021 GlobalUnited States Kurt Sutter (Sons of Anarchy, Sou...

08/03/2021

Talent Decentralization Why IVP Invested in Papaya Global

By Cack Wilhelm and Jake Storm The push to remote work and distributed teams has been underway for years, as companies and hiring managers realize that one can...

08/03/2021

McDonald & Dodds launch watched by 6.4 million viewers

McDonald & Dodds launch watched by 6.4 million viewers The launch of the new series of ITVs mismatched detectives, McDonald & Dodds, was viewed by 6.4 million ...

08/03/2021

Deborah Turness appointed as ITN CEO

March 8th 2021 Deborah Turness will be joining ITN as its Chief Executive Officer from NBC News International where she is president. Deborah became the first...

08/03/2021

A flexible approach to following the roadmap

Posted on 08 Mar 2021 The first step of Englands lockdown easing has begun as part of the Governments four-stage road map. Whether youre a retail store, leisu...

08/03/2021

GSMA PRESENTS HEALTH AND SAFETY PLAN FOR MWC21

A multi-layered plan to ensure a safe environment for everyone MWC21 takes place 28 June - 1 July 2021 in Barcelona 8 March 2021, London: The GSMA has shared...

08/03/2021

Journalism and society simply cannot afford silence

Journalism and society simply cannot afford silence...

08/03/2021

CFO Optimistic About Berklees Future

CFO Optimistic About Berklees Future Challenges remain but Berklee is on track to rebound from the largest financial crisis in its history, says Mac Hisey, seni...

08/03/2021

Grammy-Nominated Songwriter Kara DioGuardi to Host Virtual Talent Competition

Grammy-Nominated Songwriter Kara DioGuardi to Host Virtual Talent Competition DioGuardis competition, Sing It Forward, provides artists with mentoring and caree...