Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

14/05/2026

Sweetwater and Airstream Unveil Mobile Dolby Atmos Recording Studio

Sweetwater and Airstream have announced a custom-built Dolby Atmos mobile recording studio inside an Airstream trailer, set to tour music festivals, schools, tr...

14/05/2026

American Association of Professional Baseball Expands Broadcast Distribution for 2026 Season

The American Association of Professional Baseball (AAPB) has announced a new par...

14/05/2026

ESPN to Establish Week-Long Super Bowl LXI Broadcast Center on Santa Monica Beach

ESPN has announced plans to transform Santa Monica Beach into a broadcast hub du...

14/05/2026

Amagi Announces Major Upgrade to CLOUDPORT Cloud Broadcast Platform

Amagi has announced a significant update to Amagi CLOUDPORT, its cloud-based broadcast playout platform. The update includes 250-plus features shipped in FY25-2...

14/05/2026

Clear-Com to Showcase New Products and Platform Updates at InfoComm 2026

Clear-Com will exhibit at InfoComm 2026 (Booth N7005, June 17-19, Las Vegas Convention Center), introducing a new product that builds on Arcadia Central Station...

14/05/2026

Ikegami to Exhibit at BroadcastAsia 2026 with Two New Viewfinder Premieres

Ikegami will exhibit at BroadcastAsia 2026 (Stand 5D3-1, Singapore Expo, May 20-22), introducing two new viewfinders alongside its existing camera, control, and...

14/05/2026

dB Broadcast Delivers IP-Based OB Trucks for Cloudbass Featuring Grass Valley LDX 100 Cameras

Grass Valley has announced that dB Broadcast has delivered new IP-based outside ...

14/05/2026

NAGRAVISION and WPBSA Launch Play Snooker Digital Platform

NAGRAVISION, a Kudelski Group company, has announced a partnership with the World Professional Billiards and Snooker Association (WPBSA) to launch Play Snooker,...

14/05/2026

Belden To Acquire RUCKUS Networks for Approximately $1.85 Billion

Belden Inc. has announced a definitive agreement to acquire RUCKUS Networks from Vistance Networks for approximately $1.85 billion. The transaction has been app...

14/05/2026

NVIDIA Releases Content Localization Blueprint for AI-Assisted Dubbing and Speech Translation

NVIDIA has released the Content Localization Blueprint, a modular reference arch...

14/05/2026

Disney+ To Stream Banana Bowl Championship Live This October

Disney has announced that Disney will be the exclusive U.S. streaming home of the Banana Bowl, the Banana Ball league season championship, streaming live this ...

14/05/2026

Arkona and Manifold To Exhibit on Magna Systems Stand at BroadcastAsia 2026

Arkona technologies and technology partner manifold will demonstrate their production solutions on the Magna Systems and Engineering stand (Booth 5D1-1) at Broa...

14/05/2026

Haivision Webinar: New Broadcast Contribution Products Featuring Minor League Baseball Case Study

Haivision will host a webinar on Thursday, May 21 at 10 a.m. ET / 4 p.m. CET cov...

14/05/2026

The CW Network and ESPN Announce ACC Sublicense Agreement Through 2030-31 Season

The CW Network and ESPN have announced a sublicense broadcast agreement for The CW to televise ACC football and men's and women's college basketball gam...

14/05/2026

Detroit Pistons Ink Local Media Rights Deal With Scripps Sports

The agreement marks Scripps Sports' first NBA local rights deal...

14/05/2026

Report: Broadcasting Among Hardest-Hit Industries as AI Reshapes the Workforce

A new report from education-technology company Wiingy testing post-ChatGPT predictions against three years of real-world data has identified broadcasting as one...

14/05/2026

Madonna, Shakira, and BTS to Headline First-Ever FIFA World Cup Final Halftime Show

Global Citizen and FIFA have announced that Madonna, Shakira, and BTS will headl...

14/05/2026

Sundance Institute Names 2026 Episodic Lab Fellows

LOS ANGELES, CA, May 14, 2026 - The nonprofit Sundance Institute announced today the cohort selected for the 2026 Episodic Lab program, taking place at Dunaway ...

14/05/2026

Spotify Expands Music Access for Young Listeners, Extending Managed Accounts to Free Tier

At Spotify, we're focused on making every listening experience feel intentio...

14/05/2026

Spotify Brings Nashville's Songwriting Community Together for Mental Health Summit

Spotify recently welcomed songwriters, artists, executives, and music students t...

14/05/2026

Sonuscore update Lux Orchestral Strings

New articulations, ostinatos, Motion Scoring Articulation Sets & more Sonuscore's flagship cinematic string library has just been treated to a significa...

14/05/2026

Pr Recording & Residence

World-class studio opens on T rkiye's Aegean coast P r Recording & Residence have announced their official opening, introducing a new world-class reside...

14/05/2026

Nugen Audio update DialogCheck

Now supports channel layouts up to 9.1.6 Nugen Audio have just released an update for their AI-powered dialogue intelligibility and compliance tool. Set to ...

14/05/2026

UVI release Orchestral Suite 2

New recordings & one-key chord tool UVI have just announced the release of Orchestral Suite 2, a ground-up redesign of their all-in-one symphonic orchestra ...

14/05/2026

Rob Papen launch eXplorer 11

Two new arrivals & expanded factory content Rob Papen's all-encompassing plug-in and virtual instrument collection has just been treated to another upda...

14/05/2026

FSK Audio release QRonicle

Embed QR codes into DAW sessions FSK Audio's latest plug-in doesn't process audio, but serves as an organisational tool that allows QR codes to be e...

14/05/2026

SBS Board appoints Jane Palfreyman Managing Director

SBS Board appoints Jane Palfreyman Managing Director 13 May, 2026 Media releases The Special Broadcasting Service (SBS) Board of Directors is pleased to an...

14/05/2026

Transforming bold ideas into market-ready productions: Digital Originals returns

Transforming bold ideas into market-ready productions: Digital Originals returns 14 May, 2026 Media releases SBS, NITV and Screen Australia have announced ...

14/05/2026

Australia Uncovered Returns to SBS with Bold New Season Featuring John Safran on Free Speech and Powerful Stories of Justice and Survival

Australia Uncovered Returns to SBS with Bold New Season Featuring John Safran on...

14/05/2026

Rohde & Schwarz transforms spectrum complexity into situational awareness and effective countermeasures at AOC Europe 2026

Rohde & Schwarz transforms spectrum complexity into situational awareness and ef...

14/05/2026

Rohde & Schwarz and Quantum Systems join forces to redefine EW and C-UAS-enabled uncrewed operations

Rohde & Schwarz and Quantum Systems join forces to redefine EW and C-UAS-enabled...

14/05/2026

Rohde & Schwarz showcases STANAG aligned ARDRONIS Counter UAS capability at NATO Technical Interoperability Exercise 2026

Rohde & Schwarz showcases STANAG aligned ARDRONIS Counter UAS capability at NATO...

14/05/2026

Code of Silence wins Best Drama Series at the 2026 BAFTA's!

Code of Silence has won the BAFTA for Best Drama Series at Sunday night's ceremony at the Royal Festival Hall. The series, starring Rose Ayling-Ellis and w...

14/05/2026

The Wraith Shield Advantage: Transforming L3Harris Radios into AI-Enabled Counter-UAS Sensors

Soldiers equipped with Falcon IV radios will soon gain a sense-and-protect capa...

14/05/2026

Getting into the Space Nuclear Power Game with Next-Generation Technology

Artists concept of the L3Harris Next Gen RTG in flight configuration, designed to provide 250 watts of reliable power for decades-long missions in deep space....

14/05/2026

Vivid Broadcast builds remote production network around Calrec

Vivid Broadcast was embracing remote production long before it became the industry norm. Now, with Calrec's True Control 2.0-enabled Argo M and Type R conso...

14/05/2026

Nielsen data shows NZ vehicle advertisers are shifting gears as fuel pressures make EVs and hybrids an increasingly attractive option

Car ad spend rises sharply in March as more auto buyers turn to electric, hybrid...

14/05/2026

Is This the Year for Agentic AI's Breakout in Broadcast?

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

CBS LA, Los Angeles Rams Ink New TV Deal

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

CueScripts CueiT 4 0 Wins Futures Best of Show Award Pres...

CueScript's CueiT 4.0 Wins Future's Best of Show Award, Presented at 2026 NAB Show by TV Tech CueScript, a leading international developer of professio...

14/05/2026

Expert-Led Education Sessions and Development of Online T...

Expert-Led Education Sessions and Development of Online Training Program Accelerate IPMX Adoption and Deployment The Alliance for IP Media Solutions (AIMS) to...

14/05/2026

Klvr rechargeable battery launches in the USA to cut cost...

Klvr is launching in the United States with a professional-grade rechargeable battery solution that cuts costs and improves performance across live entertainmen...

14/05/2026

Shooting into the depths of Bedlam with URSA Cine 17K 65

Shooting into the depths of Bedlam with URSA Cine 17K 65 Brie Clayton May 14, 2026 0 Comments Indie feature film paired digital 65mm capture with a Bl...

14/05/2026

WeMakeColor expands with Baselight, becoming hybrid color facility

WeMakeColor expands with Baselight, becoming hybrid color facility Caroline Shawley May 14, 2026 0 Comments Boutique Mexican-based studio integrates B...

14/05/2026

Berklee's Summer in the City Returns with Free Concerts Throughout Boston Area

Berklee's Summer in the City Returns with Free Concerts Throughout Boston Ar...

14/05/2026

Chelsey Green Named to Billboard's 2026 Women in Music List

Chelsey Green Named to Billboard's 2026 Women in Music List The Berklee professor and chair of the Recording Academy Board of Trustees joins other high-pr...