Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Scripps Completes Acquisition of WTVQ Lexington

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

U.S., U.K. Consumers Now Spend 11 Hours a Day Using Media

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

Sinclair Announces 2026 Annual Scholarship Winners

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

Tom DeLay: Only Congress Can Change Station Ownership Caps

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

IAB Tech Lab Upgrades Open Framework for Agentic Advertising

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

DirecTV For Business Inks New Agreement to Sell NFL Sunday Ticket

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

IEEE Broadcast Technology Society Honors Charles Einolf

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

Homecoming Artist Spotlight: Dreion

Homecoming Artist Spotlight: Dreion The singer, songwriter, and producer talks about finding his village at Berklee ahead of his Homecoming performance on Sep...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

03/08/2026

SVGs TranSPORT Conference Returns to NYC on Oct. 20; Register Today!

SVG's annual TranSPORT conference returns to New York City for a full day of sessions focused on the current state of live-sports-transmission technology. S...

03/08/2026

SVG All-Stars: Stefan Richardson, Director, Global Broadcast, Riot Games

A five-time Emmy winner, this esports vet has built a production career on live storytelling, audience connection, and meeting viewers wherever they are The sp...

03/08/2026

MLBs Jason Hedgcock on Using the Cloud To Produce World Baseball Classic Games for Netflix Japan

Live production in the cloud is a growing trend across the industry and Major Le...

03/08/2026

Cookie Queens Is Sweet With Substance

(L-R) Nyah B., Nala B., Shannon Elizabeth S., Celia K., Olive G., and Nikki B. attend the Q&A for Cookie Queens by Alysa Nahmias, an official selection of the...

03/08/2026

Rhodes launch Rhodes Clav Pro

Recreates a 1977 Hohner Clavinet D6 After launching a software recreation of their own electric piano, Rhodes set about capturing another classic instrument...

03/08/2026

Peak Tamer from Nicky Howard

A modern solution for an old technique Peak Tamer is a new standalone application that automatically applies gain automation to keep rogue peaks under cont...

03/08/2026

Sound Particles update Beat Panner

Plug-in gains object metadata automation export Sound Particles have just released an update that adds a new Metadata Automation function to their their rec...

03/08/2026

SGL Carbon and X-energy announce capacity expansion for Nuclear-Grade Graphite

SGL Carbon and X-energy today announced a binding agreement to double SGL's production capacity for medium-grain isotropic graphite ( NBG-18 ), a specialize...

03/08/2026

Glensound launches Greater Divine Supernal at PLASA Show...

Glensound will launch the Greater Divine Supernal at PLASA Show 2026, extending its intelligent loudspeaker range for professional studio monitoring and live au...

03/08/2026

How Agua Tinta's Class Project Launched Her Songwriting Career

How Agua Tinta's Class Project Launched Her Songwriting Career Two years after graduating from Berklee, the songwriter has a Latin Grammy nomination, a So...

03/08/2026

Lightware UCX delivers campus-wide connectivity to Leiden...

Leiden University's new Campus Spui, located in the heart of The Hague, the diplomatic and administrative centre of the Netherlands, opened in February 2026...

03/08/2026

Bitfocus heads to the future with new investment

Bitfocus, the specialist in media control and monitoring, has received a further boost to its plans for continuing growth, particularly in Europe and North Amer...

03/08/2026

Leagues Cup kicks off tomorrow, August\u00A04, on\u00A0Apple\u00A0TV - UPDATE - Posted on 03 August 2026

Building on the momentum of a landmark year for soccer in North America, Leagues...

02/08/2026

Acustica introduce Dangerous Music BAX EQ Plugin

Classic EQ design goes virtual Following on from the launch of the 500-series version of their BAX EQ unit, Dangerous Music have now teamed up with modellin...

02/08/2026

Television, Resilience and National Security

The Government's Missing Dimension This article was originally published on the National Preparedness Commission website on 25 June 2026. ----------------...

01/08/2026

Harmonic Piano from The Crow Hill Company

Latest free Vaults instrument released The Crow Hill Company's Vaults collection offers a continually rotating selection of virtual instruments and samp...

01/08/2026

American Television Alliance Slams Deltavision Blackout on Verizon

Share Copy link Facebook X Linkedin Bluesky Email...

01/08/2026

FreeWheel Debuts TV Series-Level Reporting for CTV Buyers

Share Copy link Facebook X Linkedin Bluesky Email...

01/08/2026

Minnesota Timberwolves, DAZN to Launch New Streaming Offering

Share Copy link Facebook X Linkedin Bluesky Email...

01/08/2026

Early Registration Opens for 2026 SMPTE Media Technology Summit

Share Copy link Facebook X Linkedin Bluesky Email...

01/08/2026

NBCUniversal and YouTube Ink Landmark Distribution Deal

Share Copy link Facebook X Linkedin Bluesky Email...

01/08/2026

COW Jobs: Lead Generation Representative - Animation Services

COW Jobs: Lead Generation Representative - Animation Services Brie Clayton July 31, 2026 0 Comments Lead Generation Representative - Animation Service...

01/08/2026

Emilie Lowe Shoots Award Winning Short Film Where Dead Things Grow with Blackmagic Design

Emilie Lowe Shoots Award Winning Short Film Where Dead Things Grow with Blackmag...

01/08/2026

Early Registration Opens for the 2026 SMPTE Media Technology Summit

Early Registration Opens for the 2026 SMPTE Media Technology Summit Brie Clayton July 31, 2026 0 Comments Initial Program Announced Alongside New Medi...

31/07/2026

SVG Sit-Down: NHL's Steve Mayer on Building a Centralized Production Operation From Scratch

With an emphasis on local, the league is looking beyond the initial four teams t...

31/07/2026

Reflecting on the 2026 Summer Season of Our Sundance Institute Labs

The 2026 Sundance Institute Directors Lab fellows in Estes Park, Colorado (Photo by Gabe Rovick) Dear Friends, This has been an extraordinary year for our ar...

31/07/2026

Audiocube launch Audiocube Space

Offers 360-degree sound placement in any DAW Audiocube have just announced the launch of a new plug-in that embeds a complete 3D acoustic environment inside...

31/07/2026

Sonarworks expand SoundID VoiceAI library

10 new Chinese C-Pop voices introduced While Sonarworks are best known to many for their room-correction software, their product range also includes an inno...

31/07/2026

PSPaudioware release PSP BBDelay

Reimagines the vintage BBD sound character PSPaudioware have now officially launched the new delay plug-in that they were previewing at GearExpo UK. Said to...

31/07/2026

Savannah Bananas Hits a Broadcast Home Run with Calrec

The Savannah Bananas has built its reputation on refusing to play baseball by traditional rules, and the team's production arm Banana Ball TV (BTV) has draf...

31/07/2026

How Valuable Is TV to Smaller, Independent Cable Operators?

Share Copy link Facebook X Linkedin Bluesky Email...

31/07/2026

Early Registration Opens for the 2026 SMPTE Media Techno...

SMPTE, the home for media professionals, technologists and engineers, today opened early registration for the 2026 Media Technology Summit (MTS) and announced i...

31/07/2026

Central Christian Church Connects Remote Production Teams...

Clear-Com announced that Central Christian Church has transformed communication across its live production operations by implementing Clear-Com's LQ Seri...

31/07/2026

Haivision To Feature Makito ONE, Falkon X4 At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

31/07/2026

ABC Stations Tout Community Support for License Renewals

Share Copy link Facebook X Linkedin Bluesky Email...

31/07/2026

Finalists Announced for IBC2026 Innovation Awards

Share Copy link Facebook X Linkedin Bluesky Email...

31/07/2026

IBC2026 Innovation Awards finalists unveiled honouring co...

IBC has unveiled the finalists for the IBC2026 Innovation Awards, recognising collaborative projects from around the world that solve real-world challenges and ...