
Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.
Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.
NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.
Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.
In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.
A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.
DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.
On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.
The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.
In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.
It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.
A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.
The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.
And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.
These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.
The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
27/07/2026
The Audio Engineering Society Educational Foundation has named its scholarship and grant recipients for the 2026/2027 academic year. The awards support students...
27/07/2026
ESPN and the Mid-Eastern Athletic Conference (MEAC) have signed a multi-year media rights agreement covering basketball championships and select football and ba...
27/07/2026
Yahoo Sports and Cameo have partnered to allow fantasy football users to book personalized celebrity videos directly within the Yahoo Fantasy app and website. T...
27/07/2026
Eutelsat has welcomed the FCC's order establishing the regulatory framework for the transition of Upper C-band spectrum in the United States. Under the orde...
27/07/2026
ESPN and the International Dance League (IDL) have signed a global rights agreement for the 2026 season. Events will air bi-weekly on ESPN2 at 7 p.m. ET beginni...
27/07/2026
NBCUniversal and YouTube have signed a multi-year global partnership that includes bundling Peacock with YouTube Premium, extending NBCUniversal's distribut...
27/07/2026
Agenda includes presentations from the Miami Heat, Inter Miami, Florida Panthers...
27/07/2026
Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...
27/07/2026
Ten one-minute shorts will roll out across CBS, The CW, Paramount+ during the 2026 PBR Teams season
PBR is officially moving into scripted animation. The Profe...
27/07/2026
Today the nonprofit Sundance Institute shared the names of this year's cohor...
27/07/2026
New app turns vocal percussion into polished loops
Vochlea, the company behind the voice-to-MIDI instrument Dubler 2, have just announced the launch of a ne...
27/07/2026
Latest MPE-capable instrument completes Pure Trilogy line-up
Sonora Cinematic have just released the third and final instalment in their series of acoustic ...
27/07/2026
Emulates four varieties of optical compression
The latest addition to the fedDSP line-up has just been announced, and packs four distinct optical compressor...
27/07/2026
Optical & FET compressors join 500-series line-up
At our recent GearExpo UK show, we heard from Drawmer that they would soon be launching a pair of new 500-...
27/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/07/2026
Lightware announces SNMPv3-based monitoring support for selected Taurus devices, available from firmware v1.20.0. The update enables IT administrators and AV in...
27/07/2026
Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, will showcase its vision for the Real-Time...
27/07/2026
All episodes available Thursday 6 August on Sky and streaming service NOWMonday ...
27/07/2026
Why Confidence MattersIn this series, we explore the technologies, architectures and operational realities shaping modern media operations. Along the way, we ex...
27/07/2026
Apple today announced AppleCare One - a new way for customers to protect multipl...
27/07/2026
Yesterday witnessed truly memorable scenes in Croke Park, bringing the curtain d...
27/07/2026
Luxembourg, July 27, 2026 - SES, a leading space solutions company, issued the following statement following the U.S. Federal Communications Commission approval...
27/07/2026
Open source software is a critical pillar of the global economy. It underpins cloud computing, financial services, manufacturing, telecommunications, government...
26/07/2026
What is Audio Vivid? A New Destination for Immersive Audio Over the past several years, immersive audio has evolved from a niche production consideration into a...
26/07/2026
The complexity of modern chip design continues to grow as engineering teams work to develop increasingly sophisticated CPUs, GPUs and AI systems. To help meet t...
25/07/2026
New app turns vocal percussion into polished loops
Vochlea, the company behind the voice-to-MIDI instrument Dubler 2, have just announced the launch of a ne...
25/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
25/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
25/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
25/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
25/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
25/07/2026
Export FCP7 to XML Without Needing FCP7 Installed
Dave Rice July 24, 2026
0 Comments
I've created an open source tool that allows you to export FC...
25/07/2026
Documentary Sell Your House Edited with DaVinci Resolve Studio
Brie Clayton July 24, 2026
0 Comments
Filmmakers post feature documentary from opposite...
25/07/2026
Time to Act: Bringing a New Opera to Life Boston Conservatory at Berklee vocal arts students played a vital role in shaping the opera's creative developme...
25/07/2026
Beyond the Score For Dean of Music Kevin Haden, the boundless possibilities of classical music begin with process.
July 24, 2026
By
Kevin Haden
Michelle ...
25/07/2026
Reading the Room As music director of two major orchestras, conductor Jonathon Heyward aims to make the concert hall a space for everyone.
July 24, 2026
By
...
25/07/2026
Saturday 25 July 2026
First look revealed for YAGA, a seductive mystery thrille...
24/07/2026
Referee hat cameras, player/coach mics, and virtual graphics will be deployed at the three-day event for the first time
The NFL FLAG Championships return to ES...
24/07/2026
Qvest, Ateme, and Scaleway have partnered to offer broadcasters, media organizat...
24/07/2026
LTN has completed satellite-to-IP migrations for more than 3,000 broadcaster, MVPD (Multi-Channel Video Programming Distributor), head-end, and content owner si...
24/07/2026
Chyron has released Virtual Placement 8.1, an update to its live sports virtual graphics platform. The release adds workflow and graphics changes across chroma ...
24/07/2026
Riedel's RefCam Live was used for the first time at an international soccer match during a friendly between the German men's national team and Ghana in ...
24/07/2026
USSI Global said it is prepared to help broadcasters, satellite network operators, and media organizations manage the next phase of C-band spectrum clearing, fo...
24/07/2026
FuboTV and The Athletic have launched The Athletic Video Hub on Fubo, bringing The Athletic's sports video content to connected TV (CTV) for the first time....