
Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.
Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.
NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.
Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.
In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.
A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.
DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.
On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.
The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.
In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.
It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.
A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.
The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.
And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.
These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.
The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
03/09/2026
Accedo, a global provider of video streaming software and services, and MediaKind, a world-leading video technology company and the number one independent provi...
03/09/2026
International research from Big Blue Marble puts a consumer churn number against a technical failure, and sets out what streaming providers must fix before 2030...
03/09/2026
High-performance points of presence power video streaming delivery to large audiences across Latin America with guaranteed quality of experience
Broadpeak , a ...
03/09/2026
Dalet, a leading technology and service provider for media-rich organizations, will unveil the next evolution of Dalia at IBC2026 (7.A43), its media-aware agent...
03/09/2026
CGI (TSX: GIB.A) (NYSE: GIB), one of the world's largest independent IT and business consulting services firms, will showcase CGI OpenMedia Evolve and CGI d...
03/09/2026
LiveU is participating in the IBC Accelerator Incubator 2026: Smart Stories The New Knowledge Architecture for Content Production, an industry-wide initiative...
03/09/2026
Broadcast Solutions, a leading systems integrator and provider of innovative solutions for the broadcast and media industry, has taken another major step in its...
03/09/2026
Quortex today announced that it will showcase how it is accelerating innovation and the transition to IP at IBC 2026. The company will debut its newest intellig...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
The EZDRM approach to Revenue Security on streaming video can be visualized as a multi-dimensional set of solutions. A growing dimension of that approach is the...
03/09/2026
From field capture to management, control and production, Kiloview will demonstrate one connected Hybrid AV-over-IP workflow including a complete 4RU producti...
03/09/2026
Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, today announced Mediagenix Curation, a new...
03/09/2026
LucidLink, the Emmy-winning cloud-native file streaming platform for instant, secure access to large files, today announced the next step in its vision to make ...
03/09/2026
Fraudulent communications purportedly from Screen Australia 3 September 2026
Screen Australia is aware of scam attempts involving fake emails and promotional m...
03/09/2026
AJA Introduces FS8, Next-Gen 12G-SDI/SMPTE ST 2110 Media Processor
Brie Clayton September 2, 2026
0 Comments
AJA Video Systems unveiled FS8, a video a...
03/09/2026
Shooting Deadheading with Blackmagic URSA Cine 12K LF
Brie Clayton September 2, 2026
0 Comments
Bringing a small story to the big screen with dark com...
03/09/2026
OOONA Launches OOONA Go: Professional Localization Management for Small Companie...
03/09/2026
DURHAM, N.C. - East of Texas Durham has officially opened its doors at American ...
03/09/2026
First Class Takes is a new six-part series for RT KIDS that captures funny, hon...
03/09/2026
September is here with 26 more games streaming on GeForce NOW this month, led by a slam dunk: NBA 2K27 with the NVIDIA DLSS 5 3D-Guided Neural Rendering feature...
02/09/2026
Professional and collegiate clubs, leading vendors discuss current trends in stadium production
On. Aug. 12, the 2026 SVG Venues & Teams Summit explored the ev...
02/09/2026
SoFi Stadium, YouTube Theater, and Hollywood Park have completed an infrastructure upgrade expanding game presentation and broadcast production capabilities acr...
02/09/2026
Comcast Technology Solutions (CTS) has announced a suite of AI-powered workflow applications for broadcasters, content owners, and operators, built on its Video...
02/09/2026
The North American Broadcasters Association (NABA) and the Association of Professional Wireless Production Technologies (APWPT) have announced a partnership to ...
02/09/2026
Carolina Hurricanes, Anaheim Ducks, Columbus Blue Jackets, Dallas Stars, Minneso...
02/09/2026
New LED provides enhanced quality control, quicker show prep, detailed productio...
02/09/2026
Combines the Phat Drive with high-voltage valve circuit
The latest pedal from Keeley Electronics combines their popular Phat Mode drive pedal with an all-ne...
02/09/2026
Free version of virtual mixing room plug-in introduced
Lewitt's Space Replicator plug-in provides users with a virtual monitoring environment that aims ...
02/09/2026
Two new mixing & mastering plug-ins revealed
AIR Music Tech have just introduced the Stereophonic Bundle, a pair of processing plug-ins designed to provide ...
02/09/2026
Rohde & Schwarz has achieved initial verification of 3GPP Rel. 17 power-saving f...
02/09/2026
Bilbao - 3rd September 2026 -AgileTV, a leading provider of end-to-end TV and vi...
02/09/2026
Why agentic media supply chains need experience design built around visibility, reversibility and human authority
When AI can generate a functional interface ...
02/09/2026
Six-country survey finds 26% of sports fans frequently unable to determine where...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
KOKKEDAL, DENMARK, SEPTEMBER 1, 2026 DPA Microphones announces its new 6388 CORE Headset Microphone, a high-performance directional headset designed for dema...
02/09/2026
Pliant Technologies will highlight its CrewCom CXD 32CF 32x32 Dante /AES67 Digital Audio Network Interface, a powerful connectivity solution designed to expand ...
02/09/2026
iWedia, a leading provider of software solutions for connected TV devices, today announced two enhancements to Teatro Pulse, its end-to-end TV service operation...
02/09/2026
Signiant, trusted by the media industry for accelerated content movement for more than 20 years, will showcase the latest evolution of the Signiant Platform at ...
02/09/2026
With a contractual agreement to supply a leading UK sports broadcaster with outside broadcast services for 104 football matches per season over the next four ye...
02/09/2026
IBC2026 will put sport firmly at the centre of the global media and entertainment technology conversation with the strongest sports showing in the event's h...
02/09/2026
PTZOptics will bring a hands-on showcase of intelligent video production to IBC2026 (11-14 September, RAI Amsterdam), with demonstrations spanning new camera te...
02/09/2026
OOONA, a global provider of professional management and production tools for the media localization industry, today announced the launch of OOONA Go, a ready-to...
02/09/2026
EON Media will make its European debut at IBC2026 (September 11-14, RAI Amsterdam) in the show's Future Tech Zone (Hall 14.FT5). The company is launching it...
02/09/2026
Frequency, the engine behind many of the worlds leading streaming television channels, will demonstrate at IBC2026 how integrating intelligence throughout chann...