
Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.
Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.
NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.
Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.
In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.
A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.
DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.
On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.
The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.
In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.
It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.
A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.
The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.
And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.
These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.
The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
09/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/09/2026
Studio Technologies, a leading manufacturer of high-quality audio, video, and fiber-optic solutions, introduces its new Model 791 Control Console at IBC 2026 (N...
09/09/2026
Maxon, maker of powerful, approachable software solutions for creators working in 2D and 3D design, motion graphics, visual effects and visualization, is headin...
09/09/2026
NUGEN Audio unveils DialogCheck v1.2, the latest evolution of its intelligent dialog intelligibility and compliance tool, at IBC 2026 (Pod P27). With a sharpene...
09/09/2026
Trusted Partner Network Launches Fix It or Risk It to Help the Entertainment Industry Tackle Everyday Security Risks
Industry-Wide Campaign Launches at IBC2...
09/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/09/2026
AI has moved firmly beyond experimentation in UK advertising, with more than eight in ten marketers (83%) now actively using or piloting AI in production, accor...
09/09/2026
IBC2026 - Stand 7.D22
At IBC2026, Pixitmedia by DataCore will showcase its latest capabilities designed to simplify distributed production, accelerate workflow...
09/09/2026
Stockholm, Sweden September 9th, 2026 At IBC2026, on booth 1.B47, Net Insight will showcase Open Live Media, its architecture for delivering resilient live ...
09/09/2026
Net Insight today announced that DMC Production Norway is continuing the use of its Open Live Media technology for premium European football, deploying Nimbra 5...
09/09/2026
Amagi, the media industry cloud platform for unified broadcast, streaming, and monetization, will showcase four new AI-native capabilities at IBC2026, Hall 5, S...
09/09/2026
Limecraft today announces version 2026.6 of its AI-powered production platform, extending capabilities for connected media operations. An AI-assisted approach t...
09/09/2026
Telestream, a global leader in media workflow technologies, today announced the next evolution of its Practical AI strategy, evolving its comprehensive portfoli...
09/09/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the appointment of Brian Doheny as Chief Executive Officer (CEO)....
09/09/2026
LiveU is now a Clarus Integrated Reseller, bringing Starlink's fast, reliable broadband directly into LiveU's cellular bonding EcoSystem through a singl...
09/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/09/2026
Wisycom is showcasing the latest additions to its ecosystem of professional RF technologies at IBC 2026 (Stand 8.D60). This includes its new MATF Wideband Anten...
09/09/2026
AMSTERDAM, IBC2026 VisualOn, Inc. and Cires21 today released an updated joint whitepaper at IBC 2026 demonstrating measurable, high-impact improvements in visu...
09/09/2026
Sony Electronics Introduces Its First Fisheye Zoom G Lens
Brie Clayton September 8, 2026
0 Comments
Compact, lightweight SEL814G Full-frame lens deliv...
09/09/2026
Blackmagic Design Announces DaVinci Resolve 21.1
Brie Clayton September 8, 2026
0 Comments
Major update adds AI assistant integration, expanded camera...
09/09/2026
Cuez Integrates AI into a Unified Broadcast Production Workflow
Brie Clayton September 8, 2026
0 Comments
Cuez announces the expansion of its cloud-ba...
09/09/2026
At the IBC conference, running Sept. 11-14 in Amsterdam, the creative, technology and business communities are coming together to turn ideas into action and dis...
09/09/2026
RT brings three packed days of live radio, entertainment, news, weather and fun
Join RT Radio 1: Morning Ireland, Oliver Callan, Today with David McCullagh,...
08/09/2026
Comcast Technology Solutions (CTS) has announced a multi-year agreement with A E...
08/09/2026
Spectra Logic today announced the availability of Media Insight, a new capabilit...
08/09/2026
Rogers Communications and Quebecor have announced a 12-year sublicensing agreement for French-language national NHL games beginning with the 2026-27 season. TVA...
08/09/2026
Skyline Communications and Bridge Technologies will demonstrate two integrations...
08/09/2026
Boland Communications will make the European debut of its 4K55HDR7SNB 55-inch super-narrow-bezel monitor at IBC2026 (Stand 12.C57, RAI Amsterdam, September 11-1...
08/09/2026
Ateme and Comcast Technology Solutions (CTS) have announced a collaboration comb...
08/09/2026
Haivision has joined the IBC Accelerator Media Innovation Programme as part of t...
08/09/2026
Blackmagic Design has announced price reductions on its PYXIS 6K and Cinema Camera 6K models, effective immediately through Blackmagic Design resellers worldwid...
08/09/2026
AI-Media will demonstrate its LEXI Suite at IBC2026 (Booth 5.C34, September 11-1...
08/09/2026
Bitmovin and Ad Insertion Platform (AIP) have announced the first live deploymen...
08/09/2026
Grass Valley will demonstrate story-centric media workflows at IBC2026 (Booth 9.A01, RAI Amsterdam, September 11-14), connecting newsroom production, content ma...
08/09/2026
M2A Media and Ateme have announced a partnership integrating Ateme's video processing solutions into M2A Media's MediaToAnywhere platform for live sport...
08/09/2026
Ikegami will exhibit at IBC2026 (Stand 12.A31, RAI Amsterdam, September 11-14), including a Day 1 introduction of new cameras in the UNICAM XE series. Additiona...
08/09/2026
Broadcast industry analysts will be gathering plenty of insights at the 2026 IBC Show this weekend in Amsterdam, but Caretta Research has offered up some though...
08/09/2026
SENSHI and DAZN have announced a global broadcast partnership that will bring SENSHI's live events free to viewers in more than 200 markets. The partnership...
08/09/2026
New architecture built with Evertz will bring 4K HDR to Monday Night Football, N...
08/09/2026
Built for rap producers, by rap producers
Sauceware Audio, the independent plug-in company founded by Grammy-nominated producer Jacob (Travis Scott, 21 Sava...
08/09/2026
Classic Sound Techniques design revived
Warm Audio's latest release delivers a meticulous recreation of the preamp circuitry employed by the Sound Techn...
08/09/2026
Latest soft synth inspired by the Memorymoog
XILS Lab have just launched a software instrument that draws its inspiration from Moog's iconic Memorymoog,...
08/09/2026
The Airport Chaplain takes off as SBS's biggest launch of 2026
8 September, 2026
Media releases
1.67 million viewers on National Total TV to date
Best...
08/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...