
Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.
Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.
NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.
Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.
In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.
A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.
DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.
On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.
The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.
In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.
It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.
A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.
The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.
And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.
These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.
The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
Avid will unveil how it is transforming production to help media organizations unite content, agentic workflows, and business insights at IBC2026 (RAI Amsterda...
21/08/2026
New routing, modular processing, orchestration and monitoring capabilities bridge dedicated hardware and software-based processing as part of the Dynamic Media ...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
21/08/2026
When hundreds of thousands of people gather to hear a single voice, there is no room for compromise. Every word must be delivered clearly, every wireless transm...
21/08/2026
Telestream, a global leader in media workflow technologies, today announced a strategic partnership with ScorePlay, the AI-powered media infrastructure that spo...
21/08/2026
NUGEN Audio announces the launch of its new Post Plan and Producer Plan, two flexible subscription offerings designed to give audio professionals short term acc...
21/08/2026
Join the Screen Currency 2026 Ask Me Anything (AMA) Online 20 August 2026
(L-R): Adrian Coates, Grainne Brunsdon, Deirdre Brennan, Rachael Perkins
Got questio...
21/08/2026
Beat The Lotto dives into the plot behind the 1980s National Lottery scandal
24 August 2026 at 9:35pm on RT One and RT Player
Beat The Lotto is the story o...
21/08/2026
The countdown to the most wonderful night of the year has officially begun!
Host Patrick Kielty is gearing up for another unforgettable Late Late Toy Show nig...
21/08/2026
Meet puppet duo Bellie and Whizz, who are about to explore all the big feelings ...
20/08/2026
From host-broadcaster and rightsholder support in June and July to MLS, NWSL, US...
20/08/2026
In-venue and creative video staffers at the professional and collegiate level ha...
20/08/2026
At IBC this year, Veritone (14.A56) is bringing two connected themes to the booth: architectural independence in enterprise AI, and the automation of the media ...
20/08/2026
Telestream and ScorePlay have announced a partnership combining Telestream's...
20/08/2026
The Grand Slam of Curling and Rock League have announced a three-year agreement with ESPN, bringing more than 300 hours of live curling coverage to U.S. audienc...
20/08/2026
Nielsen has announced a series of enhancements to its Big Data + Panel currency measurement methodology, taking effect August 31. The updates were developed in ...
20/08/2026
Net Insight has been selected by service provider Big Blue Marble (formerly ORS Group) to provide its Nimbra platform as the core media transport solution for a...
20/08/2026
The 2026 US Open will bring a series of free fan activations across New York City ahead of and during the tournament, which runs August 23-September 13.
We...
20/08/2026
Bridge Technologies will announce a transport stream (TS) playout feature for its VB330 and VB220 compressed monitoring probes at IBC2026 (Booth 1.A71).
The fe...
20/08/2026
ENCO will demonstrate solutions across captioning, translation, AI production tools, automation, and virtual production at IBC2026 (Stand 7.A34, Amsterdam RAI, ...
20/08/2026
MultiDyne Video and Fiber Optic Systems will demonstrate camera robotics and mot...
20/08/2026
Chyron will demonstrate news workflow solutions at IBC2026 (Stand 7.A23), marking the company's 60th anniversary. Highlights include updates to PRIME HTML, ...
20/08/2026
Spectrum SportsNet LA, the exclusive television home of the Los Angeles Dodgers, is now available to Spectrum customers in former Cox markets following the clos...
20/08/2026
swXtch.io will unveil expanded Global SRT Distribution and native multicast tran...
20/08/2026
Riedel Communications has announced that Transformation Church in the United States has deployed a communications system built around the Artist intercom ecosys...
20/08/2026
The bestselling author and longtime Apple observer explores how product launches...
20/08/2026
The Orlando Magic entered last season with a dramatically upgraded in-venue prod...
20/08/2026
Pacers Sports and Entertainment has announced a partnership with DAZN, making it the exclusive streaming home of Indiana Pacers basketball. A team-branded strea...
20/08/2026
Ratings Roundup is a rundown of recent ratings news and is derived from press re...
20/08/2026
The NHL has announced two major developments tied to its Germany growth strategy...
20/08/2026
Set up your own private remote collaboration infrastructure
SoundBridge have recently put together a tutorial video that provides users with a step-by-step ...
20/08/2026
New V-Control Pro features restore basic functionality
Neyrinck's V-Control software has long been keeping older Avid control surfaces compatible with n...
20/08/2026
TV, BritBox and Mammoth Screen have announced that filming is now underway on th...
20/08/2026
New York August 19, 2026 Nielsen today shared a number of key enhancements t...
20/08/2026
New Gracenote Data Hub analysis finds sports program titles up 31.2% and individ...
20/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/08/2026
At IBC, Profuz Digital will showcase at stand 8.A88 the SubtitleNEXT platform delivering innovative AI transcription integrations and stunning new features boos...
20/08/2026
Riedel Communications today announced that Transformation Church, one of the fastest-growing and most influential digital ministries in the United States, has d...
20/08/2026
QuickLink Showcases New 4K Capabilities Across StudioEdge and StudioCall at IBC ...