Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.
Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.
NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.
Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.
In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.
A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.
DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.
On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.
The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.
In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.
It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.
A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.
The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.
And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.
These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.
The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
06/07/2026
Long-developed relationships and informal access, three production models, and c...
06/07/2026
The deal follows Lumine's recent acquisition of Synamedia's video-networ...
06/07/2026
This former SportsCenter producer is shaping local-team studio coverage to surro...
06/07/2026
IMS Productions and FOX Sports deployed more than 100 cameras, 32 onboard feeds,...
06/07/2026
Lumine Group Inc. has announced the completion of its acquisition of the Video Network Business from Synamedia. The business will operate as a standalone compan...
06/07/2026
NASCAR has announced a content and commercial partnership with Motorsport Network, a motorsports and automotive media platform, to distribute NASCAR original co...
06/07/2026
Versant Media Group has announced a definitive agreement to acquire Full Swing, a sports technology company, from Bruin Capital and a group of minority investor...
06/07/2026
DAZN has announced global broadcast rights to 3ICE, a professional hockey format featuring three skaters and a goalkeeper per side, with no power plays or offsi...
06/07/2026
Bundle gains 100 new synths & more
Synth Anthology 5 the largest and most feature-packed edition of UVI's hardware synth emulation bundle, bringing toge...
06/07/2026
Available now via free firmware update
MOTU have just announced that three of their popular audio interfaces now boast official Milan Certification. The 16A...
06/07/2026
Rohde & Schwarz completes Hunter Preliminary Design Review for the Integrated Co...
06/07/2026
The R&S IMAGER from Rohde & Schwarz uses millimeter wave imaging to inspect the ...
06/07/2026
The National Film and Video Foundation (NFVF), an agency of the Department of Sp...
06/07/2026
Digital Domain Appoints Industry Veteran Patrick Davenport as Executive Vice Pre...
06/07/2026
Gravity Media USA appoints Brittney Boston as Head of Business Development
Brie Clayton July 6, 2026
0 Comments
The strategic new appointment supports...
06/07/2026
Remote Editing Without Cloud Storage in Resolve
Michael Cioni July 6, 2026
0 Comments
This is a step-by-step guide that demonstrates how to setup your...
06/07/2026
Automatic Caption Translation in Adobe Premiere Pro
Colin Smith July 6, 2026
0 Comments
Here I showcase the innovative automatic caption translation f...
06/07/2026
Flavor's First Ever Managing Director Neal Cohen Retires; Longstanding Execu...
06/07/2026
How Adam Rosenwach Pivoted from Music to Medtech Without Missing a Beat What do the rehearsal room and the boardroom have in common? More than you might think...
06/07/2026
Zhongyi Optics unveils the Dreamy Zone T1 FF Cine Lens Series on Indiegogo with ...
06/07/2026
Techtel debuts insyght: making ST 2110 media discovery easy
intellisphereinsyght2026
6 July Written By Suzanne Costello
Sydney, Australia - 06 July 2026 T...
06/07/2026
Monday 6 July 2026
Sky agrees to acquire ITV Media & Entertainment, creating a ...
06/07/2026
Fresh from his World Cup heroics with Cabo Verde, Roberto Pico' Lopes will join the RT panel for Argentina v Egypt tomorrow and for the England v Norway q...
06/07/2026
Nations have long invested in domestic infrastructure to advance their economies, protect and use their data, and take advantage of technology opportunities in ...
06/07/2026
Every year, the International Conference on Machine Learning (ICML) reveals where thousands of AI researchers have decided to put their work.
This year's ...
06/07/2026
A death can be unsuspicious, but it has to be explained
What happened to Ke...
05/07/2026
We've gone to another level of integrating the graphics into the set architecture' - Zac Fields, SVP, Graphic Technology and Integration, FOX Sports...
05/07/2026
Browser-based and desktop apps available
Digital Sunset Studios have been busy working on an interesting project: a free software tool that provides compute...
04/07/2026
When Canada qualified for the FIFA World Cup 2026, OneSoccer knew this wasn'...
04/07/2026
Celebrates company's 80th anniversary
Rhodes have recently revealed that they will be producing a limited run of electric pianos in celebration of their...
04/07/2026
Blackmagic Design Cameras Empower Youth Broadcasting Program
Brie Clayton July 3, 2026
0 Comments
Blackmagic Pocket Cinema Camera 6K Pro and Blackmagi...
04/07/2026
iZotope Joins Boris FX
Boris Yamnitsky July 3, 2026
0 Comments
iZotope, the team behind RX and Ozone, is joining Boris FX. A letter from founder Boris...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
03/07/2026
New bundle & three Single Packs
Continuing to expand their already sizeable orchestral collection, VSL's latest release introduces three new Single Pack...
03/07/2026
Venue installs new ATC-equipped control room
LSO St Luke's, a unique music venue that also serves as the home of the London Symphony Orchestra, have rec...
03/07/2026
Caden Pearson appointed new Commissioning Editor for NITV
3 July, 2026
Media releases
National Indigenous Television (NITV) has strengthened its commitment...
03/07/2026
1 February 2023
SHARE Facebook Twitter Linkedin Email
Munich, Germany, 1st February 2023: Cinegy GmbH, the premier provider of software technology for digit...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Death Star
Andy Marken July 2, 2026
0 Comments
Look, I can't get involved. I've got work to do. It's not that I like the Empire; I hate i...
03/07/2026
Berklee's New Visual Identity: Honoring Our History, Building for What's...
03/07/2026
Scripps Research scientists awarded $2M to advance global disease surveillance Two Gates Foundation grants will expand wastewater surveillance and AI-driven dis...
03/07/2026
Joan Pulupa joins Scripps Research faculty to study the organization of DNA in brain cells and its links to neurodegeneration Using smell-sensing neurons and ad...
02/07/2026
Entering her senior year, this hometown girl is paving a career in live sports production gaining experience in replay and audio and as a TD
In the live-sports...
02/07/2026
In-venue and creative video staffers at the professional and collegiate level ha...
Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
