
Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.
Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.
NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.
Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.
In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.
A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.
DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.
On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.
The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.
In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.
It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.
A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.
The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.
And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.
These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.
The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/09/2026
Lightware is expanding its LARA (Lightware Advanced Room Automation) platform with the GUDE Power Distribution Units (PDU) driver module. GUDE devices can now b...
04/09/2026
Lightware shares the second release in its ESG communications series. Following the first piece, which focused on the companys environmental performance, this r...
04/09/2026
Big Blue Marble supported a leading European public service broadcaster in delivering the Soccer World Cup 2026 streams, supplying more than 10 Tbps of bandwidt...
04/09/2026
Outstanding achievement in engineering, science and technology
Hitomi Broadcast has been recognised with an Engineering, Science & Technology Emmy from the Ac...
04/09/2026
California, Hollywood at a Crossroads
Andy Marken September 3, 2026
0 Comments
I'm doing everything alone. Crossing roads alone. Seeing the Eiffe...
04/09/2026
Berklee Announces Winners of 2026 SongwritersdB Songwriting Contest Students Luca Alexandru, Annika Wild, and Johanna James received the top prizes for origin...
04/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/09/2026
RT ANNOUNCE NEW DRAMA THE MURDER OF MR. MOONLIGHT
BASED ON THE DISAPPEARANCE...
03/09/2026
Yospace will be at IBC from 11th - 14th September (booth 5.C78). For more inform...
03/09/2026
NEP Group has announced leadership changes at Creative Technology Group. Founder and CEO Graham Andrews will step away from day-to-day leadership at the end of ...
03/09/2026
The NFL and TikTok have announced a multi-year partnership renewal expanding the league's presence on the platform. NFL posts on TikTok increased more than ...
03/09/2026
Grass Valley has appointed Ronny Van Geel as Vice President, Cameras. Van Geel began his career in 1999 at Philips DVS and moved into product management in 2005...
03/09/2026
Telestream has announced Vantage Super Resolution, an AI-powered upscaling capability for the Vantage media workflow platform, developed in collaboration with N...
03/09/2026
Solid State Logic (SSL) will debut the TCA Tour at IBC2026 (Hall 8, Stand 8.B45), a portable production system built from System T components in a desktop frame...
03/09/2026
DPA Microphones has announced the 6388 CORE Headset Microphone, making its debut at IBC2026 (Hall 8, Stand D60). The supercardioid headset features a physicall...
03/09/2026
Qvest will present a multi-vendor Dynamic Media Facility (DMF) showcase at IBC2026 (Booth 10.C24), demonstrating how the EBU's DMF architecture can be imple...
03/09/2026
Imagine Communications will showcase the Magellan PanelFlex control surface and other control portfolio updates at IBC2026 (Stand 1.B73, RAI Amsterdam, Septembe...
03/09/2026
Bridge Technologies will announce at IBC2026 (Booth 1.A71) that its VB440 production probe is now available as a software-based solution, deployable on commerci...
03/09/2026
Audio-Technica has unveiled the BP350ST MS (mid-side) microphone in two versions: the camera-mount BP350ST-UL and the boundary or gooseneck-mountable BP350ST-UB...
03/09/2026
Telos Alliance will introduce updates to the J nger Audio flexAI audio processin...
03/09/2026
DUNKMAN, the professional dunk league founded by Shaquille O'Neal in partner...
03/09/2026
Orlando's WRDQ TV 27, WFTV Channel 9 previously named over-the-air broadcast...
03/09/2026
EverPass Media has announced a multi-year commercial distribution agreement with Netflix, expanding on an existing partnership that has delivered NFL Christmas ...
03/09/2026
Yahoo Sports and Apple TV have announced a partnership to deliver alternate broadcasts of select Friday Night Baseball and MLS games on the Yahoo Sports website...
03/09/2026
With another academic year underway, college athletic departments across the cou...
03/09/2026
Facility becomes permanent home of NFL Live on Sept. 9; centerpiece studio for c...
03/09/2026
Create musical synth bass sounds from scratch
Anthony Marinelli Music have recently announced the launch of Bass DNA, a new online course that teaches music...
03/09/2026
Most powerful Tonex hardware to date
IK Multimedia's latest release introduces their most advanced Tonex hardware unit to date, combining their AI Machi...
03/09/2026
Upmix & downmix software upgraded
Penteo Audio Plugins have announced that the latest version of their powerful upmix and downmix software is now available....
03/09/2026
With the start of the new apprenticeship year, young talent has once again begun their professional careers at SGL Carbon's sites in Germany. Recently, the ...
03/09/2026
With the industry increasingly embracing hybrid, software-defined production models, Calrec will preview a series of Argo and ImPulseV software enhancements at ...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Recognizing the ever-increasing demands for media throughput in challenging environments like post-production, two leaders in network technology have worked tog...
03/09/2026
EVS, a leader in live video technology and production infrastructure solutions, and Matrox Video, a leader in software-defined media technology and IP-based vid...
03/09/2026
AJA Debuts OG-FiDO-4 openGear Four-Channel SDI/Fiber Conversion Cards
Brie Clayton September 3, 2026
0 Comments
New product family expands 12G-SDI and...
03/09/2026
Nice Touch launches V2 of its AI edit assistant for professional editors
Brie Clayton September 3, 2026
0 Comments
Built by two editors with twelve ye...
03/09/2026
Accedo, a global provider of video streaming software and services, and MediaKind, a world-leading video technology company and the number one independent provi...
03/09/2026
International research from Big Blue Marble puts a consumer churn number against a technical failure, and sets out what streaming providers must fix before 2030...
03/09/2026
High-performance points of presence power video streaming delivery to large audiences across Latin America with guaranteed quality of experience
Broadpeak , a ...
03/09/2026
Dalet, a leading technology and service provider for media-rich organizations, will unveil the next evolution of Dalia at IBC2026 (7.A43), its media-aware agent...