
Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.
Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.
Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.
NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.
Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.
In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.
A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.
DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.
On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.
The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.
In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.
It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.
A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.
The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.
And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.
These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.
The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
27/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/08/2026
Profuz Digital makes a mark by launching a powerful upgrade to its Profuz LAPIS digital media management platform this IBC in Amsterdam at Stand 8.A88. The late...
27/08/2026
DHD.audio has chosen IBC 2026 as the launch venue for what it defines as one of the most radical advances in the companys 30 year history.
Every broadcast trad...
27/08/2026
Direct connectivity between planning, production and graphics for creativity and streamlined workflows...
27/08/2026
New Photoshop innovations bring you more choice and control at every stage of yo...
27/08/2026
First Two Seasons of Vertical Series Vultures Shot on Blackmagic Design Cameras
Brie Clayton August 27, 2026
0 Comments
Director and DoP of Vultures ...
27/08/2026
For years, the media and entertainment industry has hesitated to use AI-voiced dubbing due to concern for audience experience. New data from 3Play Media, a glob...
27/08/2026
Starfish Technologies has announced the forthcoming V9 release of its Advantage AD workstation, adding support for voice synthesis services and giving Audio Des...
27/08/2026
Lightware's UBEX is a powerful AV-over-IP product well suited for rental and staging workflows where systems must be fast to deploy, easy to repurpose, and ...
27/08/2026
Lightware has expanded the scalability of its 10G TPN AV-over-IP platform with the introduction of the new TPN-MMU-X100 Matrix Management Unit, enabling systems...
27/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/08/2026
PlayBox Neo Stand 7.C29 IBC2026 - 11 14 September
Well-established global specialists in media playout, automation, and streaming solutions, PlayBox Neo ...
27/08/2026
Rise Academy, the broadcast and media skills charity dedicated to delivering practical media technology learning experiences, careers resources and work experie...
27/08/2026
Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, will showcase the latest evolution of its ...
27/08/2026
Screen Australia opens applications for Global Fellowships Program 27 August 2026
Applications are now open for the Screen Australia Global Fellowships progra...
27/08/2026
Commissioned for RT KIDS as part of the EBU Kids' Drama Exchange The Note will premiere Sunday August 30, 5:10pm
RT One and RT Player
The Note tells a...
27/08/2026
August 27 2026, 06:00 (PDT) Dolby Laboratories Announces Leadership Transition
Kevin Yeaman to Retire from Dolby
Marc Whitten Named President, Chief Exe...
27/08/2026
NVIDIA's Gamescom announcements are revealing what's next for GeForce NOW, with new ways to play, more supported devices and platforms, and even more bi...
26/08/2026
Applicants interested in the Media Innovation Europe 3.0 Validation Booster call can now consult a new FAQ document addressing practical questions raised during...
26/08/2026
Roku Sports Channel will stream eight matches free beginning Aug. 30 as league shifts Sunday-night games away from Victory+...
26/08/2026
Evertz and Mobile TV Group detail how on-premises infrastructure and cloud resou...
26/08/2026
The Sennheiser Group will demonstrate its broadcast audio portfolio at IBC2026 (Hall 8, Booth D50), covering wireless microphone workflows, studio monitoring, a...
26/08/2026
Appear and Transmisiones Televisivas TVTEL Limitada have announced the deploymen...
26/08/2026
Leader Electronics has announced that Danish broadcast rental company Minitech has purchased a Leader LPX500 hybrid IP/SDI waveform monitor and LT4670 sync puls...
26/08/2026
Encompass will demonstrate its Altitude media services at IBC2026 (IAMT Hub, Diamond Lounge, Hall 3, RAI Amsterdam, September 11-14).
Key products on show incl...
26/08/2026
Daktronics has partnered with Massillon Washington High School in Ohio to install a new LED video display and implement the Daktronics Camino show control platf...
26/08/2026
Clear-Com has appointed Dubai-based Venuetech LLC as its main partner for the Middle East. Venuetech distributes professional audiovisual solutions across the r...
26/08/2026
NBC Sports NOW, NBC Sports' free 24/7 streaming channel, will launch a new weekday lineup beginning Monday, August 31, including two new shows.
Not So Fast...
26/08/2026
Calrec will demonstrate updates to its ImPulseV DSP software and Argo ecosystem at IBC2026 (Stand 8.C47), including previews aligned with the EBU's Dynamic ...
26/08/2026
Imagine Communications will demonstrate AI applications across its ad tech and video infrastructure portfolios at IBC2026 (Stand 1.B73, RAI Amsterdam, September...
26/08/2026
Dalet will demonstrate updates to Dalet Flex, Dalet Pyramid, and Dalia, its agentic AI platform, at IBC2026 (Hall 7, Stand 7.A43). The company will also partici...
26/08/2026
Team-owned platform will stream every locally carried Predators game beginning t...
26/08/2026
The 12-day tournament receives the broadcaster's full onsite-studio presence...
26/08/2026
A synth with no controls?
If you're into weird and wonderful instruments, then you may already be familiar with Warsaw-based SOMA Laboratory - if you...
26/08/2026
Physical modelling orchestral instrument announced
Following their recent cryptic hints, Dreamtonics have officially revealed the latest addition to their s...
26/08/2026
Discussions at the Meitingen site on competitiveness, energy supply, and fuel cell technology
Bertram Brossardt, CEO of the Bavarian metal and electrical emplo...
26/08/2026
Rohde & Schwarz USA celebrates grand opening of new manufacturing center in Fred...
26/08/2026
For large media companies, the content archive is a gold mine. There are thousands of hours of existing, high-value assets just waiting to be tapped for highlig...
26/08/2026
Ad Supported Viewing Represents 71.5% of TV in Q2
Streaming Captures Over 48% o...
26/08/2026
Charity sector ad spend reached $278 million as 44% of Australians donated to ch...
26/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
26/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
26/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...