Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

29/07/2026

Brightline Unveils Compact, Suspended MOD1 LED Light

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

NAB Meets With Trusty on Ownership Caps

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

QuickLink To Feature New StudioPro AnywhereTally At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

HBO Max Launches Vertical Shorts to Improve Content Discovery

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

New Tool Identifies Sources Of Fake Videos

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

DNEG Migrates Editorial Workflows to Foundrys Hiero

Foundry today announced that Academy Award-winning creative studio DNEG has implemented Hiero to support its editorial workflows. Part of the Nuke product famil...

29/07/2026

QuickLink Unveils New StudioPro AnywhereTally Remote Tall...

The Latest Addition to the Brand's StudioPro Ecosystem Combines Simple Deployment, Cloud Control and Professional Grade Wireless Reliability QuickLink, a l...

29/07/2026

The Secret Master Adjustment Layer Hack in After Effects

The Secret Master Adjustment Layer Hack in After Effects Graham Quince July 28, 2026 0 Comments Trish Meyer (who literally beta-tested the original Af...

29/07/2026

Post Completed with Blackmagic Cloud for Shane Kosugi's Feature Film SEEK

Post Completed with Blackmagic Cloud for Shane Kosugi's Feature Film SEEK Brie Clayton July 28, 2026 0 Comments Cannes Indie International Film Fe...

29/07/2026

Isabeau Miller Named Chair of Berklee Songwriting Department

Isabeau Miller Named Chair of Berklee Songwriting Department The Berklee alumna, who previously served on the Songwriting faculty, will begin her new role on ...

28/07/2026

LiveUs World Cup Record: 1,700 Units Support Wide Range of Transmission Needs

LiveU, the pioneer in IP-video transmission and cloud-based remote production, today detailed its historic performance records set during the 2026 FIFA World Cu...

28/07/2026

Case Study: How the Dallas Cowboys Removed Bottlenecks in Off-site Content Production

The NFL franchise rebuilt their off-site content operation for an era when teams...

28/07/2026

A New Way To Watch': How Formula 1 and Sky Sports Are Pushing Race Coverage Into a New Era

Longtime partners expand viewer choice, data-driven analysis, access, immersive ...

28/07/2026

Grass Valley Op Ed: Seven Paradoxes Shaping the Next Era of Media Production

There is a unique moment before every major production. The audience never sees it. The cameras are ready. The talent is preparing. Directors review the final...

28/07/2026

The CW Network, ESPN Ink Streaming Agreement With Launch on Aug. 4

The CW Network and ESPN have announced their highly anticipated streaming agreement to bring CW Sports' live events to the ESPN App will officially launch o...

28/07/2026

Lawo Delivers Audio Excellence for Europes Largest Classical Open-Air Concert on Bastille Day

As more than tens of thousands of spectators filled Paris' Champ de Mars and...

28/07/2026

Appear Assists Chile's First Large-Scale 4K FIFA World Cup Viewing Experience

Appear's X Platform played a central role in Chile's first large-scale c...

28/07/2026

IBC 2026: Solid State Logic Highlights Transition to Software-Defined Production

Responding to the broadcast industry's shift to embrace more flexible production platforms, Solid State Logic (SSL) is evolving its comprehensive System T p...

28/07/2026

TNT Sports Goes to Greater Heights With New DUNKMAN League

Production leverages innovative camera technologies, AI, and data to capture the action in four-week inaugural competition The first-ever professional dunking ...

28/07/2026

Rogers, Prime Video Ink 12-Year Sublicensing Deal for NHL Rights in Canada

Prime Video to exclusively broadcast Wednesday night national NHL regular-season games in Canada...

28/07/2026

CBS Sports Snags U.S. English-Language Rights for CONCACAF Champions Cup

Every match will stream live on Paramount+ beginning in February,...

28/07/2026

TK Audio unveil the TK-Lizer 3

Baxandall-inspired hardware EQ overhauled TK Audio's Baxandall-inspired hardware EQ unit has just made a return, bringing the popular unit back into the...

28/07/2026

Asparion reveal the D700 Rack

D700 system gains new live-focused rackmount surface Asparion have just announced an expansion of their D700 control surface range, introducing a new model ...

28/07/2026

Six NFVF Presidential Employment Stimulus Programme (PESP)-Funded Films Selected for the 47th Durban International Film Festival (DIFF)

27 July 2026, Johannesburg - The National Film and Video Foundation (NFVF), an a...

28/07/2026

Nielsen Launches Ad Intel AI: the Only Product of its Kind that Transforms Fragmented Ad Data into Actionable Intelligence

Ad Intel AI is Nielsen's first in a new generation of AI products built to t...

28/07/2026

aconnic is ramping up supply for the new ACCEED 1444 fiber migration

Contribution to Europe's technological autonomy in the key industry telecommunications aconnic AG (ISIN: DE000A0LBKW6), Munich, starts commercial shipments...

28/07/2026

The Tow Foundation Awards PMVP $400,000 Grant

Share Copy link Facebook X Linkedin Bluesky Email...

28/07/2026

LiveU's World Cup Support Breaks Company Records

Share Copy link Facebook X Linkedin Bluesky Email...

28/07/2026

Encompass Selected by Channel 4 to Deliver Managed Linear...

Encompass Digital Media has been selected by Channel 4, one of the UK's public service broadcasters, to deliver managed linear playout services as part of a...

28/07/2026

Cinegy partners with SAMA Digital Media Solutions to sati...

Cinegy GmbH, the premier provider of software-defined television technology, has confirmed a new partnership with leading Korean systems integrator and distribu...

28/07/2026

Limecraft Adds Multicam Player and Audio Fingerprint-Base...

Limecraft today announces version 2026.5 of its AI-powered production platform. Limecraft 2026.5 https://(www.limecraft.com/limecraft-2026-5-multicam-quad-view-...

28/07/2026

Appear powers first large-scale 4k FIFA World Cup viewing...

X Platform technology enabled a premium UHD viewing experience through a hybrid ground-to-cloud architecture developed with Chilevisi n, ClaroVTR and Pixop App...

28/07/2026

LiveU Delivers Record-Breaking LIQ and Cloud Service Depl...

LiveU, the pioneer in IP-video transmission and cloud-based remote production, today announced it has set historic performance records during the 2026 World Foo...

28/07/2026

DHD Releases New Firmware and Options for its Audio Produ...

dhd.audio announces that its latest generation firmware, Version 10.4, is now available to DHD customers. Version 10.4 introduces expanded configuration capabil...

28/07/2026

Study: TV Operating Systems Gain Influence Over Viewing

Share Copy link Facebook X Linkedin Bluesky Email...

28/07/2026

Maryland Public Television Promotes Ling Ling Sun to CTO

Share Copy link Facebook X Linkedin Bluesky Email...

28/07/2026

NUGEN Audio Helps Mix Technician Blake Bunzel Deliver Rel...

Brand's Intuitive Metering and Dialogue Tools Improve Efficiency and Maintain Clarity Across Projects Ranging from Sitcoms to Effects Heavy Fantasy Series ...

28/07/2026

TAG Video Systems Accelerates the Move to Cloud and Virtu...

TAG Video Systems, a leader in 100% software-based realtime media monitoring, multiviewing and quality control (QC), today announced the latest innovations to i...

28/07/2026

Harvard Athletics Transforms Collegiate Sports Storytelling with AJA Gear

Harvard Athletics Transforms Collegiate Sports Storytelling with AJA Gear Brie Clayton July 27, 2026 0 Comments Between the maturation of IP and remot...

28/07/2026

New Live Action Projection Mapping Film Captured with URSA Cine 12K LF

New Live Action Projection Mapping Film Captured with URSA Cine 12K LF Brie Clayton July 27, 2026 0 Comments Large format sensor allows Texas Legacy ...

28/07/2026

NVIDIA Agent Toolkit Expands With New Omniverse Libraries, Putting AI Agents to Work Building Simulation-Ready Worlds

NVIDIA Agent Toolkit Expands With New Omniverse Libraries, Putting AI Agents to ...

28/07/2026

AI Agents Expand Creative Tools to Millions

AI Agents Expand Creative Tools to Millions Brie Clayton July 27, 2026 0 Comments Hero image courtesy of Epic Games Leading creative applications are...

28/07/2026

How Rick Aggeler Built a Music Community for Boston Teens

How Rick Aggeler Built a Music Community for Boston Teens When he was a Berklee student, Aggeler volunteered at the Boys & Girls Clubs of Boston, where he hel...

28/07/2026

Football is back on Sky Sports following its most watched Premier League season ever

Tuesday 28 July 2026 Football is back on Sky Sports following its most watched ...

28/07/2026

Powerful Compute So Compact, It's Clutch - Build AI in Your Hand With NVIDIA Jetson

As a discerning AI investor who values style and substance, Sarah Guo knows this...

28/07/2026

Are you coming to the Fleadh? RT will be there

RT Raidi na Gaeltachta, RT Radio 1, RT One and RT Player to showcase the best of the reels, rhymes and rambles from Comhaltas Fleadh Cheoil na h ireann 202...