Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

MLB's Jason Hedgcock on Using the Cloud To Produce World Baseball Classic Games for Netflix Japan

Live production in the cloud is a growing trend across the industry, and Major L...

04/08/2026

Major League Soccer Names Larry Berg as Next Commissioner; Will Take Helm in 2027

LAFC co-managing owner will succeed Don Garber on Jan. 1, 2027, as league prepar...

04/08/2026

Foresight: Driving Growth Through Broadcast Flexibility - A Calrec + PGA TOUR Case Study

In 2022, when the PGA TOUR was breaking ground on its new, state-of-the-art broa...

04/08/2026

Dealer Cam Debuts as Omaha Productions Takes World Series of Poker Main Event Final Table Live on ESPN

Provided by POVORA, the new camera angle offers a dealer's-eye view of the c...

04/08/2026

USL To Trial FIFA's Football Video Support System During 2026 USL Cup

Spiideo will provide the camera setup, with NEP handling broadcast integration and off-site video operations...

04/08/2026

Game On Live Studio Deploys Blackmagic Design Workflow Across 15 Fields at Tropical 7s

Production team streamed more than 750 youth rugby matches using ATEM switchers,...

04/08/2026

NDI, KTC Partner To Develop Network-Native Commercial Displays

Agreement gives KTC early access to NDI's Advanced SDK for integration across commercial displays, digital signage, and interactive panels...

04/08/2026

IBC2026: IBC Innovation Awards Finalists Highlight Advances in AI, Cloud, Open Standards, and Accessibility

Twelve projects across four categories recognize collaborative work in productio...

04/08/2026

Ekstraklasa, Live Park Select Hawk-Eye for Tracking, Offside, and Broadcast Technology

Multi-year agreement will introduce SkeleTRACK, semi-automated offside, automate...

04/08/2026

Radio France Deploys Redundant Lawo AoIP Workflow for 2026 Concert de Paris

Three mc 56 consoles, dual A__UHD Core engines, RAVENNA networking, and HOME management supported live sound, monitoring, and international broadcast production...

04/08/2026

IBC2026: Haivision to Debut Makito ONE, Falkon X4 in Europe

New contribution products support ultra-low-latency video transport across fixed, remote, and mobile production environments...

04/08/2026

IBC2026: Chyron to Highlight AI, Cloud, HTML Graphics, and Sports-Analysis Tools

Company will showcase updates across PRIME, Chyron LIVE, Paint, virtual production, newsroom workflows, and venue control...

04/08/2026

TAB Show 2026: MultiDyne to Showcase Fiber-Transport and IP Infrastructure Tools

Exhibit will feature SilverBack, VersaBrix, VersaFrame, VF-9100, and FiberSaver-10G platforms...

04/08/2026

IBC2026: TAG Video Systems Adds MXL Support, Containerized Deployment, Expanded QC Tools

Platform updates target cloud-based production, high-density ST 2110 monitoring,...

04/08/2026

Gravity Media, Live Media Group Partner To Expand U.S. Mobile Production Facility Operations

LMG will take control of Gravity Media's U.S. mobile-unit fleet as the compa...

04/08/2026

HeadRush update Prime, Core & Flex Core firmware

Amp-modelling rigs get major upgrade HeadRush have recently released a major firmware update that brings some powerful new features to their Prime, Core and...

04/08/2026

Am I Audio launch I Am Choir

Generate a choir from a single voice track Following years of development, Am I Audio have just officially launched their debut plug-in. Created in collabor...

04/08/2026

Universal Audio's EMT 140 & 250 plug-ins go native

Official EMT-endorsed reverbs now available to all Two more of Universal Audio's renowned UAD plug-ins have just gained new native versions that can be ...

04/08/2026

SBS helps Australians understand why every Census counts

SBS helps Australians understand why every Census counts Media releases As Australia prepares for the 2026 Census on Tuesday 11 August, SBS is helping ensur...

04/08/2026

Canada invests in the future of HF communications with Rohde & Schwarz solutions

Canada invests in the future of HF communications with Rohde & Schwarz solutions After selection by the Canadian Department of National Defence (DND) as the p...

04/08/2026

Blendkit Expands Beyond Blender

Blendkit Expands Beyond Blender Brie Clayton August 4, 2026 0 Comments Popular 3D Asset Library Launches Alpha Plugins for Maya, Godot, and Rhino Blend...

04/08/2026

QC: Artificial Intelligence's Big Hammer Finds Another Nail

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

The AI Dilemma: Can C2PA Keep a Video's Provenance Intact?

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

BPS Gaining Traction in Broadcast and Other Sectors

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

Ikegami to Demonstrate Latest-Generation UNICAM Product R...

Ikegami has chosen the 11-14 September 2026 International Broadcasting Convention in Amsterdam as the European market launch venue for additions to its UNICAM p...

04/08/2026

Appear unifies hardware and software for live production...

Appear (IBC2026 booth 1.C61) will showcase new X & VX Platform capabilities that extend broadcast-grade performance from venue to cloud Appear ASA (OSE: APR), ...

04/08/2026

Broadcast Management Group Brings Amazon Prime Obsessed F...

LiveU, the global leader in live IP-video solutions, today highlighted the crucial role its EcoSystem played in Broadcast Management Group's (BMG) successfu...

04/08/2026

Scripps Completes Acquisition of WTVQ Lexington

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

U.S., U.K. Consumers Now Spend 11 Hours a Day Using Media

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

Sinclair Announces 2026 Annual Scholarship Winners

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

Tom DeLay: Only Congress Can Change Station Ownership Caps

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

IAB Tech Lab Upgrades Open Framework for Agentic Advertising

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

DirecTV For Business Inks New Agreement to Sell NFL Sunday Ticket

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

IEEE Broadcast Technology Society Honors Charles Einolf

Share Copy link Facebook X Linkedin Bluesky Email...

04/08/2026

Homecoming Artist Spotlight: Dreion

Homecoming Artist Spotlight: Dreion The singer, songwriter, and producer talks about finding his village at Berklee ahead of his Homecoming performance on Sep...

04/08/2026

Sky Sports agrees deal with SANZAAR for international rugby rights 2026 2030

Tuesday 4 August 2026 Sky Sports agrees deal with SANZAAR for international rugby rights 2026 - 2030 The deal means Sky will have exclusive rights to the inau...

04/08/2026

Kill Tony Launches on Supercast, Giving Fans Premium & Ad-Free Access to Comedy's Biggest Live Podcast

Kill Tony Launches on Supercast, Giving Fans Premium & Ad-Free Access to Comedy&...

04/08/2026

BIRTV. Beijing. August 19-22, 2026

More details coming soon....

04/08/2026

AMC Week. Mexico City. September 2-5, 2026

More details coming soon....

04/08/2026

IBC. Amsterdam. September 11-14, 2026

More details coming soon....

04/08/2026

As AI Increases Demands on Memory, Storage Steps Up

Surging AI demands are driving the need for massive datasets and context windows that burst past the confines of system memory. But rising needs aren't me...

04/08/2026

NVIDIA Alpamayo 2 Super, the Frontier Open Model for Robotaxis and Autonomous Vehicles, Now Available for Commercial Use

For robotaxis and other autonomous vehicles (AVs), the hardest problems aren'...

04/08/2026

AI Leaders Propose SAFE Guidelines for Cybersecurity Transparency

Members of the Open Secure AI Alliance - now more than 120 organizations strong - are developing new guidelines to strengthen agentic AI cybersecurity as the an...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

03/08/2026

SVGs TranSPORT Conference Returns to NYC on Oct. 20; Register Today!

SVG's annual TranSPORT conference returns to New York City for a full day of sessions focused on the current state of live-sports-transmission technology. S...

03/08/2026

SVG All-Stars: Stefan Richardson, Director, Global Broadcast, Riot Games

A five-time Emmy winner, this esports vet has built a production career on live storytelling, audience connection, and meeting viewers wherever they are The sp...

03/08/2026

MLBs Jason Hedgcock on Using the Cloud To Produce World Baseball Classic Games for Netflix Japan

Live production in the cloud is a growing trend across the industry and Major Le...