Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

09/10/2026

Dolby Expands Dolby OptiView Platform with New Capabilities at IBC 2026

September 10 2026, 06:00 (PDT) Dolby Expands Dolby OptiView Platform with New Capabilities at IBC 2026 New Sports Intelligence helps providers better unders...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

18/09/2026

The inside scoop on attention-grabbing market assets from those in the know

The inside scoop on attention-grabbing market assets from those in the know 17 September 2026 (L-R): Sophie Green, Todd Brown and Edwina Waddy Find this episo...

17/09/2026

Prime Video Embarks on Fifth Exclusive Season of Thursday Night Football With Innovation, Experience in Tow

Lions vs. Bills will also be the first regular-season game at new Highmark Stadi...

17/09/2026

Teradek Launches Bolt 7 with Integrated 900MHz Data Pipeline

Teradek has announced Bolt 7, a wireless video and data system that adds an integrated 900MHz radio to the Bolt's zero-delay video pipeline. The 900MHz chan...

17/09/2026

DAZN Announces Three Executive Appointments

DAZN has announced three appointments to its executive leadership team, all reporting to CEO Shay Segev effective immediately. Patrick Delany has been appointe...

17/09/2026

Monumental Sports and Entertainment Announces Leadership Changes

Monumental Sports and Entertainment (MSE) has announced a leadership restructuring. Zach Leonsis has been named President and Vice Chairman, a newly created pos...

17/09/2026

CP Communications and Red House Streaming Named 2026 Best of Florida Winners

CP Communications has been named Best of Florida for Audio-Visual Services, and its subsidiary Red House Streaming has been named Best Movie and Recording Studi...

17/09/2026

Scripps Sports and ION to Broadcast Shriners Childrens East-West Bowl

Scripps Sports and ION have announced a broadcast partnership for the Shriners Children's East-West Bowl, the nation's oldest college all-star football ...

17/09/2026

espnW Announces Full Agenda for 2026 Women + Sports Summit

The 17th annual espnW: Women + Sports Summit, presented by Toyota, will take place October 14-16 at the Ojai Valley Inn in Ojai, California, with virtual attend...

17/09/2026

PAMA and Shure Announce 2026 Mark Brunner Professional Audio Scholarship Recipients

The Professional Audio Manufacturers Alliance (PAMA) and Shure Incorporated have...

17/09/2026

LNK Group Migrates Five Networks to Grass Valley AMPP

Lithuanian broadcaster LNK Group has completed a migration of playout and media management for its five national television networks - LNK, BTV, TV1, InfoTV, an...

17/09/2026

Net Insight Establishes Pan-Asian Live Media Network for Nimbra 520 Trials

Net Insight has announced a pan-Asian live media network created with its regional partners, giving broadcasters, production companies, and media service provid...

17/09/2026

SVG Students to Watch: Adam Black, University of South Florida

After a high school football injury, the Orlando native found a new path in video production, gaining experience on ESPN college football broadcasts, Tampa Bay ...

17/09/2026

Image-Line & Antares reveal AutoTune Go

Popular DAW gains free pitch-correction tool Image-Line have teamed up with Antares to kit their popular DAW software out with a built-in pitch-correction p...

17/09/2026

Gibson & Baggs unveil HiFi DNA pickup

Promises unprecedented realism, dynamic nuance and clarity Gibson have teamed up with acoustic pickup experts Baggs, creating a next-generation' pickup...

17/09/2026

VSL introduce Synchron Cimbalom & World Percussion II

Two new libraries join percussion line-up VSL (Vienna Symphonic Library) have just launched two new libraries that bring some intriguing new instruments int...

17/09/2026

Arturia update the MiniFreak

Hardware & plug-in versions gain new features Arturia have just released a free update that brings some significant new features to their MiniFreak synthesi...

17/09/2026

Respected journalist Catalina Flrez to succeed Anton Enus as SBS World News presenter

Respected journalist Catalina Fl rez to succeed Anton Enus as SBS World News pre...

17/09/2026

Call for Applications: South African Producers to Attend the Berlinale Co-Production Market Visitors Programme 2027

The Department of Sport, Arts and Culture (DSAC), in partnership with the Nation...

17/09/2026

5 Reasons to Join Calrec at IBC

Calrec brings leading audio solutions and long-term business value to IBC 2026 We're looking forward to meeting up with you all on 11-14th September in Amst...

17/09/2026

Nielsen Accelerates Streaming Data Delivery with Enhancements to Streaming Content Ratings

Streaming Content Ratings (SCR) Data Shifts to Daily Delivery Cadence, Mirroring...

17/09/2026

The Gauge: Poland | August 2026

August brought further stabilization in total TV viewing time. Poles spent an average of 3 hours and 31 minutes a day watching video content on TV glass-just 2 ...

17/09/2026

Warner Bros. Discovery reclaims top spot in Nielsen's August All Screens Video Landscape Report

Post-sports seasonal shift redistributes viewership across Poland, while streami...

17/09/2026

TVC and Globecast Join Hands to Unveil The Matrix - A Ne...

Television Content Analytics Pte Ltd (TVC), a Singapore-based deep-tech company specialising in sports technology, artificial intelligence and live broadcast in...

17/09/2026

Rise Broadcast Reveals 2026 Awards Shortlist

Rise WIB, the global advocacy group championing gender diversity and career progression across the media technology industry, today announced the shortlist for ...

17/09/2026

CAPITOL BROADCASTING COMPANY AND THE CAROLINA HURRICANES EXTEND CONTENT PARTNERSHIP

RALEIGH, N.C. - Capitol Broadcasting Company (CBC) and Hurricanes Holdings annou...

17/09/2026

LNK Group Moves Five National Networks to Grass Valley AM...

Lithuanian broadcaster deploys Playout X and Framelight X across five networks, combining on-premises software-defined playout with cloud-based disaster recover...

17/09/2026

Introducing Adobe Photoshop Elements & Premiere Elements 2027

Introducing Adobe Photoshop Elements & Premiere Elements 2027 Brie Clayton September 17, 2026 0 Comments New tools make it easier than ever to enhance y...

17/09/2026

Scripps Research professor Li Ye named finalist for Blavatnik National Awards for Young Scientists

LA JOLLA, CA-While the brain orchestrates metabolic processes in the body, it al...

17/09/2026

New series of Aistear an Amhrin shares extraordinary stories behind Ireland's best-loved songs

New series of Aistear an Amhr in shares extraordinary stories behind Ireland'...

17/09/2026

The Late Late Show is back! Guests include Liam Neeson, Siobhn McSweeney and the All-Ireland-winning Mayo team

The Late Late Show is back! Liam Neeson, Siobh n McSweeney, Caitr ona Balfe a...

17/09/2026

The Neighbours Effect: Gen Z Brings Back Big Hair, Double Denim and 80s soap style

The Neighbours Effect: Gen Z Brings Back Big Hair, Double Denim and 80s soap sty...

17/09/2026

Official trailer released for Sky Original WAR

The bold, new legal drama starring Dominic West and Sienna Miller launches on Sky and NOW on 2nd OctThursday 17 September 2026 Official trailer released for Sk...

17/09/2026

AI safety needs more than rules. It needs evidence

Every week we read headlines about new threats from AI. From taking over jobs to disrupting critical infrastructure, spreading misinformation or even destroying...

17/09/2026

Arqiva selected by SANZAAR to deliver global distribution of 2026 international rugby programme

Arqiva selected by SANZAAR to deliver global distribution of 2026 international ...

17/09/2026

Widow\u2019s Bay triumphs as most decorated freshman comedy in Emmy history - PRESS RELEASE - Posted on 15 September 2026

CULVER CITY, CALIFORNIA This evening at the 78th Primetime Emmy Awards, Apple TV...

17/09/2026

Cute Critters Come to the Cloud: Aniimo' Launches on GeForce NOW

A new creature-catching adventure is ready to stream from the cloud this week. Pawprint Studio's Aniimo arrives on GeForce NOW at launch, inviting gamers to...

17/09/2026

RT Statement: 2027 Eurovision Song Contest

RT Statement: RT is today confirming that its position regarding participation in the Eurovision Song Contest remains unchanged. RT will not participate in...

16/09/2026

Grass Valley to be Awarded 2026 Technology & Engineering Emmy for Web-Based Frame-Accurate Remote Monitoring

Grass Valley has been recognized with a 2026 Technology & Engineering Emmy Award...

16/09/2026

SVG Cloud & Content Workflows Summit 2026: All Sessions Now Available To Watch on SVG PLAY

Sports-media leaders explore how cloud, AI, and virtualization are transforming ...

16/09/2026

A Summer Catch: How the Women's Pro Baseball League Built a Broadcast Plan From ScratchIn Just Six Months

With a full crew on site in Springfield, deep access to players, and a growing d...

16/09/2026

The History of Concrete Contains More Surprises Than You Can Possibly Imagine

John Wilson attends The History of Concrete premiere during the 2026 Sundance Film Festival at The Yarrow Theatre on January 22, 2026, in Park City, Utah. (Ph...

16/09/2026

Wavea release Flite V2

Powerful soft synth receives free update Wavea have just launched a new and improved version of their feature-packed soft synth, kitting it out with over 20...

16/09/2026

Modartt launch Airteq

Physical-modelling experts launch pipe-based instrument Modartt are well known for their impressive Pianoteq and Organteq offerings, and now, they've pu...

16/09/2026

FL Studio & Antares launch AutoTune Go

Popular DAW gains free pitch-correction tool Image-Line have teamed up with Antares to kit their popular DAW software out with a built-in pitch-correction p...

16/09/2026

Microwave Factory and Rohde & Schwarz enable automated SAR measurements with CMX500 support in MATEOS

Microwave Factory and Rohde & Schwarz enable automated SAR measurements with CMX...

16/09/2026

Rohde & Schwarz launches new security scanner for large events at Security Expo 2026

Rohde & Schwarz launches new security scanner for large events at Security Expo ...