Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

29/07/2026

Grass Valley Op-Ed Series: Seven Paradoxes Shaping the Next Era of Media Production

In this series, Grass Valley explores the seven paradoxes shaping the next era o...

29/07/2026

SVG Students to Watch: A Look Back at a Year of Spotlighting the Emerging Talent in Live Sports Production

Thats a wrap on Season 1 of SVG Students to Watch! Across the country, these stu...

29/07/2026

DAZN to Become Exclusive DTC In-Market Streaming Home of YES Network, MSG Networks; Gotham App to Shut Down

The agreement will move Gotham Sports App subscribers to DAZN during the 2026-27...

29/07/2026

Big South Promotes Emily Dassel to Associate Commissioner for Brand and Creative Strategy

Longtime digital-media leader expands role overseeing conference branding, video...

29/07/2026

Give Me the Backstory: Get to Know Gregg Araki, the Director of I Want Your Sex

By Bailey Pennick One of the most exciting things about the Sundance Film Festival is having a front-row seat for the bright future of independent filmmaking. ...

29/07/2026

TiME UK announce transatlantic partnership

Organisation team up with TI:ME USA Founded in 2019 with the aim of helping educators to embrace new music technology, Technology in Music Education (TiME U...

29/07/2026

Multiply 2: Free plug-in from Acon Digital

Doubling & chorus processor gets an overhaul Acon Digital have just launched the latest version of their free voice-doubling and chorus plug-in. Multiply 2 ...

29/07/2026

Origin Effects launch the EQDeluxe

New Pultec-inspired pedal revealed The latest addition to the Origin Effects line-up brings the legendary Pultec EQ into pedal format. Based on the EQP-1A, ...

29/07/2026

SBS Appoints Chief Commercial Officer and Director of Marketing and Audiences

SBS Appoints Chief Commercial Officer and Director of Marketing and Audiences 29 July, 2026 Media releases SBS is pleased to announce the appointment of Ni...

29/07/2026

Rohde & Schwarz showcases unified next-generation broadcasting ecosystem at SET Expo 2026 in Brazil

Rohde & Schwarz showcases unified next-generation broadcasting ecosystem at SET ...

29/07/2026

Central Christian Church Connects Remote Production Teams with Clear-Com's Gen-IC...

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...

29/07/2026

Streaming Embarks on Annual Summer Ascent in Nielsen's May 2026 Gauge Reports

Streaming Hits 48.6% of Total TV Usage in May YouTube Leads in Media Distributo...

29/07/2026

WVIT's Joseph DiMaggio Named SBE Flanders Engineer of the Year

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

Amagi Appoints Martin Wacker to Lead Sales for DACH -- St...

Industry veteran joins Amagi to drive growth in Europe's leading broadcast market and advance cloud-native broadcast, FAST, and AI-driven operations. LONDO...

29/07/2026

WebTransport opens new route for broadcast-quality media...

Work is ongoing to specify how to run RIST over WebTransport. This will deliver secure, stable, broadcast quality video playback and origination to any platform...

29/07/2026

Offline Files Makes the Drive Shelf Searchable on Mac

Offline Files Makes the Drive Shelf Searchable on Mac Brie Clayton July 29, 2026 0 Comments A local Mac app for finding files, photos, and footage acr...

29/07/2026

Blackmagic Design Announces New UltraStudio Mini 12G

Blackmagic Design Announces New UltraStudio Mini 12G Brie Clayton July 29, 2026 0 Comments New extremely portable Thunderbolt 4 capture and playback ...

29/07/2026

Brightline Lighting Introduces the MOD1

Brightline Lighting, a premier manufacturer of energy-efficient lighting systems for broadcast, videoconferencing, government, education and corporate applicati...

29/07/2026

Applications open for IFFI/WAVES Film Bazaar in Goa - Producer Delegation

Applications open for IFFI/WAVES Film Bazaar in Goa - Producer Delegation 28 July 2026 Image credit: Four Years Later Screen Australia has opened applications...

29/07/2026

Brightline Unveils Compact, Suspended MOD1 LED Light

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

NAB Meets With Trusty on Ownership Caps

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

QuickLink To Feature New StudioPro AnywhereTally At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

HBO Max Launches Vertical Shorts to Improve Content Discovery

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

New Tool Identifies Sources Of Fake Videos

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

DNEG Migrates Editorial Workflows to Foundrys Hiero

Foundry today announced that Academy Award-winning creative studio DNEG has implemented Hiero to support its editorial workflows. Part of the Nuke product famil...

29/07/2026

QuickLink Unveils New StudioPro AnywhereTally Remote Tall...

The Latest Addition to the Brand's StudioPro Ecosystem Combines Simple Deployment, Cloud Control and Professional Grade Wireless Reliability QuickLink, a l...

29/07/2026

The Secret Master Adjustment Layer Hack in After Effects

The Secret Master Adjustment Layer Hack in After Effects Graham Quince July 28, 2026 0 Comments Trish Meyer (who literally beta-tested the original Af...

29/07/2026

Post Completed with Blackmagic Cloud for Shane Kosugi's Feature Film SEEK

Post Completed with Blackmagic Cloud for Shane Kosugi's Feature Film SEEK Brie Clayton July 28, 2026 0 Comments Cannes Indie International Film Fe...

29/07/2026

Isabeau Miller Named Chair of Berklee Songwriting Department

Isabeau Miller Named Chair of Berklee Songwriting Department The Berklee alumna, who previously served on the Songwriting faculty, will begin her new role on ...

29/07/2026

Hayu is coming to Sky, bringing fan-favourite franchises The Real Housewives, Below Deck and more to customers, at no extra cost

Wednesday 29 July 2026 Hayu is coming to Sky, bringing fan-favourite franchises...

29/07/2026

New RT true crime podcast launches today

RT launches Unsolved, a true crime podcast, presented by crime journalist Barry Cummins and presenter Katja Mia. Unsolved is a weekly podcast exploring unsol...

28/07/2026

LiveUs World Cup Record: 1,700 Units Support Wide Range of Transmission Needs

LiveU, the pioneer in IP-video transmission and cloud-based remote production, today detailed its historic performance records set during the 2026 FIFA World Cu...

28/07/2026

Case Study: How the Dallas Cowboys Removed Bottlenecks in Off-site Content Production

The NFL franchise rebuilt their off-site content operation for an era when teams...

28/07/2026

A New Way To Watch': How Formula 1 and Sky Sports Are Pushing Race Coverage Into a New Era

Longtime partners expand viewer choice, data-driven analysis, access, immersive ...

28/07/2026

Grass Valley Op Ed: Seven Paradoxes Shaping the Next Era of Media Production

There is a unique moment before every major production. The audience never sees it. The cameras are ready. The talent is preparing. Directors review the final...

28/07/2026

The CW Network, ESPN Ink Streaming Agreement With Launch on Aug. 4

The CW Network and ESPN have announced their highly anticipated streaming agreement to bring CW Sports' live events to the ESPN App will officially launch o...

28/07/2026

Lawo Delivers Audio Excellence for Europes Largest Classical Open-Air Concert on Bastille Day

As more than tens of thousands of spectators filled Paris' Champ de Mars and...

28/07/2026

Appear Assists Chile's First Large-Scale 4K FIFA World Cup Viewing Experience

Appear's X Platform played a central role in Chile's first large-scale c...

28/07/2026

IBC 2026: Solid State Logic Highlights Transition to Software-Defined Production

Responding to the broadcast industry's shift to embrace more flexible production platforms, Solid State Logic (SSL) is evolving its comprehensive System T p...

28/07/2026

TNT Sports Goes to Greater Heights With New DUNKMAN League

Production leverages innovative camera technologies, AI, and data to capture the action in four-week inaugural competition The first-ever professional dunking ...

28/07/2026

Rogers, Prime Video Ink 12-Year Sublicensing Deal for NHL Rights in Canada

Prime Video to exclusively broadcast Wednesday night national NHL regular-season games in Canada...

28/07/2026

CBS Sports Snags U.S. English-Language Rights for CONCACAF Champions Cup

Every match will stream live on Paramount+ beginning in February,...

28/07/2026

TK Audio unveil the TK-Lizer 3

Baxandall-inspired hardware EQ overhauled TK Audio's Baxandall-inspired hardware EQ unit has just made a return, bringing the popular unit back into the...

28/07/2026

Asparion reveal the D700 Rack

D700 system gains new live-focused rackmount surface Asparion have just announced an expansion of their D700 control surface range, introducing a new model ...

28/07/2026

Six NFVF Presidential Employment Stimulus Programme (PESP)-Funded Films Selected for the 47th Durban International Film Festival (DIFF)

27 July 2026, Johannesburg - The National Film and Video Foundation (NFVF), an a...

28/07/2026

Nielsen Launches Ad Intel AI: the Only Product of its Kind that Transforms Fragmented Ad Data into Actionable Intelligence

Ad Intel AI is Nielsen's first in a new generation of AI products built to t...