Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

14/05/2026

Is This the Year for Agentic AI's Breakout in Broadcast?

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

CBS LA, Los Angeles Rams Ink New TV Deal

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

CueScripts CueiT 4 0 Wins Futures Best of Show Award Pres...

CueScript's CueiT 4.0 Wins Future's Best of Show Award, Presented at 2026 NAB Show by TV Tech CueScript, a leading international developer of professio...

14/05/2026

Expert-Led Education Sessions and Development of Online T...

Expert-Led Education Sessions and Development of Online Training Program Accelerate IPMX Adoption and Deployment The Alliance for IP Media Solutions (AIMS) to...

14/05/2026

Klvr rechargeable battery launches in the USA to cut cost...

Klvr is launching in the United States with a professional-grade rechargeable battery solution that cuts costs and improves performance across live entertainmen...

14/05/2026

Shooting into the depths of Bedlam with URSA Cine 17K 65

Shooting into the depths of Bedlam with URSA Cine 17K 65 Brie Clayton May 14, 2026 0 Comments Indie feature film paired digital 65mm capture with a Bl...

14/05/2026

WeMakeColor expands with Baselight, becoming hybrid color facility

WeMakeColor expands with Baselight, becoming hybrid color facility Caroline Shawley May 14, 2026 0 Comments Boutique Mexican-based studio integrates B...

14/05/2026

Berklee's Summer in the City Returns with Free Concerts Throughout Boston Area

Berklee's Summer in the City Returns with Free Concerts Throughout Boston Ar...

14/05/2026

Chelsey Green Named to Billboard's 2026 Women in Music List

Chelsey Green Named to Billboard's 2026 Women in Music List The Berklee professor and chair of the Recording Academy Board of Trustees joins other high-pr...

14/05/2026

Parks: Tubi, Roku Channel Are Top U.S. FAST Platforms

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

Study: Downstream Fiber Usage Outpaces Cable Broadband

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

NABLF to Honor Kidde With Corporate Leadership Award

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

LABF Awards Four 2026 Preservation Grants

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

Netflix Expands NFL Deal to Five Games

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

Scripps Seals Local Broadcast Deal with Detroit Pistons

Share Copy link Facebook X Linkedin Bluesky Email...

14/05/2026

Glensound marks 60 years of audio innovation at Broadcast...

Six decades of products built around the people who use them...

14/05/2026

Vivid Broadcast builds agile remote production network ar...

Designed to embrace multiple production processes and deliver high-end live broadcast workflows, Vivid Broadcast has combined Calrec True Control 2.0 enabled co...

14/05/2026

Jigsaw24 and EVS Partner to Strengthen Future Proof UK Li...

Today, Jigsaw24, the UK's leading media equipment supplier and systems integrator, announces a new partnership with EVS, a global leader in live video techn...

14/05/2026

Transforming bold ideas into market-ready productions: Digital Originals returns

Transforming bold ideas into market-ready productions: Digital Originals returns 14 May 2026 Top (L-R): Bronte Gosper (Musket), Mema Munro (Rogue One), Nathan ...

14/05/2026

Berklee Convenes Leaders in AI, Music for Inaugural AIMS Symposium

Berklee Convenes Leaders in AI, Music for Inaugural AIMS Symposium The three-day event puts musicians at the center of the future of music creation, ethics, a...

14/05/2026

May 13, 2026

Scripps Research establishes endowed chair honoring renowned structural biologist Ian Wilson Keren Lasker to be inaugural chair holder May 13, 2026 LA JOLLA ...

14/05/2026

Discover hidden havens for city wildlife as Back From The Brink returns for a fifth season

Back From The Brink airs Sunday 17 May and Sunday 24 May at 6.30pm on RT One an...

14/05/2026

Sea You in the Cloud: Subnautica 2' Early Access Dives Onto GeForce NOW

Dive masks on - Subnautica 2 is making a splash on GeForce NOW day-and-date with launch, so members can plunge into the title's brand-new alien ocean from a...

13/05/2026

New Adobe Premiere Color Grading Mode Accelerated on NVIDIA GPUs

New Adobe Premiere Color Grading Mode Accelerated on NVIDIA GPUs Joel Pennington May 13, 2026 0 Comments New NVIDIA RTX-accelerated features streamlin...

13/05/2026

dB Broadcast Delivers New IP-based Cloudbass Sports OB Tr...

Grass Valley announced that dB Broadcast has delivered new IP-based outside broadcast (OB) trucks for Cloudbass, featuring Grass Valley LDX 100 Series cameras a...

13/05/2026

Ikegami Announces its Broadcast Asia 2026 Innovations

Ikegami will exhibit the latest additions to its wide range of broadcast production cameras, control units, viewfinders and monitors on stand 5D3-1 at Broadcast...

13/05/2026

XRSA and FISE Partner to Deliver Immersive Action Sports...

FISE, working with the founding members of the XR Sports Alliance (XRSA), Accedo, Qualcomm Technologies, Inc. and HBS, have collaborated to develop an immersive...

13/05/2026

Canon Unveils New EOS R6 V Full-Frame EOS Camera and RF20-50mm F4 L IS USM PZ Built-In Power Zoom Lens

Canon Unveils New EOS R6 V Full-Frame EOS Camera and RF20-50mm F4 L IS USM PZ Bu...

13/05/2026

Boston Conservatory at Berklee Honors Beth Morrison and Moses Pendleton at Commencement Ceremony

Boston Conservatory at Berklee Honors Beth Morrison and Moses Pendleton at Comme...

13/05/2026

CBS Boston to Air CCBL Baseball

Share Copy link Facebook X Linkedin Bluesky Email...

13/05/2026

Study: Free Streaming Emerges as TV's New Normal

Share Copy link Facebook X Linkedin Bluesky Email...

13/05/2026

Amagi Announces Major Enhancements To CLOUDPORT

Share Copy link Facebook X Linkedin Bluesky Email...

13/05/2026

FCC Releases Updates to TVStudy Software

Share Copy link Facebook X Linkedin Bluesky Email...

13/05/2026

Nexstar Names Elizabeth Ryder EVP, General Counsel

Share Copy link Facebook X Linkedin Bluesky Email...

13/05/2026

Foundry releases Nuke Stage- simplifying virtual producti...

Creative software developer Foundry today announced the latest developments on Nuke Stage. A purpose-built application for end-to-end virtual production and in-...

13/05/2026

Tradfluencer - The Sharon Shannon Story

A definitive portrait of one of Ireland's most influential musicians New TV documentary airs Monday 18 May on RT One and RT Player at 9.35pm Watch the...

13/05/2026

Hermes Unlocks Self-Improving AI Agents, Powered by NVIDIA RTX PCs and DGX Spark

Agentic AI is changing the way users get work done. Following the success of OpenClaw, the community is embracing new open source agentic frameworks. The latest...

13/05/2026

NVIDIA, Ineffable Intelligence Team Up to Build the Future of Reinforcement Learning Infrastructure

Reinforcement-learning agents - AI systems that learn by trial and error - can c...

12/05/2026

Beyond the Hype: A Strategic Post-Hoc Analysis of NAB 2026

Beyond the Hype: A Strategic Post-Hoc Analysis of NAB 2026 If NAB Show 2026 had an underlying theme, it was a quiet, industry-wide pivot from the high-energy sp...

12/05/2026

G&D and CT Square Establish Joint Venture for India and Middle East Distribution

Guntermann and Drunck (G&D), a Panoptec Technologies Group company, and CT Square, led by Chandresh Shah, have announced a joint venture to distribute G&D and V...

12/05/2026

Telemundo Reports Record Advertiser Demand for FIFA World Cup 2026 Coverage

With 30 days until the start of the FIFA World Cup 2026, Telemundo, the exclusive Spanish-language home of the tournament in the United States, has announced th...

12/05/2026

NHLs Stanley Pup Returns for Third Year on June 8

The NHL has announced the return of Stanley Pup for its third consecutive year, a 90-minute special featuring adoptable rescue dogs competing on a miniature rin...

12/05/2026

NBCUniversal Outlines Ad Technology and Content Strategy at 2026 Upfront

NBCUniversal presented its 2026 Upfront to advertisers at Radio City Music Hall, detailing upcoming programming across NBC, Peacock, Bravo, and Versant properti...

12/05/2026

FOX Sports Funds Fandom Research Initiative at Harvard Kennedy School

FOX Sports has announced funding for the Fandom and Social Connection Initiative at Harvard Kennedy School's Shorenstein Center on Media, Politics, and Publ...

12/05/2026

TNDV and Live Media Support NCAA Final Four Weekend for 13th Consecutive Year

TNDV and Live Media, both divisions of Live Media Group, supported live broadcast coverage around NCAA Final Four weekend in Indianapolis, including the March M...

12/05/2026

European Football Alliance Signs Distribution Agreement with Fubo Sports Network

The European Football Alliance (EFA) has announced a content distribution agreement with Fubo Sports Network, the free ad-supported streaming TV (FAST) channel ...