Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

12/08/2026

ACLU Requests ABC Station Probe Documents from FCC

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Roger Keating To Retire from Hearst Television

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Netflix Closes U.S. Upfronts, Nearly Doubles Ad Commitments

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

CBS Pittsburgh Debuts New AR/VR News Studio

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Matthews Intros T-Go Extreme Rigging Solutions

Matthews Studio Equipment introduces T-Go Mounting Solutions extreme camera rigging systems, engineered for the most demanding environments in motion picture pr...

12/08/2026

Anton Bauers EDEN Becomes Only Mobile Power Station in it...

Anton/Bauer, a Videndum brand and the industry leader in mobile power solutions for broadcast and cinematic production, announces that the EDEN Mobile Power Sta...

12/08/2026

TAG Video Systems Cuts Hardware Costs for Customers with...

TAG Video Systems, the global leader in software-based IP monitoring and multiviewer solutions, today announced a memory usage optimization that lowers the tota...

12/08/2026

Unified Streaming Launches Trusted Media - C2PA in Dynami...

Unified Streaming announces Trusted Media, a new solution that enables broadcasters, streaming platforms, and premium content owners to attach verifiable authen...

12/08/2026

Tedial Powers the AI-Driven Media Supply Chain at IBC 202...

Combining intelligent automation, composable business services and user-centric experiences The media industry is entering a new phase where AI intelligence is...

12/08/2026

DPA Microphones Appoints Raul Contreras as West Coast Reg...

DPA Microphones welcomes Raul Contreras as its new West Coast Regional Sales Manager. In this role, Contreras will oversee sales and customer relationships thro...

12/08/2026

Weather is Not a Factor for Marshall Electronics New CV57...

Marshall Electronics' first all-weather, all-IP 4K POV camera, the CV574-WP, makes its European debut at IBC 2026 (Stand 10.C11). Built for the elements, th...

12/08/2026

A double off-switch helps cells respond to stress

LA JOLLA, CA-Scientists at Scripps Research have discovered that two seemingly unrelated cellular enzymes become locked together during times of stress, simulta...

11/08/2026

Apple To Bring Friday Night Baseball Live to Apple Immersive on Vision Pro Beginning Aug. 28

Weekly immersive broadcasts will feature 8K 3D video, a 180-degree field of view...

11/08/2026

Looking for a New Cameraperson? ProductionHUB Guide Lays Out Considerations, Pitfalls

By Briana Brady, Social Media Manager, Production Hub Sports content is having ...

11/08/2026

SVG Sit-Down: Backlight CEO Kathleen Barrett on Building Media Infrastructure for a Global, AI-Driven Sports Landscape

She addresses why media organizations need more than asset storage, how sports t...

11/08/2026

FloSports Dennis LeMire on Implementing Cloud-Based Workflows on Lower Tier Sports

Remote commentary has improved significantly on these broadcasts At a clip of ...

11/08/2026

Miami HEAT's Revamped IP-Based Control Room Coordinates Workflows With Meinberg Intelligent Modular Synchronization System

A pair of LANTIME IMS M2000 PTP Grandmasters deliver resilient timing infrastruc...

11/08/2026

LTN Expands Live Event Master Control Capabilities for Sports Organizations

LTN has announced expanded capabilities for its Live Event Master Control offering, providing sports broadcasters, leagues, rights holders, and content owners w...

11/08/2026

WTA Ventures Appoints IMG as Managed Production Services Partner

WTA Ventures has appointed IMG as its managed production services partner in a multi-year agreement commencing with the 2027 season. The partnership is the late...

11/08/2026

New York Jets and CBS New York Extend Multi-Year Partnership

The New York Jets and CBS New York have agreed to a multi-year extension of their partnership, keeping CBS New York as the official broadcast home of the team. ...

11/08/2026

Nielsen Acquires DoubleVerify to Create New Combined Media Intelligence Platform

Nielsen Holdings and DoubleVerify have announced a definitive agreement under which Nielsen will acquire DoubleVerify in an all-cash transaction with an enterpr...

11/08/2026

AES Elects David v.R. Bowles as President-Elect for 2027, Unveils Other Election Results

The Audio Engineering Society has announced the results of its 2026 Board of Dir...

11/08/2026

SVG Sit-Down Creator Sports Network's Barrick Prince on Building the Next Layer of the Live-Sports Rights Stack

The CSN co-founder details its network and affiliate' model, how creator wi...

11/08/2026

Gravity Media Providing Host Broadcast Operations for Esports World Cup 2026

Gravity Media is providing host broadcast operations for the Esports World Cup (EWC) 2026, supporting ESL FACEIT Group (EFG) in delivering the tournament to glo...

11/08/2026

TAG Video Systems Reduces Memory Requirements for Compressed Workflows

TAG Video Systems has released a memory usage optimization for its MCM software that reduces the cost per channel by up to two-thirds for compressed workflows. ...

11/08/2026

SMT Expands LED Production and Scoring Services at 2026 Cincinnati Open

SMT is expanding its technology partnership with the Cincinnati Open this year, adding new LED displays, production capabilities, and fan-facing experiences at ...

11/08/2026

Pac-12 to Stream Fall Olympic Sports Home Events Free on YouTube

The Pac-12 has announced that home Olympic sports broadcasts throughout the fall season will be available free on the conference's YouTube channel at youtub...

11/08/2026

Case Study: Real Life Network Taps LiveU for Broadcast-Grade Streaming

Spreading a message of faith requires both widespread accessibility and uncompromising visual quality. Real Life Network (RLN), a free, faith-based digital stre...

11/08/2026

Tennessee Titans' New Nissan Stadium Will Live Up to Its Music City Location

Taking the synergy of sports and entertainment a step further, the NFL venue in Nashville will house iterations of two iconic music clubs The marriage of Taylo...

11/08/2026

The Crow Hill Company introduce Pocket Strings

Free library captures pro classical string section There's no shortage of freebies in The Crow Hill Company's line-up, with their Vaults series offe...

11/08/2026

Emergence Audio unveil Envoy

Percussion instrument introduces all-new engine Emergence Audio have just introduced a instrument that brings together a vast collection of percussive analo...

11/08/2026

Boris FX reveal Sequoia 2026.5

Introduces new all-in-one Spatial Panner tool Boris FX have recently announced the latest major update for their mastering and broadcast-focused DAW softwar...

11/08/2026

Rohde & Schwarz installs millimeter wave security scanners at Airports of Thailand (AOT) Airports

Rohde & Schwarz installs millimeter wave security scanners at Airports of Thaila...

11/08/2026

Dolby Acquires VVC Patents From Florida Atlantic University

Share Copy link Facebook X Linkedin Bluesky Email...

11/08/2026

Alyson Stewart Joins Zixi as Senior Director, Sales

Share Copy link Facebook X Linkedin Bluesky Email...

11/08/2026

FCC Issues Alert Over Scammers Impersonating Its Commissioners

Share Copy link Facebook X Linkedin Bluesky Email...

11/08/2026

Comcast, Disney Ink Deal to End NFL Network Blackout

Share Copy link Facebook X Linkedin Bluesky Email...

11/08/2026

farmerswife and Cirkus are Exhibiting at IBC 2026

farmerswife and Cirkus will return to IBC 2026 to showcase new tools designed to help broadcast, media and post-production teams manage workflows with greater s...

11/08/2026

LiveU Powers Cross-Venue Coverage of Glasgow Commonwealth...

LiveUs LU900Q intelligent production unit played a central role in the host broadcast operation for the Games in Glasgow (July 23 to August 2, 2026). The host b...

11/08/2026

Jigsaw24 Appoints James Morse as Client Director to Drive...

Jigsaw24 has appointed James Morse as Client Director, Media Sales, as it deepens its focus on sports, live events and integrated production. His appointment co...

11/08/2026

Net Insight enables one of Europes largest broadcast grad...

Net Insight today announced that DMC Production Norway has selected its Open Live Media solution to centralize the production of Blinkfestivalen 2026. Combining...

11/08/2026

Vivaro Supports Record World Cup Audiences with the Appea...

Modular HEVC and JPEG XS workflows enabled resilient, ultra-low-latency delivery for leading linear and digital media platforms across all 104 matches Appear A...

11/08/2026

Homecoming Artist Spotlight: Fabiola Mndez

Homecoming Artist Spotlight: Fabiola M ndez The first cuatrista to graduate from Berklee talks about the community that shaped her career ahead of her Homecom...

11/08/2026

FOR-A To Highlight MixBoard, Viztrik AiDi Enhancement At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

11/08/2026

AMWA and EBU to Host Demos at IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

11/08/2026

Dalet Solution Suites Enable Media Teams to Move Like a S...

Dalet, a leading technology and service provider for media-rich organizations, today announced Dalet Solution Suites, a new portfolio of cloud-native solutions ...

11/08/2026

Mediagenix Helps Broadcasters Prepare for Brazil DTV Futu...

Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, will showcase how media companies can prep...