Sony Pixel Power calrec Sony

A Trusted Companion: AI Software Keeps Drivers Safe and Focused on the Road Ahead

19/01/2021

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don't understand subtle cues such as a driver's cognitive state, behavior or other activity that indicates whether they're ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings - such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver's eyes and mapping it to the road to check if they're able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver's emotions - a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver's state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.
LINK: https://blogs.nvidia.com/blog/2021/01/19/drive-ix-ai-software-drivers-...
See more stories from nvidia

North America Stories

28/02/2021

Six Nations Rugby on Peacock Premium, NBCSN Feb. 27

NBC Sports' coverage of Guinness Six Nations rugby continues this weekend. On Feb. 27, Italy faces Ireland in Rome. The match is live on Peacock Premium at ...

28/02/2021

AMC Turns a Profit as Streaming Subs Top Targets

AMC Networks turned a fourth-quarter profit as its streaming networks exceeded their subscriber targets, but its national cable networks faltered....

28/02/2021

Scripps Reports Higher Profits in Fourth Quarter

The E.W. Scripps Co. reported a bigger profit with a boost from political advertising and gains at its national networks business....

28/02/2021

Tom & Jerry' Movie on HBO Max Feb. 26

Tom & Jerry, a feature film starring the beloved cat and mouse, premieres on HBO Max Feb. 26. An eye-popping blend of classic animation and live action, Tom an...

28/02/2021

Primetime Ratings Thursday: Fox, ABC, CBS Split Win

ABC, CBS and Fox shared the Thursday ratings crown. All three scored a 0.6 in viewers 18-49, per the Nielsen overnights, with a 4 share. ABC had game shows, CBS...

28/02/2021

Haystack News Launches Award Show Streaming Channels

Haystack News, the ad supported streaming service, said it is launching channels dedicated to major entertainment awards starting with the Golden Globes....

28/02/2021

PBS Announces Earth Day Series With Greta Thunberg

PBS and BBC Studios have announced a new three part documentary with teen activist, Greta Thunberg as she continues to bring awareness to climate change. Greta ...

27/02/2021

Albuquerque Firm Takes Award-Winning Film Tech to The Cloud Cinnafilm PixelStrings Cloud Conversion

Albuquerque Firm Takes Award-Winning Film Tech to The Cloud Cinnafilm PixelStr...

27/02/2021

Quad Image Enhancer

Studio One's Binaural Pan processor can widen, or narrow, an instrument's stereo image. Aside from making bigger sounds that fill out the stereo sprea...

27/02/2021

Remembering Peter Burrows

Simultaneously an industry giant and a gentle one, Peter Burrows was 300 pounds of motorcycle-riding, upright bass-slapping, River Thames moxy shoehorned into a...

27/02/2021

2021 U.S. Presidential Inauguration Parade Mixed by BNY Productions with Waves eMotion LV1 Live Mixer

2021 U.S. Presidential Inauguration Parade Mixed by BNY Productions with Waves e...

26/02/2021

Final Day for Tech Leadership Product of the Year Award Nominations

WASHINGTON-The deadline for Product of the Year nominations in the Tech Leadership Awards is today, Feb. 26....

26/02/2021

Germany's Media Broadcast to Test 5G TV Production

FRANKFURT, Germany-Technical services provider Media Broadcast is set to test use cases for TV production using 5G at its 5G campus in Germany....

26/02/2021

Rise Creates Mentoring Program for Young Women

LONDON-Rise, the media tech industry women's advocacy group, has launched the Rise Up UTC Mentoring Program for young women aged 16-18 that wish to pursue a...

26/02/2021

CultureX Conversations Returns on Tuesday, March 16

The entertainment industry is examining its diversity and inclusion efforts as it continues to make inroads in recognizing the business rationale for greater re...

26/02/2021

MLB.TV Slides Onto YouTube TV

SAN BRUNO, Calif.-YouTube TV is getting ready for baseball season in its own way, announcing via Twitter that it is now adding MLB.TV as an add-on for its subsc...

26/02/2021

Comcast Brings Pay-TV Options to Flex With Xfinity Stream App

PHILADELPHIA-Comcast has announced that its Xfinity Stream app is now available on Flex 4K streaming devices, which will be available to customers at no additio...

26/02/2021

Nexstar Broadcasting President Timothy Busch to Retire

IRVING, Texas-After a 36-year career in broadcasting, including the last 20 with Nexstar, Timothy C. Bush, Nexstar's president of Broadcasting, will retire ...

26/02/2021

EditShare, Gravity Media, IHSE USA, Imagine Communications, Quantum, Signiant, Veritone, and Vizrt Group Renew Premier Sponsorships

EditShare, Gravity Media, IHSE USA, Imagine Communications, Quantum, Signiant, V...

26/02/2021

RTS Intercom Systems Boasts Sure-Fire Communications for Sports' Largest Events

RTS Intercom Systems Boasts Sure-Fire Communications for Sports' Largest Eve...

26/02/2021

Ratings Roundup: Flyers-Bruins at Lake Tahoe Becomes Most-Watched NHL Regular Season Game in NBCSN History

Ratings Roundup: Flyers-Bruins at Lake Tahoe Becomes Most-Watched NHL Regular Se...

26/02/2021

SVG Sit-Down: GV President Tim Shoulders Says the Move to the Cloud Will Transform Creativity

SVG Sit-Down: GV President Tim Shoulders Says the Move to the Cloud Will Transfo...

26/02/2021

Ikegami Introduces UHK-X700 UNICAM 4K Camera System

Ikegami Introduces UHK-X700 UNICAM 4K Camera System By Ken Kerschbaumer, Editorial Director Friday, February 26, 2021 - 2:42 pm Print This Story | Subscri...

26/02/2021

PPI ProCrewz Application Brings COVID-19 Questionnaire to Fingertips

PPI ProCrewz Application Brings COVID-19 Questionnaire to Fingertips By Ken Kerschbaumer, Editorial Director Friday, February 26, 2021 - 2:57 pm Print This...

26/02/2021

In the Drink of an AI: Startup Opseyes Instantly Analyzes Wastewater

Let's be blunt. Potentially toxic waste is just about the last thing you want to get in the mail. And that's just one of the opportunities for AI to mak...

26/02/2021

What Is Cloud Gaming?

Cloud gaming uses powerful, industrial-strength GPUs inside secure data centers to stream your favorite games over the internet to you. So you can play the late...

26/02/2021

Meet the Maker: DIY Builder Takes AI to Bat for Calling Balls and Strikes

Baseball players have to think fast when batting against blurry-fast pitches. Now, AI might be able to assist. Nick Bild, a Florida-based software engineer, ha...

26/02/2021

News Corp to Participate in the Morgan Stanley Technology, Media & Telecom Conference

News Corp to Participate in the Morgan Stanley Technology, Media & Telecom Confe...

26/02/2021

Terminator Universe is Expanding at Netflix in an All-New Anime Series

Back to All News Terminator Universe is Expanding at Netflix in an All-New Anime Series Entertainment 26 February 2021 Global Netflix and Skydance announce...

26/02/2021

Building a Legacy of Inclusion: Results From Our First Film and Series Diversity Study

Back to All News Building a Legacy of Inclusion: Results From Our First Film an...

26/02/2021

HPA Tech Retreat: How Is It Exactly the Same and Completely Different?

The HPA Tech Retreat is known for a number of things, probably primarily as a non-commercial forum where the leading minds discuss and debate and articulate the...

26/02/2021

Recognizing Outstanding Editing in Film and Television, ACE Eddie Awards to be Held April 17, 2021 at 11am PT / 2pm ET

American Cinema Editors (ACE) has announced that the 71st Annual ACE Eddie Award...

26/02/2021

Company 3 Adds Internationally-Acclaimed Yvan Lucas to Colorist Roster

Company 3 is proud to announce the addition of highly-accomplished colorist Yvan Lucas to its artist roster. Lucas has colored a highly impressive number of maj...

26/02/2021

Blackmagic Design Announces New Blackmagic Pocket Cinema Camera 6K Pro

Blackmagic Design has announced the Blackmagic Pocket Cinema Camera 6K Pro, a new more powerful model that includes features for high end digital film. This new...

26/02/2021

Sofia Coppola to Receive Board of Governors Award from American Society of Cinematographers

The American Society of Cinematographers (ASC) will present writer-producer-dire...

26/02/2021

Origin Point Illuminates the HPA Tech Retreat Supersession

At the HPA Tech Retreat, a half-a-dozen short films reveal the behind-the-scenes of the event's Supersession, a focus on the cloud-based remote production o...

26/02/2021

Company 3/Method Studios Fully Integrates Creative Editorial Team

Company 3/Method Studios' Atlanta location announces that it has merged its powerful creative editorial team (formerly provided as Beast Editorial) under th...

26/02/2021

Xytech Systems Acquires ScheduALL, Enhances Facility Management Scalability & Transmission Offerings

The acquisition, expected to complete in the coming weeks, provides customers of...

26/02/2021

More than 35 Sundance 2021 Projects Created Using Blackmagic Design

Blackmagic Design has announced that numerous projects at the 2021 Sundance Film Festival were created using Blackmagic Design products. More than 35 films, epi...

26/02/2021

New and recent this week on the AES Job Board

Posted: Friday, February 26, 2021 --...

26/02/2021

Century Han Tang Leverages AJA Technology to Streamline 4K HDR Production

Century Han Tang Leverages AJA Technology to Streamline 4K HDR Production data-src=https://creativecow.net/wp-content/uploads/avatars/263018/5fa08c562aaf4-b...

26/02/2021

Juan Cabrera, the first Spanish colorist to become an associate member of the American Society of Cinematographers

Juan Cabrera, the first Spanish colorist to become an associate member of the Am...

26/02/2021

Nielsen Swings to Profit as it Preps New Measurement System

Nielsen reported a fourth quarter profits as it prepared to revamp its audience measurement system and sell off a troublesome division....

26/02/2021

Gray Reports Higher Fourth-Quarter Profits

Gray Television reported higher fourth-quarter profits as a rush of political advertising boosted revenue....

26/02/2021

Showtime Production Halo' to Premiere on Paramount Plus

Drama Halo, the series based on the Xbox video game that's been in the works at Showtime, will premiere exclusively on Paramount Plus in early 2022. Showtim...

26/02/2021

NBCU Adds Charter to Addressable Ad Footprint

NBCUniversal said it made an advanced advertising agreement with Charter Communications that will expand NBC's U.S. addressable advertising footprint....

26/02/2021

Tastemade Adds Whip Media's Tracking to Streaming Recipe

Whip Media said it has been engaged by digital lifestyle programmer Tastemade to track the performance of its free ad supported streaming television networks....

26/02/2021

Comcast Adds Xfinity Stream App for Flex Customers

Comcast said it was making the Xfinity Stream app available to it broadband Flex customers....

26/02/2021

Primetime Ratings Wednesday: Fox on Top With Name That Tune'

Fox won the ratings race Wednesday night, with a double run of Name That Tune leading the way. Offering two hours of the musical competition series, Fox scored ...

26/02/2021

Hulu Orders Series About Mike Tyson

Hulu has ordered the limited series Iron Mike, about the wild, tragic and controversial life and career of boxer Mike Tyson, the network said. The series come...