The Audio Engineering Society's Technical Council (AESTC) has formed a group to advance the science and application of Audio for New Realities. It will serve the needs of the audio community working in virtual-, augmented- and mixed-reality environments.
New realities covers the gamut of application areas: film, games, music, communications, medicine, forensics, simulations, education, virtual tourism. The technical requirements cover obvious areas, such as spatial audio and synthesis, but extend beyond them into audio networking, semantic audio, perception, broadcast, and online delivery. There will therefore be a need for good cooperation, with existing AESTC Technical Committees providing leadership in related disciplines.
Dr. Gavin Kearney, Leader of AES Technical Committee Group On Audio for New Realities
The group will initially operate under the Audio for Games Technical Committee. SVG sat down with the leader of the new group, Dr. Gavin Kearney, researcher and lecturer at the University of York, and Michael Kelly, Vice-Chair, AESTC, to discuss the implications for broadcast audio in the VR/AR environment.
Microphones have followed cameras into racecar cockpits and onto athletes themselves. These are areas in which VR/AR can take viewers deeper. What's the nature of the audio that will have to accompany them into a virtual version of those experiences?
Gavin Kearney: Being able to use VR to give yourself the viewing perspective from inside a sports car or from within a playing field is a paradigm shift in sports spectacle. But the sound design for VR requires more than just being realistic; it needs to be hyper-real. Perhaps the end user wants more crowd sounds or more music or to spatially place the commentary at a particular position around them. Object-based audio broadcast can give the end user the power to change the soundtrack to suit the desired experience.
It comes as no surprise then that the [capture], sound design, and mixing of such events is more challenging than ever. The sound engineer has to skillfully work with and group combinations of dedicated ambient soundfield microphones, spot microphones, and traditional soundtrack elements - such as Foley and music - and transform them into an active VR audio mix in 3D space.
Since the majority of VR audio is presented over headphones, it has to be able to fully render the soundfield naturally in three dimensions and to be readily translated so that, when the viewer moves their head, the soundfield is counter-rotated to ensure stable sound-source placement around the viewer. There are three key aspects here: excellent head-tracking provided by the VR headset, good sound recording and sound design, and good sets of binaural filters that deliver spatially and tonally excellent audio quality.
Michael Kelly: VR for sports faces many of the same challenges as many other live applications (capture, mixing, recording, delivery), audio or otherwise. The challenge with sports is that these are often at their extremes: such as how would you deal with potential nausea of placing someone in a live racing car when they're at home on the sofa?
But there are additional specific audio challenges: for example, miking a rally car to give creative control of the mix as the listener would expect to hear it is no easy task. It's not a new challenge. Game-sound designers have a lot of experience in this, but that's about re-creating a live experience rather than capturing a current one. A big question in this domain is how real does the listener expect it to be? Can we use dramatic license to create a better experience by mixing the real event with Foley?
Will Dolby Atmos and MPEG-H play a role in this?
GK: The AES is committed to seeing outstanding technical delivery of VR audio at all levels, and VR/AR for sports broadcast is no exception. MPEG-H has become an important technology for the delivery of object- and scene-based audio and will certainly permeate sports broadcast. Similarly, we'll no doubt see further technical developments from companies like Dolby in supporting VR audio for a range of applications.
But what's also interesting is the adoption of older technologies, such as Ambisonic surround sound. It's easy to rotate, tilt, and tumble a binaurally rendered Ambisonic mix to counter head movements, resulting in stable source positions to match the VR visuals. As a technology that was first conceived as a competitor in the domestic Quadraphonic era, it is now fully established in VR and AR alongside object-based audio, through the support of MPEG-H.
MK: I think so. Atmos, DTS:X, and MPEG-H were conceived before VR, but the approach they take is highly applicable to VR, and, in many ways, this will totally unlock them. We are also aware of the role of Ambisonics as another highly applicable and much older format that has found its role in VR. Ultimately, these approaches all have individual drawbacks, and it will be interesting to see them evolve along with the way people use them. This is a core focus of the group.
What are the technical challenges AES researchers face in this pursuit?
GK: There are challenges at all levels. On the content-creation side, we are looking at new recording methods, workflows, and mixing environments in the VR space. In particular, how should sound designers be using immersive 360 audio for VR and AR to not only enhance the immersive sporting experience to give a feeling of you are there but also to augment it and give added value to using VR for sports?
On the reproduction side, achieving superb-quality binaural rendering is a high priority but can be difficult to achieve. Should content creators utilize their own personalized binaural filters, or generic filters when mixing? Are there standard binaural virtualization settings that we should use for th
Most recent headlines
04/09/2025
Monumental Sports & Entertainment (MSE), in collaboration with Dalet, has been a...
09/05/2025
Wooden Camera, a leading innovator in camera accessories, is proud to announce the launch of a new line of products for the Blackmagic URSA Cine Cameras. These ...
09/05/2025
LYNX Technik, provider of modular signal processing solutions is expanding its popular Series 5000 platform with three new 12G-SDI cards. The new cards provide ...
09/05/2025
Pebble, the leading automation, content management and integrated channel specialist, has completed a project to migrate ON TV, the major independent broadcaste...
09/05/2025
NUGEN Audio launches its new speech intelligibility plug-in, DialogCheck. A mono, stereo and multichannel dialog clarity meter, the software provides an objecti...
09/05/2025
Disguise the technology platform and solutions provider that powered MDL Beast Soundstorm 2024, Dubai Expo's Al Wasl Dome and visual displays at the Burj ...
09/05/2025
CVP has announced it will demonstrate its cutting-edge broadcast, live production and studio solutions at MPTS 2025. Taking place at Olympia London from 14th - ...
09/05/2025
Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, and White Peaks Solutions, the prominent e...
09/05/2025
SportsEngine Play, the first-of-its-kind streaming service for youth sports, has agreed to a multi-year extension with AI-automated sports content leader, Pixel...
09/05/2025
Friend MTS, a leading global provider of content protection services, today announced that TOD, the premier sports and entertainment OTT subscription service fo...
09/05/2025
ZEISS announces that it is in development on a new virtual lens technology that will allow visual effects and compositing artists to apply an authentic lens loo...
09/05/2025
At the 2025 Media Production & Technology Show (MPTS), Cerberus Tech will showcase the latest major enhancement to its Livelink IP video delivery platform the...
09/05/2025
Capitol Broadcasting Company welcomed Warren Thomas as the new Vice President an...
09/05/2025
CAMBRIDGE, Mass. Studio Technologies said the Harvard University athletics department has integrated Dante-enabled equipment from the vendor into its broadcast ...
09/05/2025
NEW YORK ITN, a provider of a local linear supply side platform, and Magnite, a independent sell-side advertising company, have announced that they working toge...
09/05/2025
LEEDS, U.K. Nugen Audio has launched a new speech intelligibility plug-in, DialogCheck and offered up quotes from technologists working at places like Netflix p...
09/05/2025
AJA I/O Gear: The Heart of Broadcast Solutions' VEGO Mobile Editing Solution
Brie Clayton May 8, 2025
0 Comments
When working remotely on broadcas...
09/05/2025
What do they teach in an Advanced Adobe After Effects Course?
Roland Kahlenberg May 8, 2025
0 Comments
There aren't many advanced After Effects co...
09/05/2025
Larry Jordan Sits with Trevor Morgan of OpenDrives at NAB 2025
Brie Clayton May 8, 2025
0 Comments
Trevor Morgan, COO of OpenDrives, shares how the co...
09/05/2025
Berklee Popular Music Institute Announces UK Festival Debut and Tour Dates For the first time, BPMI will bring Berklee-affiliated artists and students to the ...
09/05/2025
Here is your host, Patrick Kielty!
In the season finale of The Late Late Show, ...
09/05/2025
My Eurovision Party 2025 - join in on the fun with RT Kids!
Eurovision launche...
08/05/2025
A sinister fairy infiltrates a desperate family in Kenneth Dagatan's In My Mother's Skin, which premiered at the 2023 Sundance Film Festival. Photo co...
08/05/2025
As expected, continued weak demand from key sales markets and declining economic...
08/05/2025
A new three-part series is coming to BBC iPlayer and BBC One
(Image: The Christie Archive Trust)
The BBC has announced Agatha Christie's Endless Night, a...
08/05/2025
For skyward-bound operators, training focuses on the unique aspects of flying ISR missions, including the management of onboard surveillance equipment and the e...
08/05/2025
The cable industry has told the Federal Communications Commission it supports the National Association of Broadcasters' proposal to allow broadcasters to us...
08/05/2025
WASHINGTON The Consumer Technology Association has continued its opposition to mandates requiring that NextGen TV/ATSC 3.0 tuners be included in new TV sets, sa...
08/05/2025
TAG Video Systems, the leader in software-based IP end-to-end workflow monitoring, deep probing, and real time visualization, has named Paul Maroni as Vice Pres...
08/05/2025
This year's UK Pavilion in hall 5, once again managed by Tradefair, will provide visitors with the unique opportunity to discuss and be involved in cutting ...
08/05/2025
Rohde & Schwarz will showcase its latest energy-efficient transmitters and 5G Broadcast technologies, designed to support network operators and content provider...
08/05/2025
IRVING, Texas Nexstar Media Group has tapped Bill Nardi as vice president of station operations, responsible for overseeing the day-to-day broadcast operations ...
08/05/2025
SEATTLE LumaTouch is partnering with CNN Academy to improve mobile storytelling techniques and support training across all of CNN Academy's training simulat...
08/05/2025
WASHINGTON The Society of Broadcast Engineers has filed comments with the Federal Communications Commission that support a proposal by the National Association ...
08/05/2025
Senior adviser to the United States Agency for Global Media Kari Lake has announced that One America News Network (OAN) will provide newsfeed services for fre...
08/05/2025
EdMon Expands as AI-Driven Post Production Workflows Gains Traction in Sweden an...
08/05/2025
Using Luma Mattes in Adobe Premiere Pro
Graham Quince May 7, 2025
0 Comments
This very quick tutorial shows you how to take an RGB clip and apply its ...
08/05/2025
OpenDrives Unveils Free Your Data' Initiative with New Astraeus Cloud-Nativ...
08/05/2025
Student Spotlight: Grigori Balasanyan The Armenian composer, who was named Boston Conservatory at Berklees 2025 student commencement speaker, talks about his ...
08/05/2025
08 May 2025
VEON Shareholders Re-elect Board at 2025 AGM, Founder Augie Fabela ...
08/05/2025
Comedy and entertainment channel U&Dave bring back their #1 ranked programme of ...
08/05/2025
May 8th, 2025 Press Materials Available Here
Tribeca Festival 2025 Unveils New Premieres Spanning Film and Music
Slick Rick's Victory with Idris Elba a...
08/05/2025
May 8th, 2025 Press Materials Available Here
Tribeca Festival 2025 Announces Lineup for Inaugural Storytelling Summit
11-Day Industry Event Launches with Tal...
08/05/2025
SVG Sit-Down: Vizrt's Nicholas Jameson on AI in Workflows, Pushing Boundarie...
08/05/2025
Creating Alternative Brand Experiences: Live Sports in the Age of Fortnite, Meta...
08/05/2025
PGA TOUR's David Piccolo: Advanced Graphics and Virtual Production Tools are...
08/05/2025
Tech Focus: Advancing Immersion in Sports Broadcasting with AR and Virtual Produ...
08/05/2025
Expect even more chaos this July with Harriet Webb leading the returning cast, p...
08/05/2025
Back to All News
Now in Production: Comedy Action Film Husbands in Action'...
The Audio Engineering Society's Technical Council (AESTC) has formed a group to advance the science and application of Audio for New Realities. It will serve the needs of the audio community working in virtual-, augmented- and mixed-reality environments.
