Mixing Clever: Salsa Sound Embraces the Power of AI in MIXaiR 2.0
22/09/2021
Monday, September 20, 2021 - 10:41
Print This Story
Salsa Sound co-founder and CEO, Rob Oldfield
Regular readers of SVG Europe will be well aware of how innovative technology companies working in sports broadcasting can be, but MIXaiR from Salsa Sound looks likely to set a new bar for live audio and the craft of sound mixing.
When the pandemic caused football games to be played in silent stadiums Salsa Sound came to the rescue of broadcasters with its vCrowd real time virtual crowd atmosphere system, which rightly won plaudits. However, MIXaiR is based on technology that the company has been working on for several years, through academia, R&D, beta versions and a soft launch of the system a year ago.
Our company was founded in 2017, but prior to that we were part of the University of Salford, where we were working on a lot of innovative audio techniques, says co-founder and CEO, Rob Oldfield. In particular we were looking at how to leverage artificial intelligence (AI) to recognise sound events and then create the best possible mix, not based on tracking but just based on what is actually [captured by] those microphones.
Fast forward to the future, next month in fact, and the new version of MIXaiR, an AI-driven system that automatically creates and enhances audio mixes for live broadcast, is set to be released.
MIXaiR 2.0 is jam packed with new features and a much more intuitive, easy to use interface, says Oldfield. The idea for v2.0 is you put all your microphones feeds in and then MIXaiR creates different submixes. So you might have a crowd, a commentary, or a pitch mix or aux in, and MIXaiR will automatically balance the levels between them, apply some processing, then create the best possible mix out of it without any human interaction, other than setting it up in the first place.
Hard tackle
According to Oldfield, the hardest mix, by a country mile, is the pitch mix. Historically, it's such a dynamic process by engineers and difficult to replicate, he explains. To make sure you've always got the nearest microphone active in the mix at any one point, requires constant attention and raising and lowering of faders. It's a really clever balancing act.
Rather like the players whose kicks it tracks, MIXaiR's AI has gone through an extensive training regime. Using machine learning technology, Salsa Sound has been training MIXaiR with many hours of content, microphone recordings and mixes across leagues.
People don't realise how hard it is to mix really well. That's why we've gone through pains to get an AI solution that takes the strain from some of the really difficult grunt work of mixing
We know what makes a great mix, says Oldfield. The AI is constantly analysing all of the sounds it is hearing. We tell it what to make of these sound events and it learns what's a kick, what's a whistle, what's a ball bounce, so that when it sees or hears sound in the wild, it can make an intelligent decision based on it.
Oldfield says MIXaiR delivers an even pitch mix, without any slightly awkward transitions and unbalanced crowd noise.
Our approach is to have the AI listen to those live microphone feeds, and when it hears a sound that's interesting like a kick, whistle, ball bounce or hitting the crossbar, you name it it will automatically add that microphone feed into the mix. It tracks the game around, always choosing the microphone that's closest to the action, and [performs] a seamless transition between them.
People don't realise how hard it is to mix really well, he adds. That's why we've gone through pains to get an AI solution that takes the strain from some of the really difficult grunt work of mixing. It can enable these guys to explore creative avenues and to create more innovative content. When you're not completely locked into your screen trying to create the best mix, you can actually lean in to MIXaiR a little bit more, let that create the stems and then you can have a bit of cognitive space to craft a mix, rather than chasing a mix.
Salsa Sound's MIXaiR GUI
The commentary and crowd noise are dynamically processed to provide mix components. These sub mixes are basically ingredients that you can throw into the output mixes, explains Oldfield. Then it's just a drag and drop process. You can create whatever output mixes you want, as many as you want, in different formats. You can have international sound, a French mix, a German mix, a Spanish mix, or whatever, and within that you can have different flavours: the stereo mix, the 5.1 mix, the mono mix. The only limitation is the number of channels that you've got available on your output board. You can also apply VST plug-ins to add your own compression or EQ or effects to the submixes, so it becomes a creative tool, allowing you to craft your mixes.
Importantly, [all mixes] are loudness normalised, he adds. If you're streaming on YouTube, or your own video on demand platform, or when it is for broadcast, they all have different loudness requirements. So within MIXaiR, you just decide which requirement you want, click on it and it will ensure that the mix adheres to those standards, so you don't end up with a fine from the regulator for making too much noise, or making not enough noise.
MIXaiR users also have an option to enhance the pitch mix with pre-recorded sounds. When somebody whacks the ball in the middle of the park, the [sound supervisor] can't even hear it because it's a long way away from the microphones, says Oldfield. However the AI can pick up a kick in the middle of the pitch. The algorithm can detect a kick, go to a bank of pre-recorded kicks and pick out the most appropriate one, and then
LINK: | https://www.svgeurope.org/blog/headlines/mixing-clever-salsa-sound-emb... |
See more stories from svg |
More from Sports Video Group
24/04/2024
Howie Schwab - Beloved ESPN Researcher, Producer, and Trivia Star - Dies at 63
Howie Schwab - Beloved ESPN Researcher, Producer, and Trivia Star - Dies at 63 By SVG Staff Wednesday, April 24, 2024 - 6:46 am Print This Story | Subscri...
24/04/2024
Miss the SVG Venue Summit at Allegiant Stadium? All Panel Recordings Now Available on SVG PLAY
Miss the SVG Venue Summit at Allegiant Stadium? All Panel Recordings Now Availab...
23/04/2024
Soccer's Appeal to Younger Audiences: Inside the Gamification of European Football Coverage
Soccer's Appeal to Younger Audiences: Inside the Gamification of European Fo...
23/04/2024
With NBA and NHL Playoffs in Full Swing, Never a Dull Moment for TNT Sports Production and Ops Teams
With NBA and NHL Playoffs in Full Swing, Never a Dull Moment for TNT Sports Prod...
23/04/2024
NAB 2024: AT&T Business's Scott Beckett Details Transmission Firepower With Robo Dog, Mini Copter
NAB 2024: AT&T Business's Scott Beckett Details Transmission Firepower With ...
23/04/2024
NAB 2024: Manufacturers Focus on Cinematic Look for Live Productions
NAB 2024: Manufacturers focus on cinematic look for live productions By George Bevir Tuesday, April 23, 2024 - 10:40 Print This Story With live sports pro...
23/04/2024
Save the Date: SVG Sports Content Management Forum Returns to NYC on July 24
Save the Date: SVG Sports Content Management Forum Returns to NYC on July 24 By SVG Staff Tuesday, April 23, 2024 - 4:05 pm Print This Story | Subscribe ...
22/04/2024
NEP Battles Weather to Keep the Going Good for Grand National Coverage
NEP battles weather to keep the going good for Grand National coverage By Adrian Pennington Monday, April 15, 2024 - 11:48 Print This Story The 2024 Grand...
22/04/2024
ITV Racing Under Starters Orders Live at the Grand National Festival
ITV Racing under starters orders live at the Grand National Festival By Adrian Pennington Friday, April 12, 2024 - 09:10 Print This Story The Grand Nation...
22/04/2024
Inside the Deal: EFL Fans to View Enhanced Broadcasts For the Next Five Seasons as IMG Becomes Official Production Partner
Inside the deal: EFL fans to view enhanced broadcasts for the next five seasons ...
22/04/2024
To 4G, or Not To 4G: EMG on Bringing the Iconic 2024 London Marathon to Viewers Globally
To 4G, or not to 4G: EMG on bringing the iconic 2024 London Marathon to viewers ...
22/04/2024
NAB Show 2024: Our Full Collection of Video Interviews From the Show Floor
NAB Show 2024: Our Full Collection of Video Interviews From the Show Floor Industry insights from key leaders on the NAB Show exhibit floor By SVG Staff Mond...
22/04/2024
PENN Entertainment Adds ESPN/Disney Exec Aaron LeBerge as CTO
PENN Entertainment Adds ESPN/Disney Exec Aaron LeBerge as CTO By SVG Staff Monday, April 22, 2024 - 3:18 pm Print This Story | Subscribe Story Highlight...
19/04/2024
NAB 2024 in Review: West Hall - SportsTech Buzz From AWS, Appear, Ateme, The Switch, TATA Communications, Telstra, TVU Networks, Verizon, and More
NAB 2024 in Review: West Hall - SportsTech Buzz From AWS, Appear, Ateme, The Swi...
19/04/2024
NAB 2024 Draws 61,000+ Attendees With More Than Half Attending the Show for the First Time
NAB 2024 Draws 61,000+ Attendees With More Than Half Attending the Show for the ...
18/04/2024
Ping Pong for the Masses: Bringing an Old Sport to a Brand New Audience With World Table Tennis
Ping pong for the masses: Bringing an old sport to a brand new audience with Wor...
18/04/2024
Fanatical Fans: IMG on Bringing the World of Table Tennis to a New Global Audience for WTT
Fanatical fans: IMG on bringing the world of table tennis to a new global audien...
17/04/2024
Live From NAB Show 2024: Zixi CEO Gordon Brooks on Live Cloud Production with the NHL and Version 17 of the Zixi Platform
Live From NAB Show 2024: Zixi CEO Gordon Brooks on Live Cloud Production with th...
17/04/2024
SportsTechBuzz at NAB 2024: Wednesday's Latest From Vegas
SportsTechBuzz at NAB 2024: Wednesday's Latest From Vegas By SVG Staff Wednesday, April 17, 2024 - 3:00 am Print This Story The NAB Show is in full sw...
17/04/2024
Live From NAB Show 2024: Zixi CEO Gordon Brooks on Live Cloud Production with the NHL and Version 12 of the Zixi Platform
Live From NAB Show 2024: Zixi CEO Gordon Brooks on Live Cloud Production with th...
16/04/2024
SportsTechBuzz at NAB 2024: Tuesday's Latest From Vegas
SportsTechBuzz at NAB 2024: Tuesday's Latest From Vegas By SVG Staff Tuesday, April 16, 2024 - 3:15 am Print This Story The NAB Show is in full swing,...
16/04/2024
Live From NAB Show 2024: Calrec's Henry Goodman, Solid State Logic's Tom Knowles on Audio in the Cloud
Live From NAB Show 2024: Calrecs Henry Goodman, Solid State Logics Tom Knowles o...
16/04/2024
Live From NAB Show 2024: The Switch's George Lopez on Remote Production, The Cloud, and the Future of Connectivity
Live From NAB Show 2024: The Switchs George Lopez on Remote Production, The Clou...
16/04/2024
Live From NAB Show 2024: LiveU's Steve Wind-Mozley on IP, REMI Solutions for Light Weight Production
Live From NAB Show 2024: LiveUs Steve Wind-Mozley on IP, REMI Solutions for Ligh...
16/04/2024
Live From NAB Show 2024: Blackmagic Design's Bob Caniglia on Upgrades to ATEM Production Switchers, Recent Work in Replay and IP
Live From NAB Show 2024: Blackmagic Designs Bob Caniglia on Upgrades to ATEM Pro...
16/04/2024
Live From NAB Show 2024: Canon's Staff Gives a Tour of New Lenses, Multi-Camera Control App, Enhanced Use of Free Viewpoint
Live From NAB Show 2024: Canons Staff Gives a Tour of New Lenses, Multi-Camera C...
15/04/2024
SportsTechBuzz at NAB 2024: Monday's Latest From Vegas
SportsTechBuzz at NAB 2024: Monday's Latest From Vegas By SVG Staff Monday, April 15, 2024 - 3:00 am Print This Story The NAB Show is in full swing, a...
15/04/2024
Live From NAB Show 2024: Vizrt's Nicholas Jameson on the Democratization of High-End Graphics Tech
Live From NAB Show 2024: Vizrts Nicholas Jameson on the Democratization of High-...
15/04/2024
Live From NAB Show 2024: Sony's Hugo Gaggioni on the Power of Sharing and Connecting Live Production Resources
Live From NAB Show 2024: Sonys Hugo Gaggioni on the Power of Sharing and Connect...
15/04/2024
Live From NAB Show 2024: Lawo's Andreas Hilmer Walks Through HOME Apps, Dishes Latest on IP
Live From NAB Show 2024: Lawos Andreas Hilmer Walks Through HOME Apps, Dishes La...
15/04/2024
Live From NAB Show 2024: AWS' Julie Souza on the Maturation of Live Sports Production in the Cloud
Live From NAB Show 2024: AWS Julie Souza on the Maturation of Live Sports Produc...
15/04/2024
Live From NAB Show 2024: FUJINON Lenses' Stosh Durbacz on the Debut of the Duvo HZK14-100mm Lens, Adding Cinematic Feel to Broadcasts
Live From NAB Show 2024: FUJINON Lenses Stosh Durbacz on the Debut of the Duvo H...
14/04/2024
SportsTechBuzz at NAB 2024: Sunday's Latest From Vegas
SportsTechBuzz at NAB 2024: Sunday's Latest From Vegas By SVG Staff Sunday, April 14, 2024 - 12:41 am Print This Story The NAB Show kicks off today, a...
14/04/2024
Live From NAB Show 2024: Program Productions' Bob Carzoli on Investment Growth and Supporting Freelancers with the ProCrewz App
Live From NAB Show 2024: Program Productions Bob Carzoli on Investment Growth an...
14/04/2024
Live From NAB Show 2024: Appear's Thomas Bostrm Jrgensen, Alex Pannell, and Ed McGivern on Backing the Continued Growth of Live Sports
Live From NAB Show 2024: Appears Thomas Bostr m J rgensen, Alex Pannell, and Ed ...
14/04/2024
Live From NAB Show 2024: Amagi's Dan Marshall on Migrating On-Prem Data Centers to The Cloud and the Exciting Growth of FAST Channels
Live From NAB Show 2024: Amagis Dan Marshall on Migrating On-Prem Data Centers t...
14/04/2024
Live From NAB Show 2024: Ross Video's David Ross on Celebrating 50 Years with Major Product Intros
Live From NAB Show 2024: Ross Videos David Ross on Celebrating 50 Years with Maj...
14/04/2024
NAB 2024: NHL, Verizon Business, and AWS Roll Out 5G-enabled Cloud Broadcasting Solution
NAB 2024: NHL, Verizon Business, and AWS Roll Out 5G-enabled Cloud Broadcasting ...
12/04/2024
Inside the MultiVersus NHL Face-Off' Live Animated Broadcast With the NHL's David Lehanski and Keith Horstman
Inside the MultiVersus NHL Face-Off' Live Animated Broadcast With the NHL...
12/04/2024
NAB 2024: Suppliers Continue To Expand Networked-Audio Offerings
NAB 2024: Suppliers Continue To Expand Networked-Audio Offerings New and updated products boast IP and ST 2110, software systems, virtual control By Dan Daley,...
12/04/2024
Amazon Prime Video Chooses Onsite Production Path for Its 27 NWSL Matches
Amazon Prime Video Chooses Onsite Production Path for Its 27 NWSL Matches The streamer made its league debut with the 2024 NWSL Challenge Cup on March 15 By Kr...
12/04/2024
ESPN Rides Wave of Women's-Hoops Momentum Into WNBA Draft Production at Brooklyn Academy of Music
ESPN Rides Wave of Women's-Hoops Momentum Into WNBA Draft Production at Broo...
12/04/2024
SVG New Sponsor Spotlight: Mimir's Andr Torsvik on Enabling Remote Collaboration Through the Cloud
SVG New Sponsor Spotlight: Mimir's Andr Torsvik on Enabling Remote Collabor...
12/04/2024
Inside the MultiVersus NHL Face-Off Live Animated Broadcast With the NHL's David Lehanski and Keith Horstman
Inside the MultiVersus NHL Face-Off Live Animated Broadcast With the NHL's D...
11/04/2024
Case Study: Inside ESPN's Monday Night Football AR Graphics Launch
Case Study: Inside ESPN's Monday Night Football AR Graphics Launch By SVG Staff Thursday, April 11, 2024 - 8:18 am Print This Story | Subscribe Stor...
11/04/2024
All Love: Creative Media Producer Jhalen Wingate on Capturing South Carolina Women's Basketball's National Championship
All Love: Creative Media Producer Jhalen Wingate on Capturing South Carolina Wom...
11/04/2024
Miami Marlins Leap Into Artificial Intelligence With 2024 Team Open
Miami Marlins Leap Into Artificial Intelligence With 2024 Team Open The Famous Group played a pivotal role in its third consecutive project with the club By Kr...