Sony Pixel Power calrec Sony

Mixing Clever: Salsa Sound Embraces the Power of AI in MIXaiR 2.0

22/09/2021

Mixing clever: Salsa Sound embraces the power of AI in MIXaiR 2.0 By Michael Burns

Monday, September 20, 2021 - 10:41

Print This Story

Salsa Sound co-founder and CEO, Rob Oldfield

Regular readers of SVG Europe will be well aware of how innovative technology companies working in sports broadcasting can be, but MIXaiR from Salsa Sound looks likely to set a new bar for live audio and the craft of sound mixing.

When the pandemic caused football games to be played in silent stadiums Salsa Sound came to the rescue of broadcasters with its vCrowd real time virtual crowd atmosphere system, which rightly won plaudits. However, MIXaiR is based on technology that the company has been working on for several years, through academia, R&D, beta versions and a soft launch of the system a year ago.

Our company was founded in 2017, but prior to that we were part of the University of Salford, where we were working on a lot of innovative audio techniques, says co-founder and CEO, Rob Oldfield. In particular we were looking at how to leverage artificial intelligence (AI) to recognise sound events and then create the best possible mix, not based on tracking but just based on what is actually [captured by] those microphones.

Fast forward to the future, next month in fact, and the new version of MIXaiR, an AI-driven system that automatically creates and enhances audio mixes for live broadcast, is set to be released.

MIXaiR 2.0 is jam packed with new features and a much more intuitive, easy to use interface, says Oldfield. The idea for v2.0 is you put all your microphones feeds in and then MIXaiR creates different submixes. So you might have a crowd, a commentary, or a pitch mix or aux in, and MIXaiR will automatically balance the levels between them, apply some processing, then create the best possible mix out of it without any human interaction, other than setting it up in the first place.

Hard tackle

According to Oldfield, the hardest mix, by a country mile, is the pitch mix. Historically, it's such a dynamic process by engineers and difficult to replicate, he explains. To make sure you've always got the nearest microphone active in the mix at any one point, requires constant attention and raising and lowering of faders. It's a really clever balancing act.

Rather like the players whose kicks it tracks, MIXaiR's AI has gone through an extensive training regime. Using machine learning technology, Salsa Sound has been training MIXaiR with many hours of content, microphone recordings and mixes across leagues.

People don't realise how hard it is to mix really well. That's why we've gone through pains to get an AI solution that takes the strain from some of the really difficult grunt work of mixing

We know what makes a great mix, says Oldfield. The AI is constantly analysing all of the sounds it is hearing. We tell it what to make of these sound events and it learns what's a kick, what's a whistle, what's a ball bounce, so that when it sees or hears sound in the wild, it can make an intelligent decision based on it.

Oldfield says MIXaiR delivers an even pitch mix, without any slightly awkward transitions and unbalanced crowd noise.

Our approach is to have the AI listen to those live microphone feeds, and when it hears a sound that's interesting like a kick, whistle, ball bounce or hitting the crossbar, you name it it will automatically add that microphone feed into the mix. It tracks the game around, always choosing the microphone that's closest to the action, and [performs] a seamless transition between them.

People don't realise how hard it is to mix really well, he adds. That's why we've gone through pains to get an AI solution that takes the strain from some of the really difficult grunt work of mixing. It can enable these guys to explore creative avenues and to create more innovative content. When you're not completely locked into your screen trying to create the best mix, you can actually lean in to MIXaiR a little bit more, let that create the stems and then you can have a bit of cognitive space to craft a mix, rather than chasing a mix.

Salsa Sound's MIXaiR GUI

The commentary and crowd noise are dynamically processed to provide mix components. These sub mixes are basically ingredients that you can throw into the output mixes, explains Oldfield. Then it's just a drag and drop process. You can create whatever output mixes you want, as many as you want, in different formats. You can have international sound, a French mix, a German mix, a Spanish mix, or whatever, and within that you can have different flavours: the stereo mix, the 5.1 mix, the mono mix. The only limitation is the number of channels that you've got available on your output board. You can also apply VST plug-ins to add your own compression or EQ or effects to the submixes, so it becomes a creative tool, allowing you to craft your mixes.

Importantly, [all mixes] are loudness normalised, he adds. If you're streaming on YouTube, or your own video on demand platform, or when it is for broadcast, they all have different loudness requirements. So within MIXaiR, you just decide which requirement you want, click on it and it will ensure that the mix adheres to those standards, so you don't end up with a fine from the regulator for making too much noise, or making not enough noise.

MIXaiR users also have an option to enhance the pitch mix with pre-recorded sounds. When somebody whacks the ball in the middle of the park, the [sound supervisor] can't even hear it because it's a long way away from the microphones, says Oldfield. However the AI can pick up a kick in the middle of the pitch. The algorithm can detect a kick, go to a bank of pre-recorded kicks and pick out the most appropriate one, and then
LINK: https://www.svgeurope.org/blog/headlines/mixing-clever-salsa-sound-emb...
See more stories from svg

More from Sports Video Group

15/10/2021

The Fall 2021 SVG SportsTech Journal Is Now Available Online!

The Fall 2021 SVG SportsTech Journal Is Now Available Online! Go behind the scenes at the Tokyo Olympics with NBC, OBS, and more By SVG Staff Friday, Octobe...

15/10/2021

SVG Sit-Down: NEP UK's Simon Moorhead talks Brexit and Pandemic Challenges to the Mobile Production Truck Business in Europe

SVG Sit-Down: NEP UK's Simon Moorhead talks Brexit and Pandemic Challenges t...

15/10/2021

SVG Sit-Down: Fox Deportes' Sergio Verdu Aims To Synergize Technologies With Fox Sports

SVG Sit-Down: Fox Deportes' Sergio Verdu Aims To Synergize Technologies With...

14/10/2021

An Onscreen Boost: Netco Sports Delivers Extensive Graphics and Data Package for Lega Serie A

An onscreen Boost: Netco Sports delivers extensive graphics and data package for...

14/10/2021

NHL Puck Drop 2021: NHL Network Expands Showcase Schedule, Boosts Onsite Presence at Marquee Events

NHL Puck Drop 2021: NHL Network Expands Showcase Schedule, Boosts Onsite Presenc...

14/10/2021

The SVG Podcast: WNBA Production with ESPN Senior Managing Producer Sara Gaiero

The SVG Podcast: WNBA Production with ESPN Senior Managing Producer Sara Gaiero Go inside the details of ESPNs new REPCO production model and the exciting futur...

14/10/2021

Inside the Sports-Betting-Production Ops of Bleacher Report, ESPN, PGA Tour, and Yahoo Sports

Inside the Sports-Betting-Production Ops of Bleacher Report, ESPN, PGA Tour, and...

13/10/2021

NHL Puck Drop 2021: State of the Art New Sportsnet Studios Facility Officially Unveiled Tonight

NHL Puck Drop 2021: State of the Art New Sportsnet Studios Facility Officially U...

13/10/2021

ESPN Inks Five-Year Deal With NFL to Air New Monday Night Wild Card Game; Agreement Begins This Season

ESPN Inks Five-Year Deal With NFL to Air New Monday Night Wild Card Game; Agreem...

13/10/2021

NHL Puck Drop 2021: NHL SVP, Technology Grant Nodine on an Amazing Ecosystem of Data, Video

NHL Puck Drop 2021: NHL SVP, Technology Grant Nodine on an Amazing Ecosystem o...

13/10/2021

NHL Puck Drop 2021: Turner Sports Celebrates Start of Hockey Deal With Flashy Studio at Atlanta HQ

NHL Puck Drop 2021: Turner Sports Celebrates Start of Hockey Deal With Flashy St...

13/10/2021

NHL Puck Drop 2021: NHL on TNT Debuts With High-Speed Cameras, Tracking-Fueled Graphics

NHL Puck Drop 2021: NHL on TNT Debuts With High-Speed Cameras, Tracking-Fueled G...

12/10/2021

NHL Puck Drop 2021: League, APM Music Cut Production-Music Deal

NHL Puck Drop 2021: League, APM Music Cut Production-Music Deal Agreement covers league and team uses from broadcast to social media By Dan Daley, Audio Editor...

12/10/2021

NHL Puck Drop 2021: ESPN Creative Studio Team Unveils New Creative Identity, Graphics Package

NHL Puck Drop 2021: ESPN Creative Studio Team Unveils New Creative Identity, Gra...

12/10/2021

BetQL Network Taps Flowics for Live Sports Betting Data Integration

BetQL Network Taps Flowics for Live Sports Betting Data Integration By Ken Kerschbaumer, Editorial Director Tuesday, October 12, 2021 - 10:37 am Print This...

12/10/2021

Sony Leverages Innovations, Acquisitions to Transform Cloud Production Options

Sony Leverages Innovations, Acquisitions to Transform Cloud Production Options By Ken Kerschbaumer, Editorial Director Tuesday, October 12, 2021 - 12:12 pm ...

12/10/2021

Sony HDC-F5500 Brings Shallow-Depth-of-Field Capabilities to Broadcast Camera Form Factor

Sony HDC-F5500 Brings Shallow-Depth-of-Field Capabilities to Broadcast Camera Fo...

12/10/2021

BitFire Networks Opens Next-Gen LED-Based Studios in Phoenix

BitFire Networks Opens Next-Gen LED-Based Studios in Phoenix By Ken Kerschbaumer, Editorial Director Tuesday, October 12, 2021 - 1:45 pm Print This Story ...

12/10/2021

NHL Puck Drop 2021: ESPN To Deploy Shallow-Depth-of-Field Steadicam on the Ice, SupraCam in the Air

NHL Puck Drop 2021: ESPN To Deploy Shallow-Depth-of-Field Steadicam on the Ice, ...

12/10/2021

NHL Puck Drop 2021: In Bristol, ESPN Opens New Studio for The Point, Continues To Push the REMI Envelope

NHL Puck Drop 2021: In Bristol, ESPN Opens New Studio for The Point, Continues T...

12/10/2021

NHL Puck Drop 2021: ESPN Director Doug Holmes and Producer Jeff Dufine on What To Expect From the Reimagined NHL on ESPN

NHL Puck Drop 2021: ESPN Director Doug Holmes and Producer Jeff Dufine on What T...

11/10/2021

Evolution Not Revolution: Sky Sport, Telefonica Discuss 5G in Sports Broadcasting and Production

Evolution not revolution: Sky Sport and Telefonica discuss the use of 5G in spor...

11/10/2021

ESPN's Monday Night Football Crew Tests Rare S35 PL Mount FUJINON Lens on Near Sideline Cart Camera

ESPN's Monday Night Football Crew Tests Rare S35 PL Mount FUJINON Lens on Ne...

11/10/2021

With Unified IO Sports Platform for XR Tools, Vizrt Charts Course for the Future

With Unified IO Sports Platform for XR Tools, Vizrt Charts Course for the Future Launch is intended to future-proof' development of live virtual graphics ...

10/10/2021

Live From Fury-Wilder III: Fox Sports Works With ESPN to Chronicle Last Installment of Heated Heavyweight Rivalry

Live From Fury-Wilder III: Fox Sports Works With ESPN to Chronicle Last Installm...

08/10/2021

Grass Valley's Tim Shoulders Says Business Is Back; Company Rebrands, Focuses on Live Production

Grass Valley's Tim Shoulders Says Business Is Back; Company Rebrands, Focuse...

08/10/2021

The SVG Summit Returns to NYC Dec. 13-14, Registration Now Open

The SVG Summit Returns to NYC Dec. 13-14, Registration Now Open SVG set to reunite industry at the annual conference in New York City By SVG Staff Friday, O...

07/10/2021

MLB Postseason 2021: Fox's New Graphics Package Showcases Gamification, Natural Elements of Baseball

MLB Postseason 2021: Fox's New Graphics Package Showcases Gamification, Natu...

07/10/2021

Thursday Night Football Returns to Amazon Prime Video With Multiple Audio Options, Expanded AI-Driven X-Ray Features

Thursday Night Football Returns to Amazon Prime Video With Multiple Audio Option...

07/10/2021

MLB Postseason 2021: Fox Sports To Televise All Playoff Games in 4K HDR; Megalodon To Make Playoff Debut

MLB Postseason 2021: Fox Sports To Televise All Playoff Games in 4K HDR; Megalod...

07/10/2021

Sony Pulls Out of 2021 IBC Show

Sony Pulls Out of 2021 IBC Show By SVG Staff Thursday, October 7, 2021 - 12:53 pm Print This Story | Subscribe Story Highlights Sony has announced that...

07/10/2021

MLB Postseason 2021: MLB Network Excited to Be Back on Site for ALDS

MLB Postseason 2021: MLB Network Excited to Be Back on Site for ALDS By Ken Kerschbaumer, Editorial Director Thursday, October 7, 2021 - 1:02 pm Print This...

06/10/2021

Salsa Sound Eyes U.S. Market for Automated Audio Mixing

Salsa Sound Eyes U.S. Market for Automated Audio Mixing College football is an early target sector By Dan Daley, Audio Editor Wednesday, October 6, 2021 - 6:...

06/10/2021

MLB Postseason 2021: After the COVID Year, Fox Sports Counts on Great Sound

MLB Postseason 2021: After the COVID Year, Fox Sports Counts on Great Sound The kind of stuff audio people really like hearing' By Dan Daley, Audio Editor...

06/10/2021

Tech Focus: Immersive Monitoring, Part 1 - Sports Broadcasting Seems a Good Fit

Tech Focus: Immersive Monitoring, Part 1 - Sports Broadcasting Seems a Good Fit Questions remain on the format's deployment, particularly in production truc...

06/10/2021

MLB Postseason 2021: TBS Preps for Big-Sound Wild Card Game

MLB Postseason 2021: TBS Preps for Big-Sound Wild Card Game A1 Dave Grundtvig will deploy a primarily conventional approach By Dan Daley, Audio Editor Wednes...

06/10/2021

Tech Focus: Immersive Monitoring, Part 2 - Options for Mobile Production Continue To Expand

Tech Focus: Immersive Monitoring, Part 2 - Options for Mobile Production Continu...

05/10/2021

Ryder Cup Reflections: PGA TOUR Entertainment's Greg Hopfe On Cross-Atlantic Teamwork

Ryder Cup Reflections: PGA TOUR Entertainment's Greg Hopfe On Cross-Atlantic...

05/10/2021

MLB Postseason 2021: After Chaotic Regular-Season Finish, ESPN Heads to Boston for Epic AL Wild Card Game

MLB Postseason 2021: After Chaotic Regular-Season Finish, ESPN Heads to Boston f...

05/10/2021

MLB Postseason 2021: Turner Sports Tackles National League With New AR-Heavy Set, In-Game Graphic Animations

MLB Postseason 2021: Turner Sports Tackles National League With New AR-Heavy Set...

04/10/2021

World Padel Tour Among First Customers as LaLiga Tech Launches Digital Services for Leagues and Federations

World Padel Tour among first customers as LaLiga Tech launches digital services ...

04/10/2021

NBC's Molly Solomon to Receive NY Festivals Lifetime Achievement Award on Oct. 12 (Virtually That Is!)

NBC's Molly Solomon to Receive NY Festivals Lifetime Achievement Award on Oc...

04/10/2021

Flowics Provides Real-Time Graphics for BCC Live's Stream of Mainova IRONMAN

Flowics Provides Real-Time Graphics for BCC Live's Stream of Mainova IRONMAN Data-infused elements keep fans updated throughout the eight-hour event By Kri...

04/10/2021

Ryder Cup Reflections: World Feed Efforts Hint at New Ways to Collaborate

Ryder Cup Reflections: World Feed Efforts Hint at New Ways to Collaborate By Ken Kerschbaumer, Editorial Director Monday, October 4, 2021 - 3:07 pm Print T...

01/10/2021

ESL Expands and Upgrades Polish Production Facility for REMI With Blackmagic Design

ESL expands and upgrades Polish production facility for REMI with Blackmagic Des...

01/10/2021

Audio Teams Look Forward to a Naturally Noisy NFL Season

Audio Teams Look Forward to a Naturally Noisy NFL Season Both A1s and management welcome a return to the old' normal By Dan Daley, Audio Editor Friday, ...

01/10/2021

London Marathon: BBC Sport Looks Back in Order to Move Forward With Coverage of One of Sport's Most Joyful Events

London Marathon: BBC Sport looks back in order to move forward with coverage of ...

01/10/2021

New Book Looks Back at How The NFL Today on CBS Changed Sports Broadcasting

New Book Looks Back at How The NFL Today on CBS Changed Sports Broadcasting Nine Sports Broadcasting Hall of Famers have been connected to the show By Ken Kers...