Mixing clever: Salsa Sound embraces the power of AI in MIXaiR 2.0 By Michael Burns
Monday, September 20, 2021 - 10:41
Print This Story
Salsa Sound co-founder and CEO, Rob Oldfield
Regular readers of SVG Europe will be well aware of how innovative technology companies working in sports broadcasting can be, but MIXaiR from Salsa Sound looks likely to set a new bar for live audio and the craft of sound mixing.
When the pandemic caused football games to be played in silent stadiums Salsa Sound came to the rescue of broadcasters with its vCrowd real time virtual crowd atmosphere system, which rightly won plaudits. However, MIXaiR is based on technology that the company has been working on for several years, through academia, R&D, beta versions and a soft launch of the system a year ago.
Our company was founded in 2017, but prior to that we were part of the University of Salford, where we were working on a lot of innovative audio techniques, says co-founder and CEO, Rob Oldfield. In particular we were looking at how to leverage artificial intelligence (AI) to recognise sound events and then create the best possible mix, not based on tracking but just based on what is actually [captured by] those microphones.
Fast forward to the future, next month in fact, and the new version of MIXaiR, an AI-driven system that automatically creates and enhances audio mixes for live broadcast, is set to be released.
MIXaiR 2.0 is jam packed with new features and a much more intuitive, easy to use interface, says Oldfield. The idea for v2.0 is you put all your microphones feeds in and then MIXaiR creates different submixes. So you might have a crowd, a commentary, or a pitch mix or aux in, and MIXaiR will automatically balance the levels between them, apply some processing, then create the best possible mix out of it without any human interaction, other than setting it up in the first place.
Hard tackle
According to Oldfield, the hardest mix, by a country mile, is the pitch mix. Historically, it's such a dynamic process by engineers and difficult to replicate, he explains. To make sure you've always got the nearest microphone active in the mix at any one point, requires constant attention and raising and lowering of faders. It's a really clever balancing act.
Rather like the players whose kicks it tracks, MIXaiR's AI has gone through an extensive training regime. Using machine learning technology, Salsa Sound has been training MIXaiR with many hours of content, microphone recordings and mixes across leagues.
People don't realise how hard it is to mix really well. That's why we've gone through pains to get an AI solution that takes the strain from some of the really difficult grunt work of mixing
We know what makes a great mix, says Oldfield. The AI is constantly analysing all of the sounds it is hearing. We tell it what to make of these sound events and it learns what's a kick, what's a whistle, what's a ball bounce, so that when it sees or hears sound in the wild, it can make an intelligent decision based on it.
Oldfield says MIXaiR delivers an even pitch mix, without any slightly awkward transitions and unbalanced crowd noise.
Our approach is to have the AI listen to those live microphone feeds, and when it hears a sound that's interesting like a kick, whistle, ball bounce or hitting the crossbar, you name it it will automatically add that microphone feed into the mix. It tracks the game around, always choosing the microphone that's closest to the action, and [performs] a seamless transition between them.
People don't realise how hard it is to mix really well, he adds. That's why we've gone through pains to get an AI solution that takes the strain from some of the really difficult grunt work of mixing. It can enable these guys to explore creative avenues and to create more innovative content. When you're not completely locked into your screen trying to create the best mix, you can actually lean in to MIXaiR a little bit more, let that create the stems and then you can have a bit of cognitive space to craft a mix, rather than chasing a mix.
Salsa Sound's MIXaiR GUI
The commentary and crowd noise are dynamically processed to provide mix components. These sub mixes are basically ingredients that you can throw into the output mixes, explains Oldfield. Then it's just a drag and drop process. You can create whatever output mixes you want, as many as you want, in different formats. You can have international sound, a French mix, a German mix, a Spanish mix, or whatever, and within that you can have different flavours: the stereo mix, the 5.1 mix, the mono mix. The only limitation is the number of channels that you've got available on your output board. You can also apply VST plug-ins to add your own compression or EQ or effects to the submixes, so it becomes a creative tool, allowing you to craft your mixes.
Importantly, [all mixes] are loudness normalised, he adds. If you're streaming on YouTube, or your own video on demand platform, or when it is for broadcast, they all have different loudness requirements. So within MIXaiR, you just decide which requirement you want, click on it and it will ensure that the mix adheres to those standards, so you don't end up with a fine from the regulator for making too much noise, or making not enough noise.
MIXaiR users also have an option to enhance the pitch mix with pre-recorded sounds. When somebody whacks the ball in the middle of the park, the [sound supervisor] can't even hear it because it's a long way away from the microphones, says Oldfield. However the AI can pick up a kick in the middle of the pitch. The algorithm can detect a kick, go to a bank of pre-recorded kicks and pick out the most appropriate one, and then
Most recent headlines
04/08/2024
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet, a leading technology and service provider for media-rich organizations, a...
28/04/2024
Mediahaus delivers the first SRT live-streaming sports production over 5G with U...
23/04/2024
Psychology can play an important role in helping us understand what audiences respond to (Dr Jon Roozenbeek, Cambridge Social Decision-Making Lab)
For newsroom...
23/04/2024
IN ADDITION TO FICTION AND DOCUMENTARY FEATURES, THE SELECTION INCLUDES:
PRO...
23/04/2024
Today, we announced our first quarter 2024 earnings. In a solid start to the year, Spotify reported
Monthly Active Users grew 19% Y/Y to 615 million.
Subscri...
23/04/2024
Idag publicerar Spotify sina finansiella resultat f r f rsta kvartalet 2024. ret har inletts stabilt:
M natliga aktiva anv ndare kade med 19 % j mf rt med f...
23/04/2024
In the last decade, as K-Pop cemented its status as a global phenomenon, Spotify has helped fans around the world keep their fingers on the pulse at every step ...
23/04/2024
Social entrepreneur Hedayat Osyan wins Les Murray Award for Refugee Recognition
23 April, 2024
Media releases
The prestigious award is now in its third yea...
23/04/2024
LONDON IBC is now accepting nominations for its IBC2024 Innovation Awards, which honor achievements in technological innovation and social impact media & entert...
23/04/2024
WASHINGTON, D.C. In response to the introduction of the Broadcast VOICES Act that seeks to encourage more diversity in the ownership of TV and radio stations, t...
23/04/2024
MIAMI, Fla. As part of a push to expand its FAST channel and streaming operations, Hemisphere Media Group has launched its free ad supported television (FAST) c...
23/04/2024
LOS ANGELES DirecTV is adding seven new channels to its rapidly expanding streaming lineup through new licensing agreements with Cineverse Corp. and Scripps Net...
23/04/2024
Entitled Our Creative Future, the manifesto calls for radical new action' t...
23/04/2024
The company has secured investment from private equity firm PAI Partners, which also holds a stake in outside broadcasting company EMG
By Jenny Priestley
Pub...
23/04/2024
Cameron also said he recognised Marlow as a potential UK and European base for his new company, LIGHTSTORM 3D
By Matthew Corrigan
Published: April 23, 2024 ...
23/04/2024
The five categories cover Content Creation, Content Distribution, Content Everywhere, Social Impact and Environment & Sustainability
By Matthew Corrigan
Publ...
23/04/2024
He played a central role in securing key broadcast rights, including the FIFA World Cup 2018 and 2022
By Matthew Corrigan
Published: April 23, 2024
He pla...
23/04/2024
IBC is preparing to honour the highest achievements in technological innovation and social impact with the launch of its prestigious IBC2024 Innovation Awards, ...
23/04/2024
BREA, Calif. Bolin Technology is integrating Network Device Interface (NDI) across its product line, bringing support for the latest NDI technologies to its PTZ...
23/04/2024
WYOMISSING, Pa. PENN Entertainment has announced that longtime Disney and ESPN executive Aaron LaBerge has been named chief technology officer effective July 1,...
23/04/2024
In response to an FCC ruling in March proposing a $1.2 million fine against Nexstar and a ruling that Mission Broadcasting sell WPIX, Nexstar has filled a respo...
23/04/2024
Sony has announced new details about how the National Hot Rod Association (NHRA) is using Sony's Ci Media Cloud platform to streamline its workflows and spe...
23/04/2024
Esports' rapid rise to popularity can be attributed to many factors, from technological advancements that have enhanced the quality of modern gaming experie...
23/04/2024
OpenDrives, Inc., the global provider of software-defined media workflow solutions and data management capabilities, has been selected as a winner in the 2024 N...
23/04/2024
BCNEXXT is pleased to announce that Vpe has won the 2024 NAB Show Product of the Year Award in the Asset Management, Automation and Playout category. The offici...
23/04/2024
Gravity Media, a world leading global provider of complex live creative production and media services, today outlined the broadcast technology and television pr...
23/04/2024
Scality, a global leader in cyber-resilient storage for the AI era, today announced the appointment of Peter Brennan as Chief Executive Officer (CEO) of Scality...
23/04/2024
Blackmagic Design today announced ATEM 1 M/E Constellation 4K and ATEM 2 M/E Constellation 4K switchers, two new Ultra HD models of the ATEM Constellation famil...
23/04/2024
Blackmagic Design today announced Blackmagic Videohub 120x120 12G, the newest model in the Videohub 12G router family. The massive Blackmagic Videohub 120x120 1...
23/04/2024
Blackmagic Design today announced new lower prices for Blackmagic Video Assist 3G and 12G models. Customers can save US$200 on all models. These lower prices ma...
23/04/2024
Adder Technology continues to shine brightly, with its groundbreaking product, the ADDERLink INFINITY 3000 (ALIF3000), once again taking center stage. Hot on t...
23/04/2024
Pliant Technologies' CRP-C12 Compact Radio Pack is an Audio Production, Processing and Networking category winner in the 2024 NAB Show Product of the Year A...
23/04/2024
SipRadius, the specialists in live media transport and transcoding, will showcase their world-leading implementation of AV1 and JPEG-XS over transport streams a...
23/04/2024
Composer Rafael Krux has added a Prism Sound Dream ADA-128 modular conversion system to his production studio where he creates music for commercials, trailers, ...
23/04/2024
PMC proudly provides the ultimate large-scale Dolby Atmos audio system for the legendary musician Steven Wilson at the High End, Munich. He will share his exten...
23/04/2024
Particles emitting from a 3D rotating shape's edge
Graham Quince April 22, 2024
0 Comments
In Fixin' FX, people send me their problematic proj...
23/04/2024
Hollyland Technology Announces a New Line of Video Transmitters: Introducing the...
23/04/2024
Charlotte Deleste, anchor at WISC Madison, Wisconsin, is leaving the station. Her final day on the air is April 24. Deleste has spent 18 years there....
23/04/2024
Top WNBA draft choice Caitlin Clark will appear on local broadcast for 17 games next season in a deal between Tegna and the Indiana Fever....
23/04/2024
The merger will provide vertical integration potential and enhanced product offerings to capture regional and international opportunities for revenue growth and...
23/04/2024
Abu Dhabi, 23 April, 2024 : Partnership delivers greater cybersecurity to Thura...
23/04/2024
Soccer's Appeal to Younger Audiences: Inside the Gamification of European Fo...
23/04/2024
With NBA and NHL Playoffs in Full Swing, Never a Dull Moment for TNT Sports Prod...
23/04/2024
NAB 2024: AT&T Business's Scott Beckett Details Transmission Firepower With ...
23/04/2024
NAB 2024: Manufacturers focus on cinematic look for live productions By George Bevir
Tuesday, April 23, 2024 - 10:40
Print This Story
With live sports pro...
23/04/2024
Save the Date: SVG Sports Content Management Forum Returns to NYC on July 24 By SVG Staff
Tuesday, April 23, 2024 - 4:05 pm
Print This Story | Subscribe
...
23/04/2024
Published On: April 23rd, 2024
data-orig-src=https://www.sgo.es/wp-content/upl...
23/04/2024
Rohde & Schwarz introduces new R&S SMB100B microwave signal generator for analog...
Mixing clever: Salsa Sound embraces the power of AI in MIXaiR 2.0 By Michael Burns 
