What do you do when you've been planning a live interactive theatrical concert all semester, only to have the entire world moved online and all live events cancelled due to the COVID-19 pandemic? If you're Berklee's Electronic Production and Design (EPD) program, you quickly pivot and find innovative ways to use technology to create a virtual performance experience.
The project initially got rolling last fall with Grammy-nominated visiting artist Nona Hendryx, who has been collaborating with EPD for the past several years on projects ranging from a dance piece with producer Hank Shocklee to an interactive performance with alumnus and Living Colour drummer Will Calhoun 86. Hendryx worked with faculty member Lori Landay's Digital Narrative Theory and Practice classes to develop an interactive script, not only for the multimedia element, but also for the staging of the event.
In the spring semester, Akito van Troyer, assistant professor of EPD, taught the department's Special Projects Seminar. Van Troyer, who had previously worked at the MIT Media Lab, was charged with developing a technology for a live interactive performance that would also incorporate the use of virtual reality (VR) into a performance environment.
The project coalesced around the Mi.Mu performance gloves developed by artist and producer Imogen Heap, who visited Berklee both in Boston and at its campus in Valencia, Spain, where she received an honorary doctorate in 2019. Mi.Mu is the gold standard for controlling visual and sound using gloves, said Michael Bierylo, chair of the Electronic Production and Design Department. We got two pairs of the gloves in February and were planning to use them in a theatrical piece with music, with students doing the sound design and visuals-a lot of which would be controlled in real time with the Mi.Mu gloves.
When the shift to remote learning happened in March, the project quickly evolved into a narrative video game called Dream Machine. With the help of consultants, and under the supervision of van Troyer and Landay, students developed software for an online multiplayer environment in Unity, a game engine platform. We had always thought there would be a game component, but that the performance would happen in some representation of the game environment, said Landay. Then the game environment became the whole environment and the questions became: What are the game mechanics? What are the performance mechanics? And who is the player?
For students and musicians to be doing this, coding in an undergraduate program that's not a computer science program, was pretty remarkable.
-Michael Bierylo, chair of Electronic Production and Design
After a six-week sprint to develop Dream Machine-during which Hendryx often said, we were building the airplane as we were flying -50 people played the game simultaneously, navigating through the game world as avatars as part of the performance experience hosted on Twitch.tv in late May. For students and musicians to be doing this, coding in an undergraduate program that's not a computer science program, was pretty remarkable, said Bierylo.
The EPD Department also collaborated on the project with accomplished Dutch multimedia artist Chagall, who performs frequently using the Mi.Mu gloves. Chagall had spent a week on campus as a visiting artist working with students just before the shutdown began. A return visit was planned for later in the semester, but when that became impossible, she motion-captured herself performing some of the songs she would have done in person, and the students created videos to her performance.
Are You an Electronic Musician?
Apply to Berklee with an electronic digital instrument (EDI). Learn how to audition.
I do use a lot of technology; there are the gloves, sensors on my body, and visuals on the screen, said Chagall. The reason is because I feel there is a need for electronic music to be performed in a way that allows the audience to connect emotionally with what's going on inside the computer and with the music itself. So, for me, bringing people together in live music is really important, and that's exactly the thing we're all not allowed to do right now.
Bierylo sees the Dream Machine project as a prototype going forward. This is a way that we can work as artists and distribute our work in a global way, he said. We can have it be entertaining, we can have it be interactive, we can have it be all of the things that we originally thought of for a live performance. This was a living laboratory for one kind of future for performance.
Watch a video representation of Dream Machine featuring Chagall:
Video of Dream Machine Interactive Virtual Performance
Related categories:
News
Related Reading
Daedelus Joins Berklee Electronic Production and Design Department The acclaimed artist, DJ, and producer has worked with Diplo, Death Cab for Cutie, Flying Lotus, and more.
Overwatch Composer Adam Burgess Charts a Path from Berklee to Blizzard Entertainment The 2014 graduate breaks down some of the epic music heard by 50 million gamers worldwide, and shares career tips for aspiring game composers.
30 under 30 To celebrate Berklee Todays 30th anniversary, we are highlighting a selection of alumni who were born after that inaugural issue was printed.
More from Berklee Now
Most Popular
Alumna Kirbys Viral Video Helped Bring Down Aunt Jemima Brand The video has been viewed 4.7 million times and has been credited with pushing Quaker Oats to scrap the brand of breakfast products.
Latest
A Time to Be Heard: What Performers Can Expect This Fall The Professional Performance Division offers guidance on what ensembles and private instruction will look like this upcoming semester.
What to Expect of the Online Experience This Fall Regardless
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
15/06/2026
Detach from Direct-Attached: How Remote Editing with EVO Keeps Creative Teams Mo...
14/06/2026
Library captures 1960s R&B/pop drum sound
Following on from their recent wave of plug-in effects, Iconic Instruments have just launched an all-new virtual d...
14/06/2026
HBO Comedy Rooster Shot with URSA Cine 17K 65
Brie Clayton June 14, 2026
0 Comments
Large format brings viewers intimately close to characters.
Black...
13/06/2026
Latest expansion pack includes 252 presets
Devious Machines have recently introduced another expansion for their powerful multi-effects plug-in, Infiltrator...
13/06/2026
Create custom DAW/plug-in controllers using prompts
MetaGrid have recently introduced an all-new AI Builder function to their touchscreen-based control surf...
13/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
13/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
YES Network and The Gotham Sports App will air seven Athletes Unlimited Softball...
12/06/2026
The United Football League will host its FAST Innovation Suite at the 2026 United Bowl presented by Credit One Bank on Saturday, June 13 at 3:00 p.m. ET at Audi...
12/06/2026
PTZOptics and LayerJot will present live demonstrations at InfoComm 2026 showing how natural-language AI prompting, robotic camera control, and on-device comput...
12/06/2026
MultiDyne Video and Fiber Optic Systems will exhibit at InfoComm 2026, featuring...
12/06/2026
Ateme has announced that Eurovision Services is using Ateme's software-based frame-rate conversion technology for international live event workflows. The de...
12/06/2026
Bitmovin and Simplestream have announced a partnership with Xperi to simplify the launch of OTT streaming services on TiVo OS smart TVs and devices. The collabo...
12/06/2026
Net Insight has announced that a multinational technology company is deploying a...
12/06/2026
MLB Players Inc., the business arm of the MLB Players Association, has announced a partnership with Athletes First to develop and sell brand partnerships across...
12/06/2026
Guntermann and Drunck (G&D) and VuWall have announced the CommandKeyboard-Advanc...
12/06/2026
Comcast Smart Solutions announces a new smart technology deployment with Major L...
12/06/2026
Elevation Worship completed the initial leg of its Elevation Nights 2026 tour ...
12/06/2026
AJA Video Systems has announced KONA IP25 support for Colorfront Transkoder and ...
12/06/2026
Audinate Group Limited (ASX: AD8) will exhibit at InfoComm 2026 (Booth C7321, Ce...
12/06/2026
Pac-12 Commissioner Teresa Gould has announced the appointment of Scott Adametz as Chief Technology Officer. The Pac-12 describes the hire as the first CTO appo...
12/06/2026
Grass Valley has announced AMPP Edge Live, a production system combining Grass Valley hardware, NVIDIA Blackwell GPU acceleration, and AMPP OS in a single platf...
12/06/2026
At one time a trailblazer with the launch of the Longhorn Network, the Universit...
12/06/2026
Ratings Roundup is a rundown of recent rating news and is derived from press rel...
12/06/2026
Chyron has announced PAINT 10.4, an update to its illustrated replay and sports ...
12/06/2026
SVP, Production, Mark Gross: With the new schedule, with not having every Sunday night, it has given us an opportunity to take a step back and reimagine what o...
12/06/2026
For Televisa Technical Engineering Manager Roberto N nez Ibarra and the small team of 12 technicians and two production personnel at the IBC things are already ...
12/06/2026
Simple Steps to Better Acoustics - Taming The Small Room
Most of us mix in spare rooms and small spaces, where the acoustics fight us at every turn. At Gear...
12/06/2026
Latest addition expands vintage-inspired effects palette
Meris' Ottobit pedal range draws its inspiration from vintage gaming consoles, and the latest a...
12/06/2026
Soundbox-based chamber strings series expanded
Sonora Cinematic have just announced the launch of the second instalment in their Soundbox-based chamber stri...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
AJA Announces KONA IP25 Integration with Colorfront Software
Brie Clayton June 12, 2026
0 Comments
Collaboration enables uncompressed SMPTE ST 2110 I/O ...
12/06/2026
URSA Cine 12K LF Used to Create Visuals for STUTS' K-Arena Concert
Brie Clayton June 12, 2026
0 Comments
Organic visuals projected on a giant scre...
12/06/2026
MTI FILM Acquires Mango New Edit, Expanding its Global Post-Production Services ...
12/06/2026
AI Point Tracking Speeds Up Complex VFX Tracks in Mocha Pro
Jessie Electa Petrov June 12, 2026
0 Comments
The 2026.5 release adds automatic point trac...
12/06/2026
Bitmovin, a provider of video streaming solutions, has partnered with Simplestream, a provider of OTT and broadcast solutions, and technology provider Xperi, to...
12/06/2026
Leostream Corporation, creator of the world-leading Leostream Remote Desktop Access Platform, today announced Jigsaw24, a leading B2B IT solutions provider wit...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
How Aussie indie games and screen are levelling up with IP 11 June 2026
Ari Harrison, Pro Jank Footy
Head of Games Joey Egger and Ari Harrison of Umbrella sha...
12/06/2026
AgentPerf from Artificial Analysis, the industry's first agentic AI benchmark, gives developers, enterprises and infrastructure providers a clear way to com...
What do you do when you've been planning a live interactive theatrical concert all semester, only to have the entire world moved online and all live events cancelled due to the COVID-19 pandemic? If you're Berklee's Electronic Production and Design (EPD) program, you quickly pivot and find innovative ways to use technology to create a virtual performance experience.
