What do you do when you've been planning a live interactive theatrical concert all semester, only to have the entire world moved online and all live events cancelled due to the COVID-19 pandemic? If you're Berklee's Electronic Production and Design (EPD) program, you quickly pivot and find innovative ways to use technology to create a virtual performance experience.
The project initially got rolling last fall with Grammy-nominated visiting artist Nona Hendryx, who has been collaborating with EPD for the past several years on projects ranging from a dance piece with producer Hank Shocklee to an interactive performance with alumnus and Living Colour drummer Will Calhoun 86. Hendryx worked with faculty member Lori Landay's Digital Narrative Theory and Practice classes to develop an interactive script, not only for the multimedia element, but also for the staging of the event.
In the spring semester, Akito van Troyer, assistant professor of EPD, taught the department's Special Projects Seminar. Van Troyer, who had previously worked at the MIT Media Lab, was charged with developing a technology for a live interactive performance that would also incorporate the use of virtual reality (VR) into a performance environment.
The project coalesced around the Mi.Mu performance gloves developed by artist and producer Imogen Heap, who visited Berklee both in Boston and at its campus in Valencia, Spain, where she received an honorary doctorate in 2019. Mi.Mu is the gold standard for controlling visual and sound using gloves, said Michael Bierylo, chair of the Electronic Production and Design Department. We got two pairs of the gloves in February and were planning to use them in a theatrical piece with music, with students doing the sound design and visuals-a lot of which would be controlled in real time with the Mi.Mu gloves.
When the shift to remote learning happened in March, the project quickly evolved into a narrative video game called Dream Machine. With the help of consultants, and under the supervision of van Troyer and Landay, students developed software for an online multiplayer environment in Unity, a game engine platform. We had always thought there would be a game component, but that the performance would happen in some representation of the game environment, said Landay. Then the game environment became the whole environment and the questions became: What are the game mechanics? What are the performance mechanics? And who is the player?
For students and musicians to be doing this, coding in an undergraduate program that's not a computer science program, was pretty remarkable.
-Michael Bierylo, chair of Electronic Production and Design
After a six-week sprint to develop Dream Machine-during which Hendryx often said, we were building the airplane as we were flying -50 people played the game simultaneously, navigating through the game world as avatars as part of the performance experience hosted on Twitch.tv in late May. For students and musicians to be doing this, coding in an undergraduate program that's not a computer science program, was pretty remarkable, said Bierylo.
The EPD Department also collaborated on the project with accomplished Dutch multimedia artist Chagall, who performs frequently using the Mi.Mu gloves. Chagall had spent a week on campus as a visiting artist working with students just before the shutdown began. A return visit was planned for later in the semester, but when that became impossible, she motion-captured herself performing some of the songs she would have done in person, and the students created videos to her performance.
Are You an Electronic Musician?
Apply to Berklee with an electronic digital instrument (EDI). Learn how to audition.
I do use a lot of technology; there are the gloves, sensors on my body, and visuals on the screen, said Chagall. The reason is because I feel there is a need for electronic music to be performed in a way that allows the audience to connect emotionally with what's going on inside the computer and with the music itself. So, for me, bringing people together in live music is really important, and that's exactly the thing we're all not allowed to do right now.
Bierylo sees the Dream Machine project as a prototype going forward. This is a way that we can work as artists and distribute our work in a global way, he said. We can have it be entertaining, we can have it be interactive, we can have it be all of the things that we originally thought of for a live performance. This was a living laboratory for one kind of future for performance.
Watch a video representation of Dream Machine featuring Chagall:
Video of Dream Machine Interactive Virtual Performance
Related categories:
News
Related Reading
Daedelus Joins Berklee Electronic Production and Design Department The acclaimed artist, DJ, and producer has worked with Diplo, Death Cab for Cutie, Flying Lotus, and more.
Overwatch Composer Adam Burgess Charts a Path from Berklee to Blizzard Entertainment The 2014 graduate breaks down some of the epic music heard by 50 million gamers worldwide, and shares career tips for aspiring game composers.
30 under 30 To celebrate Berklee Todays 30th anniversary, we are highlighting a selection of alumni who were born after that inaugural issue was printed.
More from Berklee Now
Most Popular
Alumna Kirbys Viral Video Helped Bring Down Aunt Jemima Brand The video has been viewed 4.7 million times and has been credited with pushing Quaker Oats to scrap the brand of breakfast products.
Latest
A Time to Be Heard: What Performers Can Expect This Fall The Professional Performance Division offers guidance on what ensembles and private instruction will look like this upcoming semester.
What to Expect of the Online Experience This Fall Regardless
Most recent headlines
11/12/2025
Dalet, a leading provider of cloud-native, end-to-end media workflow solutions, ...
01/12/2025
L3Harris and PentenAmio formalise their teaming agreement at MilCIS 2025, streng...
01/12/2025
Artemis II is NASA's first crewed flight test of the Space Launch System rocket and Orion spacecraft. The crew, from left: Commander Reid Wiseman, Pilot Vic...
01/12/2025
IRVINE, Calif. Wooden Camera has introduced its new Accessory Collection for the Canon EOS C50. The new lineup includes a low-profile, gimbal-ready cage, expand...
01/12/2025
WASHINGTON The Federal Communications Commission has released a tentative agenda for its Dec. 18 Open Commission Meeting that will include a vote on a report an...
01/12/2025
In most years, a graph of annual local TV ad spending is about as predictable as an electrocardiogram of a reasonably healthy patient in a doctor's office. ...
01/12/2025
Many industries have seen big-ticket hardware turn into software. Switchers, though, demand a combination of real-time performance and sheer bandwidth that has ...
01/12/2025
GENEVA Shanghai will host the next quadrennial Radiocommunication Assembly (RA-27) and World Radiocommunication Conference (WRC-27), Oct. 11-Nov. 12, 2027. This...
01/12/2025
NEW YORK Just in time for Giving Tuesday tomorrow (Dec. 2), the Broadcasters Foundation of America is seeking out donations to help television and radio industr...
01/12/2025
STOCKHOLM, Sweden Net Insight CEO Crister Fritzson has informed the company's board that he will retire from the video transport and media cloud technology ...
01/12/2025
01 Dec 2025
Kyivstar and Ukrainian Ministry of Digital Transformation Select Go...
01/12/2025
The prequel to the global hit Sky Original mob crime saga, Gomorrah' is a s...
01/12/2025
Featuring Florence Welch, Red Hot Chilli Peppers' Flea, designer Bella Freud...
01/12/2025
Back to All News
A Down to Earth, All Too Relatable Hero: Cashero' Teaser ...
01/12/2025
In the rush to deliver content to every screen, many broadcasters overlook one of the most crucial steps in the workflow: format and frame rate conversion. Get ...
01/12/2025
Fox Corporation Chief Financial Officer Steve Tomsic to Participate in Upcoming ...
01/12/2025
Arvato Systems' Virtual Private Cloud (VPC) Receives BSI C5 Certification (T...
01/12/2025
Summary This short video gives you an summary of the changes in under two minutes.
--...
01/12/2025
As the festive season approaches, RT Supporting the Arts is proud to showcase a...
01/12/2025
Researchers worldwide rely on open-source technologies as the foundation of their work. To equip the community with the latest advancements in digital and physi...
01/12/2025
Festive specials of Christmas in Kilmainham presented by Marty Whelan, High Road Low Road, Callan Kicks the Year and Keys to My Life
Ring in the New Year with ...
01/12/2025
Architect and television presenter Hugh Wallace, best known to RT audiences as a long-serving judge on Home of the Year, has died at the age of 68.
In a state...
28/11/2025
Nadia Fall attends the 2025 Sundance Film Festival premiere of Brides at the Egyptian Theatre on January 24, 2025, in Park City, Utah. (Photo by Donyale West/...
28/11/2025
It's easy to ignore those little red update available badges. But when it ...
28/11/2025
WASHINGTON Federal Communications Commission has released a tentative agenda for the December Open Commission Meeting scheduled for Thursday, December 18, 2025 ...
28/11/2025
The Professional Fighters League is looking to super-serve fans of mixed martial...
28/11/2025
Fubo has released in beta on select Roku devices a new feature that lets users display up to four simultaneous streams at once....
28/11/2025
The WNBA playoffs and Week 4 of the NFL regular season highlight the list of live sports events airing on television this weekend....
28/11/2025
The 32nd class of honorees to the B+C Hall of Fame took to the stage at New York's Ziegfeld Ballroom on September 26 for a gala induction event. Click below...
28/11/2025
We hold in our hands the very last Next Text for Next TV, the weekly back-and-fo...
28/11/2025
DirecTV said it made a deal with EchoStar to buy EchoStar's video businesses, including satellite-TV provider Dish TV and virtual MVPD Sling TV, for $1 plus...
28/11/2025
The Broadcasting+Cable Hall of Fame, the premier industry event paying tribute to the influencers, innovators and shining lights of broadcast, cable and streami...
28/11/2025
Friday 28 November 2025
Sky Sports x Slawn drop limited-edition football jersey...
28/11/2025
Rohde & Schwarz shows resilience in a challenging environment, revenue exceeds t...
28/11/2025
Unwrapped: The Toy Show Appeal - airing this Sunday on RT One and RT Player- s...
27/11/2025
LONDON Vizrt has added several AI-driven advanced features offering improved speed, intelligence and accuracy in the newest version of its media asset managemen...
27/11/2025
Prime Video has launched AI-powered video season recaps in a beta version for select English-language Prime Original series in the U.S., a move Amazon is callin...
27/11/2025
Back to All News
Netflix's Raat Akeli Hai: The Bansal Murders Marks a Grand...
27/11/2025
27 Nov 2025
GSMA brings M360 Eurasia 2026 to Samarkand in partnership with VEON...
27/11/2025
Tahar Rahim and Izuka Hoyle star in the gripping six-part Sky Original from Acad...
27/11/2025
Thursday 27 November 2025
Sky Arts Reveals the Nation's Greatest Basslines - and Queen Reign Supreme
The UK's most iconic basslines have been revealed...
27/11/2025
Back to All News
Stranger Things 5': Prepare for One Last Adventure With O...
27/11/2025
The media industry has a paradox at its core. It's an industry built on light, color and imagination, yet behind the scenes, it's powered by one of the ...
27/11/2025
Rating reflects rating progress across areas including policies, diversity & inclusion, health & safety and Net Zero leadership
Winchester, UK, 27 November 202...
27/11/2025
What are the industry standards for Retail Media? Kathryn explains that certification is based on the IAB Europe Retail Media Measurement Standards and the IAB ...
27/11/2025
World champion boxer and Irish sporting icon Katie Taylor will be in studio this...
27/11/2025
Roblox, one of the world's most popular online gaming platforms for primary ...
27/11/2025
Black Friday is leveling up. Get ready to score one of the biggest deals of the season - 50% off the first three months of a new GeForce NOW Ultimate membership...
26/11/2025
SVG Sit-Down: Prime Video EP Mike Muriano Previews Massive Black Friday Slate Fe...
What do you do when you've been planning a live interactive theatrical concert all semester, only to have the entire world moved online and all live events cancelled due to the COVID-19 pandemic? If you're Berklee's Electronic Production and Design (EPD) program, you quickly pivot and find innovative ways to use technology to create a virtual performance experience.
