Replay Technologies freeD 60 Replay Brings New Perspective to US Open Tennis

A new technology is turning heads at the Billie Jean King USTA Tennis Center this year: Replay Technologies' freeD 360-degree-style replay system. Installed at Arthur Ashe Stadium and making its Grand Slam debut on ESPN, the freeD (Free Dimensional Video) system relies on 28 5K cameras, each capturing a 2D image, and a custom algorithm to process the images into the 3D scene. The operator can freeze the action, then turn the perspective to show any angle in the replay.

Replay Technologies' Dror Elbaz

The system itself is very robust, says Dror Elbaz, VP, engineering, Operations Business Unit, Replay Technologies. [It] can be implemented in various fields of sports with minor designated adjustments, derived from the specific sport characteristics (e.g., expected speed of the ball, physical dimensions of the field, indoor/outdoor stadium, etc.).

In the past four years, he continues, freeD systems have already been installed in baseball, basketball, soccer, football, tennis, athletics, and [other] fields. As for tennis in general and Arthur Ashe Stadium in particular, the environment is very convenient in terms of angles and field (background) color, which eases the algorithm's work of extracting the players from the background.

How It Works

The 28-camera complement comprises two wide-angles to see the entire stadium, six cameras with high-magnification zoom, and traditional-coverage cameras. Each camera is connected via optic fibers to a dedicated server custom-designed by Replay Technologies (the server room is adjacent to the freeD control room at Ashe).

Camera positions are defined by two main guidelines: first, to have an optimum area of coverage for the entire court; second, to have an optimal angle enabling coverage of a play from each angle. Elbaz describes it as 28 cameras essentially working together as a 3D scanner to pinpoint any voxel (a value on a regular grid in 3D space) and allow unlimited camera movements from any angle in space.

He explains that the 2D images from the cameras are processed together and the entire space is reconstructed by an algorithm into 3D. After the 3D scene is rendered, he says, the system generates seamless and unlimited virtual camera movement.

The images from the 28 freeD cameras at Arthur Ashe Stadium

Inside the Replay Technologies control room, a pilot and a navigator control the system itself. The pilot works with a producer to define where the camera perspective is to move in the virtual space and determine the exact clip to provide to ESPN. Meanwhile, the navigator works intimately with each aspect of the system to verify that everything is running smoothly in terms of the camera and servers (which are responsible for both processing the images and saving the data generated by the cameras). At the US Open, the Replay Technologies team also has the ability to manually wipe out spectators blocking various cameras in a shot.

Generally, when everything is working automatically, clips are generated in 90-120 seconds, says Elbaz. But here at Ashe, most of the cameras are blocked by spectators, so manual involvement is required to clear the spectators prior to clip rendering. In general, it takes about five minutes.

The clips, he continues, are being generated based on guidelines (such as specific frame to freeze, requested perspective, clip duration and high level path for the virtual camera) by the ESPN producer. The mutual work and coordination are very good, as one side knows the game and desired perspective while the other side knows the way to provide it with the system.

Truly a Multisport Affair

freeD has been a staple at AT&T Stadium in Arlington, TX, used for NBC Sunday Night Football telecasts featuring the Cowboys over the past two seasons, as well as at the 2014 NCAA Men's Basketball Final Four. Although it has worked tennis at the BNP Paribas Open from Indian Wells, CA, this marks freeD's first-ever appearance on any network at a tennis Slam.

From above, a freeD corner camera between stadium lights

Replay Technologies has also deployed freeD for the New York Yankees at Yankee Stadium for YES Network telecasts, the Los Angeles Dodgers at Dodger Stadium for SportsNet LA telecasts, and NBA's All-Star Weekend 2014 at Smoothie King Center in New Orleans for Turner Sports. In addition, the Baltimore Ravens have installed the system at M&T Bank Stadium for the 2015 NFL season.

One of the [company's] main guidelines is pursuing for constant improvements in quality and performance, Elbaz points out. The improvements [are] derived from intensive R&D work, which relies on the system's install base over the different sports.

Each installation, he continues, is being evaluated based on customer demands, the company's knowledge base, and previous similar installations. In Ashe Stadium, the system works with 28 cameras, [but] there are fields in which more cameras are required (e.g., soccer) to cover the entire field or focus on main areas of interest.

The freeD video format, which is a three-dimensional representation of reality, serves multiple platforms and services.

Broadcasters and teams are using it to enhance the fan experience in venues and at home. The next step is to allow people at home to view interactive content from the game. Using a laptop, cellphone, or VR (virtual-realty) headset, fans can create personalized content -how they want to see the game, even from inside the action - and share it with friends.

Viewing freeD content on VR allows fans to place themselves on the field itself. Although it might sound like science fiction, Elbaz notes, several collaborations between Replay and VR-hardware companies are enabling a new level of experience.

LINK:	http://sportsvideo.org/main/blog/2015/09/replay-technologies-freed-360...
	See more stories from svg

Most recent headlines