Live From NBA All-Star Weekend: Volumetric Video Poised To Drive Quality VR Sports Experiences, Player Analytics

Live From NBA All-Star Weekend: Volumetric Video Poised To Drive Quality VR Sports Experiences, Player Analytics Demo at Tech Summit allows guests to have their own the VR experience By Ken Kerschbaumer, Editorial Director

Sunday, February 19, 2023 - 11:08 am

Print This Story | Subscribe

Story Highlights

NBA All-Star Game Weekend always has a heavy focus on technology when it comes to events and activities. For example, for the past 23 years, the weekend has begun with an off-the-record Tech Summit where NBA stakeholders share their vision around technology and its role in growing and developing the game for fans, players, and rightsholders. And then there are ancillary conversations and demonstrations that occur throughout the day in suites and meeting rooms. One of the sessions this year focused on the NBA's efforts with Canon around volumetric capture.

At CES, the two companies held a press conference about how the Cleveland Cavaliers have used the system to create a demo of a VR experience for fans and also tied into Hollywood: director M. Night Shyamalan used the Canon system to create a VR experience for his upcoming film Knock at the Cabin.

Canon's Tsuyoshi Wakazono (left) and Michael Larson were on hand at NBA All-Star weekend in Salt Lake City to give NBA executives and stakeholders a look at a new VR experience.

At NBA All-Star weekend in Salt Lake City, guests were invited to check out the experience for themselves. In a demo room, they donned a VR headset to dive into the volumetric experience.

The biggest thing we're testing now is the VR component, using the feed from the cameras to create a fully immersive VR experience, says Michael Larson, senior fellow, new business, Canon. We've been doing some pilots with the [Cleveland] Cavaliers where we had an activation pregame in a social-zone area on the concourse to introduce the content to fans. It was amazing: the fans can be placed on the court with the players during a game.

To produce the activations, more than 100 Canon Cinema EOS-based cameras were installed at Cleveland's Rocket Mortgage FieldHouse (and Brooklyn's Barclays Center) and capture video information. That information is processed on proprietary Canon image-processing boxes behind each camera and turned into data. The data from all the cameras is sent to a server system that creates a point-cloud 3D model of the players and other objects on the court. Images are built using voxels, which are units of graphic information defining a point in three-dimensional space (basically, the 3D equivalent of a pixel), combined with RGB values.

Broadcasters can deploy the system as a new way to capture replays: an operator can fly a virtual camera through the 3D graphical representation of the game, capturing replays from virtually any angle (ESPN broadcast an entire game using the system last season).

The VR experience, however, is controlled by individual users. Donning a VR headset, they change their perspective via the VR controllers or by simply moving their body.

The demonstration in Salt Lake City had two clips: the first from an overhead position la Skycam, the second from down on the court. With each, moving from a standing to a sitting position lowered the viewing angle. Stepping forward while lowering one's head created a swooping-in-for-a-closer-look effect. And, of course, turning one's head changed the field of view.

NBA EVP, Media Operations and Technology, Steve Hellmuth believes that, because the graphical representations are more in tune with the world of VR, volumetric capture offers a better VR experience than relying on video cameras. And improvements are easier: the Canon Free Viewpoint system doesn't require cameras and lenses to be replaced to enhance the experience.

The goal, says Larson, is to update the firmware of the cameras, image-processing boxes, and image-processing software to improve the experience.

Moreover, improvements are more cost-effective to implement, because camera systems don't need to be swapped out (and software improvements can be developed more rapidly and deployed more easily).

With various applications hanging off the data with APIs, we could capture player-tracking statistics, player health and wellbeing, or even photography as the camera system can support an application for everything. You could even get to the point where you can tell if a shot is a finger roll versus a dunk.

Currently, the Canon system can be used to create a linear feed, with up to six streams (that is what ESPN did last year for its alternative broadcast), that is about three seconds delayed. The next step is to deliver a live VR feed to a limited number of fans and, eventually, to integrate the technology into direct-to-consumer platforms.

We're also working on how we optimize the delivery to mobile devices so users can use traditional, dragging, and pinching to control their perspectives, says Larson.

Besides sports, volumetric capture can be used for VR experiences around other video, allowing viewers to dive into a VR version, say, of a favorite movie scene or even documentary clips. More important, the technology is improving. Hellmuth considers the current Canon Free Viewpoint experience to be about 30% improved compared with its NBA launch at last year's All-Star Game in Cleveland.

The real key, as Hellmuth notes, is that deployment of a volumetric-capture system is about not just one application but many. The dollars-and-cents discussions don't concern generating an ROI around consumer subscriptions or advertising. Discussions around VR have almost always fallen apart in the past, but that may change when the VR experience involves a system that is deployed to help a team not only put wins on the board but also cut down on injuries. Invaluable intangible

LINK:	https://www.sportsvideo.org/2023/02/19/live-from-nba-all-star-weekend-...
	See more stories from svg

Most recent headlines