Light-field Advances Virtual Production

The essential mechanics of recording light as it enters a camera haven t changed for 150 years but recent advances in light field technology could have profound consequences for everything we understand about moving image content creation.

Rather than recording a flat 2D picture, light-field captures all the light falling on the camera in five dimensions including the direction and intensity of each ray. Research took off in the 1990s and centered on two means of acquisition: arranging multiple cameras to simultaneously record different angles of the same scene, or using a single camera fitted with a micro-lens array (MLA) comprising hundreds of thousands, even millions, of miniature lenses.

You would then need software to compute the data, massive processing power, or compression - or both - to crunch the data in anything like realtime, a means of manipulating it in post and of presenting it.

While much of this chain is rudimentary there are solutions coming to market, notably at the front end. Indeed, the technology is moving rapidly from experimentation to commercialisation. Light-field has made it onto the agenda at the SMPTE and at the MPEG JPEG committees, where a new working group on the topic has been established. Hollywood has taken note too.

Light-field is the future of imaging, declares Jon Karafin, Head of Light-field Video at Lytro which has just announced a prototype light-field cinema camera. It is not a matter of if but when .

The benefits of being able to manipulate such unprecedented scene detailing data range from a more cost effective path to creating visual effects, to refocusing any part of any frame or even relighting a scene. Shutter angle and frame rate can be computationally altered in post making different release mastering from the same original material almost as simple as pressing a button.

The tech is also believed superior to current means of shooting 360-video for Virtual Reality. Conventional VR cameras give you a left and right eye flat stereo view akin to 3D in cinema, whereas a light-field would give you a spatial sense of actually being present, says Simon Robinson, Chief Scientist, The Foundry, which has helped devise plug-ins for both Fraunhofer and Lytro systems.

One significant implication of light-field is that it takes the onus of creative decision away from set and into post production. If any optical parameters can be changed including exposure and camera position - does light-field mean the end of shooting for post and the beginning of shooting in post?

Ultimately, this is a disruptive technology and we need the feedback of the studio, cinematographer and VFX community to make sure what we are creating meets their needs as well as helping them understand the ultimate creative control this unleashes, Karafin acknowledges.

While some developers are working on a single-camera solution, researchers at Germany s Fraunhofer IIS prefer a multiple camera array claiming that a MLA will not provide sufficient parallax to deliver a rounded impression of a scene. Lytro counters that obtaining parallax from a single MLA is far easier and far more accurate than any number of seperate cameras can ever be.

Others, such as Raytrix, a maker of light-field optics for industrial applications, says all existing light field systems are limited by the laws of physics. They are workable with close-up subjects like a face but if you want to extract depth information for scenes ten meters away you might as well use standard stereo 3D cameras, says Co-Founder Christian Perwass.

In the short term, light-field data will be used for manipulation of imagery and output to standard 2D screens, but in the longer term the technique might be used to create live action holographs. Several companies, including Zebra, Holographica and Leia3D, have designs on projecting a three-dimensional interactive image.

Imagine looking out of a window in your home. Now imagine that as a holographic picture," says Robinson. "That is where we are headed.

LINK:	http://www.ibc.org/hot-news/light-field-advances-virtual-production...
	See more stories from ibc

Most recent headlines