GoPro Open Sources Video Metadata

tools of the trade

May 16, 2017

words: David Newman, GoPro Technical Fellow

met a da ta

noun, a set of data that describes and gives information about other data.

Well that sounds boring, but it is potentially very cool. Every photo you take, on any camera, is likely filled with metadata. This data often contains location, camera model, lens type, and general information about how the image was taken. It is this data that makes your photo library easier to browse, events are automatically grouped, and exposures auto corrected. Photo metadata has enabled entirely new types of media, like timelapse mining from public photo archives (see here). All this is possible because the metadata for photos is standardized and free.

Video, where is your metadata? If random consumer photos can be formed into timelapses, the potential for video is even greater. Metadata should answer, how was this file made? In what environment? At what location? If, when and how was the camera moving? At GoPro, we've had the ambition to solve these questions, but it turned out to be treading new ground. The popular consumer video file format MP4 had no clear provision to store telemetry or any time varying metadata. Yet we must store telemetry within the video, not in a separate file, so consumers can't misplace side-car files, or be expected to upload them separately. Metadata can't be lost because you ejected the battery before the capture was complete*, or crashed into a tree that caused the same, so the metadata must be regularly interleaved with video and audio, not simply added to the end upon a file close. Finally, it must store all of today's and tomorrow's wide range of sensor data, and be easily retrieved in more than GoPro's own utilities, without technical or legal hurdles. We needed a new standard, so GoPro created one, and made it completely free (open source linked below.)

The first release of this new MP4 embedded metadata came with the HERO5. The HERO5 has extensive metadata for photos and video, adding sensors for accelerometer, gyroscope and high frequency GPS location. Running the GPS 18 times faster than your average smartphone, combining that with IMU data, allows every bike jump, wave drop in, racing turn, and top speed reached, to be accurately measured and displayed. You can display a lot of this today using the free Quik for Desktop app. However, a rendered display of your best lap, or highest jump, is only scratching the surface for the use of telemetry and the broader category of metadata. The HERO5 stores much more than location and camera motion in its video files; it stores its own temperature, image exposure, shutter speed, the very precise GPS time, detailed flight telemetry from Karma, and data from GoPro-enabled third-party Bluetooth devices. The amount and range of metadata will continue to expand over GoPro's future.

GPMF, the new format without a precise acronym, GoPro Metadata Format or now General Purpose Metadata Framework, is a new track within an MP4 file, working just like video and audio which have their own tracks. The timing relationship between metadata collection and the video and audio captured is stored using existing MP4 mechanisms, so if a developer can handle video and audio within an MP4, metadata is very similar. The richness of GPMF comes from its ability to nest multiple sensor streams into a single MP4 track, and do it in a self-describing way. The self-describing nature is critical, it allows future sensors to be added without updating reader applications, or even registering the sensor's data format with a GPMF managing entity (doesn't exist, not needed.) The big plus, this is all completely free for anyone to use. A GPMF-parser was recently made available as an open source project (https://github.com/gopro/gpmf-parser) for developers to easily include video metadata within their application. This is just one of many new open source projects you will see from GoPro.

* GoPro has amazing file recovery where an impact prevents a camera from safely finishing a recording, recovering video, audio and now metadata in most situations, automatically repairing incomplete files the next time you power the camera on.

LINK:	https://gopro.com/news/gopro-video-metadata-open-source-explained...
	See more stories from gopro

Most recent headlines