Airline Big Data Madne by Jay Livens

A few months ago I ran across an interesting article at ComputerWorldUK aptly titled, Boeing 787s to create half a terabyte of data per flight, says Virgin Atlantic. The article got me thinking about the big data needs of airlines and how growing information volumes will force them to re-think how they store and manage data. While the 787 is a driving factor for storage growth in airlines, the reality is that companies in all industries are facing similar challenges.

While the article states that the 787 creates 500GB of data, most airline fleets contain many different types of aircraft. Thus, the big data problem transcends the 787, and my guess is that airlines are probably storing hundreds of PB of this kind of data. The 787 provides a compelling example of the challenge. So let's do the math in the context of one of the larger buyers of this technology, United Airlines (UAL).

3 Estimated 787 Average flights/day/aircraft (source)

6 Estimated 787 Flight days/week (source)

936 Flights/year/787

468TB Information created annually by each 787

50 Number of 787s ordered by United (UAL), (source)

23.4PB(!) Data generated by all UAL 787s in a year

450TB Data generated per month by UAL's entire 787 fleet

The math indicates that UAL's 787 fleet will general approximately 450TB per month when fully operational. That is a significant quantity of data and based on the price of an inexpensive, entry-level disk array, UAL will need to purchase approximately $280,000 worth of disk storage every month or spend about $3.3M each year. (To be fair, it is unlikely that UAL would rely on a consumer oriented disk array like quoted here and so their prices would probably be higher.) As you can imagine, these problems will only get worse as even more data generating aircraft are added to the fleet. The obvious question is how airlines can cost effectively store these ever growing amounts of data. The answer might surprise you, tape.

There is no doubt that airplane data is critical for airlines to maintain for operational purposes. They must know the status of every plane and what items need to be or have been fixed. However, the value of the data declines over time. Airlines must know immediately if something is wrong; however as data ages, the access urgency declines and slower access times can more readily be tolerated.

Tape provides a compelling storage option for UAL. If you consider LTO-6, a cartridge costs about $100 and provides 6.25TB of capacity, and so storing 450TB on LTO-6 will cost approximately $72,000 per month or just under 4x the cost of the disk option described above. At the same time tape delivers data availability and longevity that exceeds disk and a performance profile that is only marginally slower. UAL should use disk to store the most recent and frequently accessed data and then move the older information to tape as access needs decline. The result is a system that combines the best of both of both options (e.g. fast performance on high cost disk for active data and reduced cost and increased reliability for older data).

The simple reality is that UAL and many other companies are faced with massive data growth. IT has traditionally addressed this challenge by adding expensive disk storage, but budgets are rarely growing at the same rate as data and so this strategy has become untenable. Hybrid solutions combining disk and tape provide a compelling alternative for companies of all sizes.

LINK:	http://blog.ironmountain.com/2013/service-lines/data-backup-and-recove...
	See more stories from ironmountain

Most recent headlines