Framestore Frames a Better Storage Approach

By Samuel Greengard

Few companies face the enormous processing and storage demands of a motion picture special-effects company. United Kingdom-based Framestore—which has produced visual effects for commercials, television programs and films such as Avatar, The Golden Compass, Harry Potter and the Deathly Hallows, Skyfall, Lincoln and War Horse—is very dependent on IT systems to produce art and keep projects on schedule.

“The setting is not dissimilar to a high-performance computing environment,” says Steve MacPherson, chief technology officer.

About two years ago, Framestore found itself facing substantial IT challenges. While working on the upcoming science fiction film Gravity, which required peak-processing power of 15,000 cores simultaneously, bottlenecks and slowdowns ensued. In some cases, systems slowed to a crawl as network and storage systems drained all available bandwidth.

“We recognized that the existing IT environment was not adequate,” MacPherson notes. “We required a more robust storage infrastructure.”

Altogether, Framestore, which operates from five primary sites based in London, Montreal and Los Angeles, requires between 1.2 petabytes and 1.4 PB of storage space. Individual shows or films can range from about 50 terabytes to upward of 500 TB.

“Depending on the amount of detail in an animation, we can wind up with very large data sets that can take a long time to load on a workstation,” MacPherson explains. “In the past, we used Lustre, an open-source custom file system, but we needed to dramatically improve system performance. It isn’t acceptable to wait minutes for a scene to load on a computer.”

Framestore’s IT group wanted to partition storage so that artists and others could pull from storage pools separate from those being used for other tasks within the company. But the challenge was magnified by multiple sites that had to function within a cohesive whole.

“In the past, there was too much shuffling of data in the background,” he explains. “We wanted to build an environment that relies on caching within a low latency and has an infrastructure that is transparent to end users.”

The company turned to Avere for a next-generation solid-state-drive-based network-attached storage solution. Using the vendor’s FXT 4500 Edge filers with FlashMove and FlashMirror software, Framestore introduced real-time tiering that allows data to be moved between NAS systems in a transparent and nondisruptive manner.

The system keeps replicated data closely in sync by sending updates directly and in parallel to both the primary and secondary NAS core filers. In addition, FlashMirror offloads the replication processing load from the storage and supports clustering to scale data replication performance to any level required.

MacPherson says that the initiative has paid dividends and dramatically improved network performance and workflows for 40-plus artists and other workers. In a business that’s under constant pressure to perform faster and better, the most efficient use of resources is critical.

“Despite the enormous demands on the workstations, there’s now an infrastructure in place to accommodate the needs of end users,” MacPherson says. “We are able to get the right data and jobs to the right people quickly and efficiently.”