Advanced Storage Helps a Nonprofit Protect History

By Samuel Greengard  |  Posted 2015-04-21 Email Print this article Print
Storage Strategy

Densho deploys a streamlined storage strategy to boost efficiency and reduce costs for preserving 60,000 photographs, documents, video and historical items.

Managing historical documents and photographs is an enormously challenging task. Paper decays, and there's also the challenge of cataloging and managing images and then making them accessible.

At Densho, a Seattle not-for-profit based organization that collects and preserves historical images related to the internment of Japanese Americans during World War II, the road to digitization has been paved with more than a few storage challenges.

"We have about 60,000 photographs, documents and historical items in our repository," says Geoff Froh, deputy director and CIO for Densho. "We also have about 900 video oral history interviews that add up to about 1,800 hours of video."

All of this content requires approximately 45 terabytes of data storage, and there's periodically a need to add storage devices. "What makes things extraordinarily difficult is [the fact] that we are a small organization with limited resources, but we have huge storage requirements," Froh points out.

In the past, the organization relied on a low-capacity storage area network (SAN), along with network-attached storage (NAS). However, the hybrid approach was expensive and was prone to glitches and failures. It also delivered subpar performance and lacked flexibility and scalability.

"We constantly had to move data and content back and forth across devices," Froh explains. "It was an extremely cumbersome and inefficient process" for the organization's 10-person production staff.

Scaling Massively at an Affordable Price Point

As a result, Densho turned to four Qumulo Q0626 hybrid storage appliances and an Arista 10 GbE switch. The system, which relies on both solid state drives (SSDs) and hard drives, "allowed us to scale massively at a much more affordable price point," he says. "We were able to consolidate our storage onto a single platform that is highly manageable."

The organization began testing the technology about a year ago and went live with the storage solution last fall. Densho is now exclusively on the Qumulo production platform. Employees work directly off the Qumulo-based cluster via the network file system (NFS) to ingest, manage and catalog digitized historic files.

The benefits have been substantial, Froh says. Densho has increased its overall storage capacity and introduced a more scalable framework, and the move to a single platform has greatly simplified administration tasks. Froh and other staff now have analytics tools available to view performance in real time and better understand available space and how the storage devices use it.

Real-time analytics built into the file system ultimately allows Froh to view the precise data and information he requires at any given moment. "Knowing which files are in use is extremely powerful," he points out. "We can see exactly which files or directories are consuming lots of IOPS. If there's a problem or  a need for troubleshooting, we can determine where the problem lies and take immediate and specific action."

Froh reports that the technology has helped him reduce the time spent on storage management from about 10 percent of his workweek to less than five minutes per week—while essentially eliminating system downtime. "We now have a storage platform that helps us maximize our time and our dollars," he concludes.


Samuel Greengard, a Baseline contributor, writes about business and technology for Baseline, CIO Insight and other publications. His most recent book is The Internet of Things (MIT Press, 2015).


Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters