Government IT Faces a Big Challenge in Big Data

Not a fan of Big Government? Well this will shock you: Government agencies will add a petabyte–that’s equal to one quadrillion bytes, or 1,024 terabytes over the next two years, according to a MeriTalk survey of 151 Federal government CIOs and IT managers. To put that in perspective, one petabyte of data is equal to 20 million four-drawer filing cabinets filled with text. Meanwhile, government agencies are struggling to reap the benefits of Big Data as information piles up, lacking the data storage and access, computational power, and personnel, the survey indicated.

The report, titled "The big data Gap," found that as agencies look to leverage big data, the technology and applications needed to successfully leverage big data are still emerging. Sixty percent of civilian agencies and 42 percent of Department of Defense/intelligence agencies say they are just now learning about big data and how it can work for their agency. Federal IT professionals say improving overall agency efficiency is the top advantage of big data (59 percent) followed by improving speed and accuracy of decisions (51 percent) and the ability to forecast (30 percent).

Whether it is an opportunity or a challenge, data continues to grow: 87 percent of IT professionals say their stored data has grown in the last two years, and 96 percent expect their data to grow in the next two years (by an average of 64 percent). Agencies reported, on average, it would take them at least 3 years to take full advantage of big data. In fact, agencies are doing very little with their data, according to survey results. Only 40 percent of those surveyed said they are making strategic decisions with the data, and just 28 percent collaborate with other agencies to analyze shared data.

There are other challenges as well: More than half (57 percent) of agencies reported a break in their system, saying at least one data set has outgrown their current management tools. While 64 percent of the IT professionals surveyed said their agency could scale to meet data growth, they estimated it would take 10 months on average to double their short- to medium-term capacity. In addition, some agencies are taking steps to improve their ability to manage and make decisions with big data. Top tactics include investing in IT infrastructure to optimize data storage (39 percent), training IT professionals to manage/analyze big data (33 percent), and improving the security of stored data (31 percent).

"Government has a gold mine of data at its fingertips," Mark Weber, president of U.S. Public Sector for NetApp, which sponsored the survey, said in a prepared statement. "The key is turning that data into high-quality information that can increase efficiencies and inform decisions. Agencies need to look at big data solutions that can help them efficiently process, analyze, manage, and access data, enabling them to more effectively execute their missions."

To read the original eWeek article, click here: Big Data Still a Big Challenge for Government IT