New York City Takes an Open Approach to Data
By Samuel Greengard
Obtaining and using government data has never been a simple proposition. Too often, data is stored in proprietary, if not arcane, formats, and formatting it in accessible and open formats has been next to impossible.
So, when New York City passed Local Law 11 of 2012, mandating agencies to systematically categorize data and make it available to the public, many observers focused a keen eye on how successful the city would be in making that goal a reality.
One year later, the initiative is changing the face of government. Today, there are approximately 1,750 unique representations of more than 1,000 raw data sets available at no charge via NYC OpenData. These span more than 60 city agencies, commissions and business improvements districts.
Available data sets span city operations, including cultural affairs, education, health, housing, property, public safety, social services and transportation. And the city has created a set of APIs to boost availability.
"Data are the building blocks of the digital age, and our open-data legislation—the most ambitious and comprehensive in the country—ensures that New York City will continue to blaze the technological trail for the next generation," stated Rahul Merchant, the chief information and innovation officer, Department of Information Technology and Telecommunications (DoITT).
Most data can be viewed through a standard Web browser and is available for download in several file formats, including CSV, PDF, RDF, RSS, XLS and XML.
The city is now using data and analysis in unprecedented ways to develop policy, drive operations and improve service to citizens. The goal is to allow developers, entrepreneurs and academics to put data to work in new and innovative ways.
"Anyone can utilize their skills and creativity to put this data to work to further improve the quality of life and strengthen New York City," noted Michael Flowers, chief analytics and open platform officer.
Among the data offerings: 10 years worth of city performance data; building information, including building complaints, permits and building jobs; emergency information encompassing evacuation zone information and evacuation centers; historical crime data for the past 12 years; and a list of licensed taxi drivers. The last list is updated on a daily basis.
The city had to adapt its data practices to accommodate the initiative. In September 2012, it created an "Open Data Policy and Technical Standards Manual," which outlines how New York City agencies can gather, structure and automate data flows to meet the requirements of Local Law 11.
In order to ensure that the process was collaborative and met the needs of citizens, DoITT enlisted the aid of the civic tech community and city agencies to create the document. The agency also launched a wiki that let citizens amend or comment on the document as it was being created.
The effort is ongoing, and the one-year anniversary of Local Law 11 highlights only part of the story. In September, DoITT will publish an open-data compliance plan that categorizes all public data sets held by the city and outlines plans to make them available by the end of 2018.
"We are putting an end to the days of city agencies posting information in confusing PDFs that cannot be searched or manipulated," stated New York Council Member Gale Brewer. "By opening up troves of data to the public in its raw form, we are giving technologists, good-government advocates and the public unprecedented access to New York City's municipal data."