Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

August 18 2013

Tactical Technology Collective | Turning information into action

Tactical Technology Collective | Turning information into action
https://www.tacticaltech.org

Tactical Tech is an organisation dedicated to the use of information in activism.

We focus on the use of data, design and technology in campaigning through our Evidence & Action programme and on helping activists understand and manage their digital security and privacy risks through our Privacy & Expression programme.

https://www.tacticaltech.org/sites/dev.tacticaltech.org/files/boulder-datadesign.png

https://drawingbynumbers.org/data-design-basics/note-7-get-the-details
https://camp2013.tacticaltech.org/content/track-summaries

http://nu.org.za/the-power-of-data-as-evidence

http://schoolofdata.org

#information #data-journalism #militer

April 18 2013

Sprinting toward the future of Jamaica

Creating the conditions for startups to form is now a policy imperative for governments around the world, as Julian Jay Robinson, minister of state in Jamaica’s Ministry of Science, Technology, Energy and Mining, reminded the attendees at the “Developing the Caribbean” conference last week in Kingston, Jamaica.

photo-22photo-22

Robinson said Jamaica is working on deploying wireless broadband access, securing networks and stimulating tech entrepreneurship around the island, a set of priorities that would have sounded of the moment in Washington, Paris, Hong Kong or Bangalore. He also described open access and open data as fundamental parts of democratic governance, explicitly aligning the release of public data with economic development and anti-corruption efforts. Robinson also pledged to help ensure that Jamaica’s open data efforts would be successful, offering a key ally within government to members of civil society.

The interest in adding technical ability and capacity around the Caribbean was sparked by other efforts around the world, particularly Kenya’s open government data efforts. That’s what led the organizers to invite Paul Kukubo to speak about Kenya’s experience, which Robinson noted might be more relevant to Jamaica than that of the global north.

Kukubo, the head of Kenya’s Information, Communication and Technology Board, was a key player in getting the country’s open data initiative off the ground and evangelizing it to developers in Nairobi. At the conference, Kukubo gave Jamaicans two key pieces of advice. First, open data efforts must be aligned with national priorities, from reducing corruption to improving digital services to economic development.

“You can’t do your open data initiative outside of what you’re trying to do for your country,” said Kukubo.

Second, political leadership is essential to success. In Kenya, the president was personally involved in open data, Kukubo said. Now that a new president has been officially elected, however, there are new questions about what happens next, particularly given that pickup in Kenya’s development community hasn’t been as dynamic as officials might have hoped. There’s also a significant issue on the demand-side of open data, with respect to the absence of a Freedom of Information Law in Kenya.

When I asked Kukubo about these issues, he said he expects a Freedom of Information law will be passed this year in Kenya. He also replied that the momentum on open data wasn’t just about the supply side.

“We feel that in the usage side, especially with respect to the developer ecosystem, we haven’t necessarily gotten as much traction from developers using data and interpreting cleverly as we might have wanted to have,” he said. “We’re putting putting more into that area.”

With respect to leadership, Kukubo pointed out that newly elected Kenyan President Uhuru Kenyatta drove open data release and policy when he was the minister of finance. Kukubo expects him to be very supportive of open data in office.

The development of open data in Jamaica, by way of contrast, has been driven by academia, said professor Maurice McNaughton, director of the Center of Excellence at the Mona School of Business at the University of the West Indies (UWI). The Caribbean Open Institute, for instance, has been working closely with Jamaica’s Rural Agriculture Development Authority (RADA). There are high hopes that releases of more data from RADA and other Jamaican institutions will improve Jamaica’s economy and the effectiveness of its government.

Open data could add $35 million annually to the Jamaican economy, said Damian Cox, director of the Access to Information Unit in the Office of the Prime Minister, citing a United Nations estimate. Cox also explicitly aligned open data with measuring progress toward Millennium Development Goals, positing that increasing the availability of data will enable the civil society, government agencies and the UN to more accurately assess success.

The development of (open) data-driven journalism

Developing the Caribbean focused on the demand side of open data as well, particularly the role of intermediaries in collecting, cleaning, fact checking, and presenting data, matched with necessary narrative and context. That kind of work is precisely what data-driven journalism does, which is why it was one of the major themes of the conference. I was invited to give an overview of data-driven journalism that connected some trends and highlighted the best work in the field.

I’ve written quite a bit about how data-driven journalism is making sense of the world elsewhere, with a report yet to come. What I found in Jamaica is that media there have long since begun experimenting in the field, from the investigative journalism at Panos Caribbean to the relatively recent launch of diGJamaica by the Gleaner Company.

diGJamaica is modeled upon the Jamaican Handbook and includes more than a million pages from The Gleaner newspaper, going back to 1834. The site publishes directories of public entities and public data, including visualizations. It charges for access to the archives.

Legends and legacies

Usain Bolt in JamaicaUsain Bolt in Jamaica

Olympic champion Usain Bolt, photographed in his (fast) car at the UWI/Usain Bolt Track in Mona, Jamaica.

Normally, meeting the fastest man on earth would be the most memorable part of any trip. The moment that left the deepest impression from my journey to the Caribbean, however, came not from encountering Usain Bolt on a run but from within a seminar room on a university campus.

As a member of a panel of judges, I saw dozens of young people present after working for 30 hours at a hackathon at the University of the West Indies. While even the most mature of the working apps was still a prototype, the best of them were squarely focused on issues that affect real Jamaicans: scoring the risk of farmers that needed banking loans and collecting and sharing data about produce.

The winning team created a working mobile app that would enable government officials to collect data at farms. While none of the apps are likely to be adopted by the agricultural agency in its current form, or show up in the Google Play store this week, the experience the teams gained will help them in the future.

As I left the island, the perspective that I’d taken away from trips to Brazil, Moldova and Africa last year was further confirmed: technical talent and creativity can be found everywhere in the world, along with considerable passion to apply design thinking, data and mobile technology to improve the societies people live within. This is innovation that matters, not just clones of popular social networking apps — though the judges saw more than a couple of those ideas flow by as well.

In the years ahead, Jamaican developers will play an important role in media, commerce and government on the island. If attracting young people to engineering and teaching them to code is the long-term legacy of efforts like Developing the Caribbean, it will deserve its own thumbs up from Mr. Bolt. The track to that future looks wide open.

photo-23photo-23

Disclosure: the cost of my travel to Jamaica was paid for by the organizers of the Developing the Caribbean conference.

March 22 2013

Sensoring the news

When I went to the 2013 SXSW Interactive Festival to host a conversation with NPR’s Javaun Moradi about sensors, society and the media, I thought we would be talking about the future of data journalism. By the time I left the event, I’d learned that sensor journalism had long since arrived and been applied. Today, inexpensive, easy-to-use open source hardware is making it easier for media outlets to create data themselves.

“Interest in sensor data has grown dramatically over the last year,” said Moradi. “Groups are experimenting in the areas of environmental monitoring, journalism, human rights activism, and civic accountability.” His post on what sensor networks mean for journalism sparked our collaboration after we connected in December 2011 about how data was being use in the media.

AP Beijing Air Quality graphicAP Beijing Air Quality graphic

Associated Press visualization of Beijing air quality. See related feature.

At a SXSW panel on “sensoring the news,” Sarah Williams, an assistant professor at MIT, described how the Civic Data Design Project had partnered with the Associated Press to independently measure air quality in Beijing.

Prior to the 2008 Olympics, the coaches of the Olympic teams had expressed serious concern about the impact of air pollution on the athletes. That, in turn, put pressure on the Chinese government to take substantive steps to improve those conditions. While the Chinese government released an index of air quality, explained Williams, they didn’t explain what went into it, nor did they provide the raw data.

The Beijing Air Tracks project arose from the need to determine what the conditions on the ground really were. AP reporters carried sensors connected to their cellphones to detect particulate and carbon monoxide levels, enabling them to report air quality conditions back in real-time as they moved around the Olympic venues and city.

The sensor data helped the AP measure the effect of policy decisions that the Chinese government made, said Williams, from closing down factories to widespread shutdowns of different kinds of industries. The results from the sensor journalism project, which showed a decrease in particulates but conditions 12 to 25 times worse than New York City on certain days, were published as an interactive data visualization.

AP Beijing mash-up of particulate levels and photography in Beijing.AP Beijing mash-up of particulate levels and photography in Beijing.

Associated Press mashup of particulate levels and photography at the Olympic stadium in Beijing over time.

This AP project is a prime example of how sensors, data journalism, and old-fashioned, on-the-ground reporting can be combined to shine a new level of accountability on official reports. It won’t be the last time this happens, either. Around the world, from the Amazon to Los Angeles to Japan, sensor data is now being put to use by civic media and journalists.

Sensing civic media

There are an increasing number of sensors in our lives, said John Keefe, a data news editor for WNYC, speaking at his SXSW panel in Austin. From the physical sensors in smartphones to new possibilities built with Arduino or Raspberry Pi hardware, Keefe highlighted how journalists could seize hold of new possibilities.

“Google takes data from maps and Android phones and creates traffic data,” Keefe said. “In a sense, that’s sensor data being used live in a public service. What are we doing in journalism like that? What could we do?”

The evolution of Safecast offers a glimpse of networked accountability, collecting and publishing radiation data through sensors, citizen science and the Internet. The project, which won last year’s Knight News Challenge on data, is now building the infrastructure to enable people to help monitor air quality in Los Angeles.

Sensor journalism is also being applied to make sense of the world in using remote sensing data and satellite imagery. The director of that project, Gustavo Faleiros, recently described how environmental reporting can be combined with civic media to collect data, with relevant projects in Asia, Africa and the Americas. For instance, Faleiros cited an environmental monitoring project led by Eric Paulos of the University of California at Berkeley’s Center for New Media, where sensors on taxis were used to gather data in Accra, Ghana.

Another direction that sensor data could be applied lies in social justice and education. At SXSW, Sarah Williams described [slides] how the Air Quality Egg, an open source hardware device, is being used to make an argument for public improvements. At the Cypress Hills Community School, kids are bringing the eggs home, measuring air quality and putting data online, said Williams.

Air Quality Eggs at Cypress Hill Community SchoolAir Quality Eggs at Cypress Hill Community School

Air Quality Eggs at Cypress Hill Community School.

“Health sensors are useful when they can compare personal real-time data against population-wide data,” said Nadav Aharony, who also spoke on our panel in Austin.

Aharony talked about how Behavio, a startup based upon his research on smartphones and data at MIT, has created funf, an open source sensing toolkit for Android devices. Aharony’s team has now deployed an integration with Dropbox that requires no coding ability to use.

According to Aharony, the One Laptop Per Child project is using funf in tablets deployed in Africa, in areas where there are no schools. Researchers will use funf as a behavioral tool to sense how children are interacting with the devices, including whether tablets are next to one another.

Sensing citizen science

While challenges lie ahead, it’s clear that sensors will be used to create data where there was none before. At SXSW, Williams described a project in Nairobi, Kenya, where cellphones are being used to map informal bus systems.

The Digital Matatus project is publishing the data into the General Transit Feed Standard, one of the most promising emerging global standards for transit data. “Hopefully, a year from now [we] will have all the bus routes from Nairobi,” Williams said.

Map of Matatus stops in Nairobi, KenyaMap of Matatus stops in Nairobi, Kenya

Map of Matatus stops in Nairobi, Kenya

Data journalism has long depended upon official data released by agencies. In recent years, data journalists have begun scraping data. Sensors allow another step in that evolution to take place, where civic media can create data to inform the public interest.

Matt Waite, a professor of practice and head of the Drone Journalism Lab at the University of Nebraska-Lincoln, joined the panel in Austin using a Google Hangout and shared how he and his students are experimenting with sensors to gather data for projects.

Journalists are going to run up against stories where no one has data, he said. “The old way was to give up,” said Waite. “I don’t think that’s the way to do it.”

Sensors give journalists a new, interesting way to enlist a distributed audience in gathering needed data, he explained. “Is it ‘capital N’ news? Probably not,” said Waite. “But it’s something people are really interested in. The easy part is getting a parts list together and writing software. The hard part is the creative process it takes to figure out what we are going to measure and what it means.”

In an interview with the Nieman Journalism Lab on sensor journalism, Waite also raised practical concerns with the quality of data collection that can be gathered with inexpensive hardware. “One legitimate concern about doing this is, you’re talking about doing it with the cheapest software you can find,” Waite told the Nieman Lab’s Caroline O’Donovan. “It’s not expertly calibrated. It’s not as sensitive as it possibly could be.”

Those are questions that will be explored practically in New York in the months ahead, when New York City’s public radio station will be collaborating with the Columbia School of Public Health to collect data about New York’s environmental conditions. They’ll put particulate detectors, carbon dioxide monitors, leg motion sensors, audio monitors, cameras and GPS trackers on bicycles and ride around the city collecting pollution data.

“At WNYC, we already do crowdsourcing, where we ask our audience to do something,” said Keefe. “What if we could get our audience to do something with this? What if you could get an audience to work with you to solve a problem?”

Keefe also announced the Cicada Project, where WNYC is inviting its listeners to build homemade sensors and track the emergence of cicadas this spring across New Jersey, New York and the Northeast region.

This cicada tracker project is a 21st century parallel to the role that birders have played for decades in the annual Christmas Bird Count, creating new horizons for citizen science and public media.

Update: WNYC’s public is responding in interesting ways that go beyond donations. On Twitter, Keefe highlighted the work of a NYC-based hacker, Guan, who was able to make a cicada tracker for $20, 1/4 the cost of WNYC’s kit.

Sensing challenges ahead

Just as civic technologists need to be mindful of “solutionism,” so too will data journalists need to be aware of the “sensorism” that exists in the health care world, as John Wilbanks pointed out this winter.

“Sensorism is rife in the sciences,” Wilbanks wrote. “Pick a data generation task that used to be human centric and odds are someone is trying to automate and parallelize it (often via solutionism, oddly — there’s an app to generate that data). What’s missing is the epistemic transformation that makes the data emerging from sensors actually useful to make a scientific conclusion — or a policy decision supposedly based on a scientific consensus.”

Anyone looking to practice sensor journalism will face interesting challenges, from incorrect conclusions based upon faulty data to increased risks to journalists carrying the sensors, to gaming or misreporting.

“Data accuracy is both a real and a perceived problem,” said Moradi at SXSW. “Third-party verification by journalists or other non-aligned groups may be needed.”

Much as in the cases of “drone journalism” and data journalism, context, usage and ethics have to be considered before you launch a quadcopter, fire up a scraper or embed sensors around your city. The question you come back to is whether you’re facing a new ethical problem or an old ethical problem with new technology, suggested Waite at SXSW. “The truth is, most ethical issues you can find with a new analogue.”

It may be, however, that sensor data, applied to taking a “social MRI” or other uses, may present us with novel challenges. For instance, who owns the data? Who can access or use it? Under what conditions?

A GPS device is a form of sensor, after all, and one that’s quite useful to law enforcement. While the Supreme Court ruled that the use of a GPS device for tracking a person without a warrant was unconstitutional, sensor data from cellphones may provide law enforcement with equal or greater insight into a target’s movements. Journalists may well face unexpected questions about protecting sources if their sensor data captures the movements or actions of a person of interest.

“There’s a lot of concern around privacy,” said Moradi. “What data can the government request? Will private companies abuse personal data for marketing or sales? Do citizens have the right to personal data held by companies and government?”

Aharony outlined many of the issues in a 2011 paper on stealing reality, exploring what happens when criminals become data scientists.

“It’s like a slow-moving attack if you attach yourself to someone’s communication,” said Aharony, in a follow-up interview in Austin. “‘iPhonegate‘ didn’t surprise people who know about mobile app data or how the cellular network is architected. Look at what happened to Path. You can make mistakes without meaning to. You have to think about this and encrypt the data.”

This post is part of our series investigating data journalism.

February 07 2013

Looking at the many faces and forms of data journalism

Over the past year, I’ve been investigating data journalism. In that work, I’ve found no better source for understanding the who, where, what, how and why of what’s happening in this area than the journalists who are using and even building the tools needed to make sense of the exabyte age. Yesterday, I hosted a Google Hangout with several notable practitioners of data journalism. Video of the discussion is embedded below:

Over the course of the discussion, we talked about what data journalism is, how journalists are using it, the importance of storytelling, ethics, the role of open source and “showing your work” and much more.

Participants

Guests on the hangout included:

Projects

Here are just a few of the sites, services and projects we discussed:

In addition, you can see more of our data journalism coverage here.

January 10 2013

Want to analyze performance data for accountability? Focus on quality first.

Here’s an ageless insight that will endure well beyond the “era of big data“: poor collection practices and aging IT will derail any institutional efforts to use data analysis to improve performance.

According to an investigation by the Los Angeles Times, poor record-keeping is holding back state government efforts to upgrade California’s 911 system. As with any database project, beware “garbage in, garbage out,” or “GIGO.”

As Ben Welsh and Robert J. Lopez reported for the L.A. Times in December, California’s Emergency Medical Services Authority has been working to centralize performance data since 2009.

Unfortunately, it’s difficult to achieve data-driven improvements or manage against perceived issues by applying big data to the public sector if the data collection itself is flawed. The L.A. Times reported quality issues stemmed from how response times were measured to record keeping on paper to a failure to keep records at all.

lafdanalysislafdanalysis
Image Credit: Ben Welsh, who mapped 911 response time data for Los Angeles Times.

When I shared this story with the Radar team, Nat Torkington suggested revisiting the “Observe, Orient, Decide, and Act” (OODA) loop familiar to military strategists.

“If your observations are flawed, your decisions will be too,” wrote Nat, in an email exchange. “If you pump technology investment into the D phase, without similarly improving the Os, you’ll make your crappy decisions faster.”

Alistair Croll explored the relevance of OODA to big data in his post on the feedback economy last year. If California wants to catalyze the use of data-driven analysis to improve response times that vary by geography and jurisdictions, start with the first “O.”

The set of factors at play here, however, means that there won’t be a single silver bullet for putting California’s effort back on track. Lack of participation and reporting standards, and old IT systems are all at issue — and given California’s ongoing financial issues, upgrading the latter and requiring local fire departments and ambulance firms to spend time and money on data collection will not be an easy sell.

Filed from the data desk

The investigative work of the L.A. Times was substantially supported by its Data Desk, a team of reporters and web developers that specializes in maps, databases, analysis and visualization. I included their interactive visualization mapping how fast the Los Angeles Fire Department responded to calls in my recent post on how data journalism is making sense of the world. When I profiled Ben Welsh’s work last year in our data journalist series, he told me this kind of project is exactly the sort of work he’s most proud of doing.

“As we all know, there’s a lot of data out there,” said Welsh, in our interview, “and, as anyone who works with it knows, most of it is crap. The projects I’m most proud of have taken large, ugly datasets and refined them into something worth knowing: a nut graf in an investigative story or a data-driven app that gives the reader some new insight into the world around them.”

The Data Desk set a high bar in this most recent investigation by not only making sense of the data, but also in releasing the data behind the open source maps of California’s emergency medical agencies it published as part of the series.

This isn’t the first time they’ve made code available. As Welsh noted in a post about the series, the Data Desk has “previously written about the technical methods used to conduct [the] investigation, released the base layer created for an interactive map of response times and contributed the location of LAFD’s 106 fire station to the Open Street Map.”

Creating an open source newsroom is not easy. In sharing not only its code but its data, the Los Angeles Times is setting a notable example for the practice of open journalism in the 21st century, building out the newsroom stack and hinting at media’s networked future.

This post is part of our series investigating data journalism.

Want to analyze performance data for accountability? Focus on quality first.

Here’s an ageless insight that will endure well beyond the “era of big data“: poor collection practices and aging IT will derail any institutional efforts to use data analysis to improve performance.

According to an investigation by the Los Angeles Times, poor record-keeping is holding back state government efforts to upgrade California’s 911 system. As with any database project, beware “garbage in, garbage out,” or “GIGO.”

As Ben Welsh and Robert J. Lopez reported for the L.A. Times in December, California’s Emergency Medical Services Authority has been working to centralize performance data since 2009.

Unfortunately, it’s difficult to achieve data-driven improvements or manage against perceived issues by applying big data to the public sector if the data collection itself is flawed. The L.A. Times reported quality issues stemmed from how response times were measured to record keeping on paper to a failure to keep records at all.

lafdanalysislafdanalysis
Image Credit: Ben Welsh, who mapped 911 response time data for Los Angeles Times.

When I shared this story with the Radar team, Nat Torkington suggested revisiting the “Observe, Orient, Decide, and Act” (OODA) loop familiar to military strategists.

“If your observations are flawed, your decisions will be too,” wrote Nat, in an email exchange. “If you pump technology investment into the D phase, without similarly improving the Os, you’ll make your crappy decisions faster.”

Alistair Croll explored the relevance of OODA to big data in his post on the feedback economy last year. If California wants to catalyze the use of data-driven analysis to improve response times that vary by geography and jurisdictions, start with the first “O.”

The set of factors at play here, however, means that there won’t be a single silver bullet for putting California’s effort back on track. Lack of participation and reporting standards, and old IT systems are all at issue — and given California’s ongoing financial issues, upgrading the latter and requiring local fire departments and ambulance firms to spend time and money on data collection will not be an easy sell.

Filed from the data desk

The investigative work of the L.A. Times was substantially supported by its Data Desk, a team of reporters and web developers that specializes in maps, databases, analysis and visualization. I included their interactive visualization mapping how fast the Los Angeles Fire Department responded to calls in my recent post on how data journalism is making sense of the world. When I profiled Ben Welsh’s work last year in our data journalist series, he told me this kind of project is exactly the sort of work he’s most proud of doing.

“As we all know, there’s a lot of data out there,” said Welsh, in our interview, “and, as anyone who works with it knows, most of it is crap. The projects I’m most proud of have taken large, ugly datasets and refined them into something worth knowing: a nut graf in an investigative story or a data-driven app that gives the reader some new insight into the world around them.”

The Data Desk set a high bar in this most recent investigation by not only making sense of the data, but also in releasing the data behind the open source maps of California’s emergency medical agencies it published as part of the series.

This isn’t the first time they’ve made code available. As Welsh noted in a post about the series, the Data Desk has “previously written about the technical methods used to conduct [the] investigation, released the base layer created for an interactive map of response times and contributed the location of LAFD’s 106 fire station to the Open Street Map.”

Creating an open source newsroom is not easy. In sharing not only its code but its data, the Los Angeles Times is setting a notable example for the practice of open journalism in the 21st century, building out the newsroom stack and hinting at media’s networked future.

This post is part of our series investigating data journalism.

December 26 2012

Big, open and more networked than ever: 10 trends from 2012

In 2012, technology-accelerated change around the world was driven by the wave of social media, data and mobile devices. In this year in review, we look back at some of the stories that mattered here at Radar and look ahead to what’s in store for 2013.

Below, you’ll find 10 trends that held my interest in 2012. This is by no means a comprehensive account of “everything that mattered in the past year” — try The Economist’s account of the world in 2012 or The Atlantic’s 2012 in review or Popular Science’s “year in ideas” if you’re hungry for that perspective — but I hope you’ll find something new to think about as 2013 draws near.

Social media

Social media wasn’t new in 2012, but it was bigger and more mainstream than ever. There were some firsts, from the first Presidential “Ask Me Anything” on Reddit to the first White House Google Hangout on Google Plus to presidential #debates to the first billion-user social network. The election season had an unprecedented social and digital component, from those hyperwired debates to a presidential campaign built like a startup. Expect even more blogging, tweeting, tumbling, streaming, Liking and pinning in 2013, even if it leaves us searching for context.

Open source in government

Open source software made more inroads in the federal government, from a notable policy at the Consumer Financial Protection Agency to more acceptance in the military.

The White House made its first commits on GitHub, including code for its mobile apps and e-petition platform, where President Obama responded personally to an e-petition for the first time.. The House Oversight Committee’s crowdsourced legislative platform  also went on GitHub. At year’s end, the United States (code) was on GitHub.

Responsive design

According to deputy technical lead Jeremy Vanderlan, the new AIDS.gov, launched in June, was the first full-site implementation of responsive web design for a federal government domain. They weren’t the first to automatically adapt how a website is displayed for the device a visitor is using — you can see next-generation web design at open.nasa.gov or in the way that fcc.gov/live optimizes to provide video to different mobile devices — but this was a genuine milestone for the feds online. By year’s end, Congress had also become responsive, at least with respect to its website, with a new beta at Congress.gov.

Free speech online

Is there free speech on the Internet? As Rebecca MacKinnon, Ethan Zuckerman and others have been explaining for years, what we think of as the new “public square online” is complicated by the fact that these platforms for free expression are owned and operated by private companies. MacKinnon explored these issues, “Consent of the Networked,” one of best technology policy books of the year. In 2012, “Twitter censorship” and the Terms of Service for social networking services caused many more people to suggest a digital Bill of Rights, although “Internet freedom” is an idea that varies with the beholder.

Open mapping

On January 9th, I wondered whether 2012 would be “the year of the open map.” I started reporting on digital maps made with powerful new software and open data last winter. The prediction was partially born out, from Foursquare’s adoption to StreetEast moving from Google Maps to new investments in OpenStreetMap. In response to the shift, Google slashed its price for using the Google Maps API by 88%. In an ideal world, the new competition will result in both better maps and more informed citizens.

Data journalism

Data journalism took on new importance for society. We tracked its growing influence, from the Knight News Challenge to new research initiatives to Africa, and are continuing to investigate data journalism with a series of interviews and a forthcoming report.

Privacy and security

Privacy and security continued to dominate technology policy discussions in the United States, although copyright, spectrum, patents and Internet governance had significant prominence. While the Supreme Court decided GPS monitoring constitutes search under the 4th Amendment, expanded rules for data sharing in the U.S. government raised troubling questions.

In another year that will end without updated baseline privacy legislation from Congress, bills did advance in the U.S. Senate to reform electronic privacy and address location-based technology. After calling for such legislation, the Federal Trade Commission opened an investigation into data brokers.

No “cyber security” bill passed the Senate either, leaving hope that future legislation will balance protections with civil liberties and privacy concerns.

Networked politics

Politics were more wired in Election 2012 than they’d ever been in history, from social media and debates to the growing clout of the Internet. The year started off with the unprecedented wave of networked activism that stopped the progress of the Stop Online Piracy Act (SOPA) and PROTECT-IP Act (PIPA) in Congress.

At year’s end, the jury remains out on whether the Internet will act as a platform for collective action to address societal challenges, from addressing gun violence in the U.S. to a changing climate.

Open data

As open data moves from the information age to the action age, there are significant advances around the globe. As more data becomes available, its practical application has only increased in importance.

After success releasing health care data to fuel innovation and startups, US CTO Todd Park sought to scale open data and agile thinking across the federal government.

While it’s important to be aware of the ambiguity of open government and open data, governments are continuing to move forward globally, with the United Kingdom relaunching Data.gov.uk and, at year’s end, India and the European Commission launching open data platforms. Cities around the world also adopted open data, from Buenos Aires to Berlin to Palo Alto.

In the United States, friendly competition to be the nation’s preeminent digital city emerged between San Francisco, Chicago, Philadelphia and New York. Open data releases became a point of pride. Landmark legislation in New York City and Chicago’s executive order on open data made both cities national leaders.

As the year ends, we’re working to make dollars and sense of the open data economy, explicitly making a connection between releases and economic growth. Look for a report on our research in 2013.

Open government

The world’s largest democracy officially launching an open government data platform was historic. That said, it’s worth reiterating a point I’ve made before: Simply opening up data is not a replacement for a Constitution that enforces a rule of law, free and fair elections, an effective judiciary, decent schools, basic regulatory bodies or civil society — particularly if the data does not relate to meaningful aspects of society. Adopting open data and digital government reforms is not quite the same thing as good government. Beware openwashing in government, as well as in other areas.

On that count, at year’s end, The Economist found that global open government efforts are growing in “scope and clout.” The Open Government Partnership grew, with new leadership, added experts and a finalized review mechanism. The year to come will be a test of the international partnership’s political will.

In the United States, an open government reality check at the federal level showed genuine accomplishments, but it leaves many promises only partially fulfilled, with a mixed record on meeting goals that many critics found transparently disappointing. While some of the administration’s transparency failures concern national security — notably, the use of drones overseas — science journalists reported restricted access to administration officials at the Environmental Protection Agency, Federal Drug Administration and Department of Health and Human Services.

Efforts to check transparency promises also found compliance with the Freedom of Information Act lacking. While a new FOIA portal is promising, only six federal agencies were on it by year’s end. The administration record on prosecuting whistleblowers has also sent a warning to others considering coming forward regarding waste or abuse in the national security.

Despite those challenges, 2012 was a year of continuing progress for open government at the federal level in the United States, with reasons for hope throughout states and cities. Here’s hoping 2013 sees more advances than setbacks in this area.

Coming tomorrow: 14 trends to watch in 2013.

Reposted bycheg00 cheg00

December 21 2012

Six ways data journalism is making sense of the world, around the world

When I wrote that Radar was investigating data journalism and asked for your favorite examples of good work, we heard back from around the world.

I received emails from Los Angeles, Philadelphia, Canada and Italy that featured data visualization, explored the role of data in government accountability, and shared how open data can revolutionize environmental reporting. A tweet pointed me to a talk about how R is being used in the newsroom. Another tweet linked to relevant interviews on social science and the media:

Two of the case studies focused on data visualization, an important practice that my colleague Julie Steele and other editors at O’Reilly Media have been exploring over the past several years.

Several other responses are featured at more length below. After you read through, make sure to also check out this terrific Ignite talk on data journalism recorded at this year’s Newsfoo in Arizona.

Visualizing civic health

Meredith Broussard, a professor at the University of Pennsylvania, sent us a link to a recent data journalism project she did for Hidden City Philadelphia, which won an award from the National Council on Citizenship and the Knight Foundation. The project, measuring Philadelphia’s civic health, won honorable mention in Knight’s civic data challenge. Data visualization was a strong theme among the winners of that challenge.

Data journalism in PhiladelphiaData journalism in Philadelphia

Mapping ambulance response times

I profiled the data journalism work of The Los Angeles Times earlier this year, when I interviewed news developer Ben Welsh about the newspaper’s Data Desk, a team of reporters and web developers that specializes in maps, databases, analysis and visualization.

Recently, the Data Desk made an interactive visualization that mapped how fast the Los Angeles Fire Department responds to calls.

LA Times fire response timesLA Times fire response times

Visualizing UK government spending

The Guardian Datablog is one of the best sources of interesting, relevant data journalism work, from sports to popular culture to government accountability. Every post demonstrates an emerging practice when its editors make it possible for readers to download the data themselves. Earlier this month, the Datablog put government spending in the United Kingdom under the microscope and accompanied it with a downloadable graphic (PDF).

The Guardian’s data journalism is particularly important as the British government continues to invest in open data. In June, the United Kingdom’s Cabinet Office relaunched Data.gov.uk and released a new open data white paper. The British government is doubling down on the notion that open data can be a catalyst for increased government transparency, civic utility and economic prosperity. The role of data journalism in delivering those outcomes is central.

(Note: A separate Radar project is digging into the open data economy.)

An Italian data job

The Italian government, while a bit behind the pace set in the UK, has made more open data available since it launched a national platform in 2011.

Elisabetta Tola, an Italian data journalist, wrote in to share her work on a series of Wired Magazine articles that feature data on seismic risk assessment in Italian schools. The interactive lets parents search for schools, a feature that embodies service journalism and offers more value than a static map.

Italian schools and earthquakes visualizationItalian schools and earthquakes visualization

Tola highlighted a key challenge in Italy that exists in many other places around the world: How can data journalism be practiced in countries that do not have a Freedom of Information Act or a tradition of transparency on government actions and spending? If you have ideas, please share them in the comments or email me.

Putting satellite imagery to work

Brazil, by way of contrast, notably passed a freedom of information law this past year, fulfilling one of its commitments to the Open Government Partnership.

Earlier this year, when I traveled to Brazil to moderate a panel at the historic partnership’s annual meeting, I met Gustavo Faleiros, a journalist working with open data focusing on the Amazon rainforest. Faleiros is as a Knight International Journalism Fellow, in partnership with Washington-based organizations International Center for Journalists and Internews. Today, Faleiros continues that work as the project coordinator for InfoAmazonia.org, a beautiful mashup of open data, maps and storytelling.

Faleiros explained that the partnership is training Brazilian journalists to use satellite imagery and collect data related to forest fires and carbon monoxide. He shared this video that shows a data visualization that came out of that work:

As 2012 comes to an end, the rate of Amazon deforestation has dropped to record lows. These tools help the world see what’s happening from high above.

Data-driven broadcast journalism?

I also heard about work in much colder climes when Keith Robinson wrote in from Canada. “As part of large broadcast organizations, one thing that is very satisfying about data journalism is that it often puts our digital staff in the driver’s seat — what starts as an online investigation often becomes the basis for original and exclusive broadcast content,” he wrote in an email.

Robinson, the senior producer for specials and interactive at Global News in Canada, highlighted several examples of their Data Desk’s work, including:

Robinson expects 2013 will see further investment and expansion in the data journalism practice at Global News.

Robinson also pointed to a practice that media should at least consider adopting: Global News is not only consuming and displaying open data, but also publishing the data they receive from the Canadian government. “As we make access to information requests, we’re trying to make the data received available to the public,” he wrote.

From the big picture to next steps

It was instructive to learn more about the work of two large media organizations, the Los Angeles Times and Canada’s Global News, which have been building their capacity to practice data journalism. The other international perspectives in my inbox and tweet stream, however, were a reminder that big-city newsrooms that can afford teams of programmers and designers aren’t the only players here.

To put it another way, acts of data journalism by small teams or individuals aren’t just plausible, they’re happening — from Italy to Brazil to Africa. That doesn’t mean that the news application teams at NPR, The Guardian, ProPublica or the New York Times aren’t setting the pace for data journalism when it comes to cutting edge work — far from it — but the tools and techniques to make something worthwhile are being democratized.

That’s possible in no small part because of the trend toward open source tools and social coding I’m seeing online, from Open Street Map to more open elections.

It’s a privilege to have a global network to tap into for knowledge and, in the best moments, wisdom. Thank you — and please keep the responses coming, whether you use email, Twitter or the phone. Your input is helping shape a report I’m developing that ties together our coverage of data journalism. Look for that to be published early in the new year.

Related

Reposted bycheg00 cheg00

November 29 2012

As digital disruption comes to Africa, investing in data journalism takes on new importance

This interview is part of our ongoing look at the people, tools and techniques driving data journalism.

I first met Justin Arenstein (@justinarenstein) in Chişinău, Moldova, where the media entrepreneur and investigative journalist was working as a trainer at a “data boot camp” for journalism students. The long-haired, bearded South African instantly makes an impression with his intensity, good humor and focus on creating work that gives citizens actionable information.

Justin ArensteinJustin ArensteinWhenever we’ve spoken about open data and open government, Arenstein has been a fierce advocate for data-driven journalism that not only makes sense of the world for readers and viewers, but also provides them with tools to become more engaged in changing the conditions they learn about in the work.

He’s relentlessly focused on how open data can be made useful to ordinary citizens, from Africa to Eastern Europe to South America. For instance, in November, he highlighted how data journalism boosted voter registration in Kenya, creating a simple website using modern web-based tools and technologies.

For the last 18 months, Arenstein has been working as a Knight International Fellow embedded with the African Media Initiative (AMI) as a director for digital innovation. The AMI is a group of the 800 largest media companies on the continent of Africa. In that role, Arenstein has been creating an innovation program for the AMI, building more digital capacity in countries that are as in need of effective accountability from the Fourth Estate as any in the world. That disruption hasn’t yet played itself out in Africa because of a number of factors, explained Arenstein, but he estimates that it will be there within five years.

“Media wants to be ready for this,” he said, “to try and avoid as much of the business disintegration as possible. The program is designed to help them grapple with and potentially leapfrog coming digital disruption.”

In the following interview, Arenstein discusses the African media ecosystem, the role of Hacks/Hackers in Africa, and expanding the capacity of data journalism.

Why did you adopt the Hacks/Hackers model and scale it? Why is it relevant to what’s happening around Africa?

Justin Arenstein: African journalists are under-resourced but also poorly trained, probably even more so than in the U.S. and elsewhere. Very, very few of them have any digital skills, never mind coding skills. Simply waiting for journalists to make the leap themselves and start learning coding skills and more advanced digital multimedia content production skills is just too — well, we don’t have enough time to do that, if we’re going to beat this disruption that’s coming.

The idea was to clone parts of the basic model of Hacks/Hackers from the U.S., which is a voluntary forum and society where journalists, UI people, designers, graphics people and coders meet up on a regular basis.

Unlike in the U.S., where Hacks/Hackers is very focused on startup culture, the African chapters have been very focused on data-driven journalism and imparting some basic skills. We’re trying to avoid some of the pitfalls experienced in the U.S. and get down to using data as a key tool in creating content. A big weakness in a lot of African media is that there’s very little unique content, firstly, and that the unique content that is available is not particularly well produced. It’s not deep. It’s not substantiated. It’s definitely not linked data.

We’ve been focusing on improving the quality of the content so that the companies where these journalists work will be able to start weaning themselves from some of the bad business practices that they are guilty of and start concentrating on building up their own inventory. That’s worked really well in some of the African countries along the coastlines where there’s data access, because you’ve got cables coming in. In the hinterland of Africa, data and Internet are not widely available. The Hacks/Hackers chapters there have been more like basic computer-assisted reporting training organizations.

Like in the U.S., they all run themselves. But unlike in the U.S., we have a structured agenda, a set of protocols, an operating manual, and we do subsidize each of the chapters to help them meet the physical needs of cost. They’re not quite as voluntary as the U.S. ones; it’s a more formal structure. That’s because they’re designed to surface good ideas, to bring together a challenge that you wouldn’t ordinarily find in the media ecosystem at least, and then to help kick-start experimentation.

Do you see any kind of entrepreneurial activity coming out of them now?

Justin Arenstein: I’m not aware of any notable startups. We’ve had ideas where people are collaborating to build toward startups. I haven’t seen any products launched yet, but what we have seen is journalist-led startups that were outside of these Hacks/Hackers chapters now starting to come into the fold.

Why? Because this is where they can find some of the programming and engineering skills that they need, that they were struggling to find outside of the ecosystem. They are finding engineers or programmers, at least, but they’re not finding programmers who are tuned to content needs or to media philosophies and models. There’s a better chance that they’ll find those inside of these chapters.

The chapters are fairly young, though. The oldest chapter is about six months old now, and still fairly small. We’re nowhere near the size of some of the Latin American chapters. We have forged very strong links with them, and we follow their model a lot more closely than the U.S. model. The biggest chapter is probably about 150 members. They all meet, at a minimum, once a month. Interestingly, they are becoming the conduits not just for hackathons and “scrape-a-thons,” but are also now our local partners for implementing thinks like our data boot camps.

Those are week-long, intensive hands-on experiential training, where we’re flying in people from the Guardian data units, the Open Knowledge Foundation and from Google. We’re actually finding the guys behind Google Refine and Google Fusion Tables and flying in some of those people, so they can see end-users in a very different environment to what they’re used to. People walk into those boot camps not knowing what a spreadsheet is and, by the end of it, they’re producing their first elementary maps and visualizations. They’re crunching data.

What stories have “data boot camp” participants produced afterward?

Justin Arenstein: Here’s an example. We had a boot camp in Kenya. NTV, the national free-to-air station, had been looking into why young girls in a rural area of Kenya did very well academically until the ages of 11 or 12 — and then either dropped off the academic record completely or their academic performance plummeted. The explanation by the authorities and everyone else was that this was simply traditional; it’s tribal. Families are pulling them out of school to do chores and housework, and as a result, they can’t perform.

Irene Choge [a Kenyan boot camp participant who attended data journalism training] started mining the data. She came from that area and knew it wasn’t that [cause]. So she looked into public data. She first assumed it was cholera, so she looked into medical records. Nothing there. She then looked into water records. From water, she started looking into physical infrastructure and public works. She discovered these schools had no sanitation facilities and that the schools with the worst performing academics were those that didn’t have sanitation facilities, specifically toilets.

What’s the connection?

Justin Arenstein: When these girls start menstruating, there’s nowhere for them to go to attend to themselves, other than into the bushes around the school. They were getting harassed and embarrassed. They either stopped going to school completely or they would stop going during that part of their cycle and, as a result, their schoolwork suffered dramatically. She then produced a TV documentary that evoked widespread public outcry and changed policies.

In addition to that, her newsroom is working on building an app. A parent who watches this documentary and is outraged will then be able to use the app to find out what’s happening at their daughter’s school. If their daughter’s school is one of those that has no facilities, the app then helps them through a text-based service to sign a petition and petition the responsible official to improve the situation, as well as link up with other outraged parents. It mobilizes them.

What we liked about her example was that it was more than just doing a visualization, which is what people think about when you say “data journalism.”

First, she used data tools to find trends and stories that had been hidden to solve a mystery. Secondly, she then did real old-fashioned journalism and went out in the field and confirmed the data wasn’t lying. The data was accurate.

Thirdly, she then used the data to give people the tools to actually act on the information. She’s using open data and finding out in your district, this is your school, this is how you impact it, this is the official you should be emailing or writing to about it. That demonstrates that, even in a country where most people access information through feature phones, data can still have a massive impact at grassroots level.

These are the kinds of successes that we are looking for in these kinds of outreach programs when it comes to open data.

How does the practice of data-driven journalism or the importance of computer-assisted reporting shift when a reporter can’t use rich media or deploy bandwidth-heavy applications?

Justin Arenstein: We’re finding something that maybe you’re starting to see inklings of elsewhere as well: data journalism doesn’t have to be the product. Data journalism can also be the route that you follow to get to a final story. It doesn’t have to produce an infographic or a map.

Maps are very good ways to organize information. They’re very poor mechanisms for consuming information. No one kicks back on a Sunday afternoon laying on their sofa, reading a map, but if a map triggers geofenced information and pushes relevant local information at you in your vicinity, then it becomes a useful mechanism.

What we’re doing in newsrooms is around investigative journalism. For example, we’re funding projects around extractive industries. We’re mapping out conversations and relationships between people. We’re then using them as analytical tools in the newsroom to arrive at better, deeper and evidence-driven reporting, which is a major flaw and a major weakness in many African media.

What capacity needs to be built in these areas? What are people doing now? What matters most?

Justin Arenstein: Investigative journalism in Africa, like in many other places, tends to be scoop-driven, which means that someone’s leaked you a set of documents. You’ve gone and you’ve verified them and often done great sleuth work. There are very few systematic, analytical approaches to analyzing broader societal trends. You’re still getting a lot of hit-and-run reporting. That doesn’t help us analyze the societies we’re in, and it doesn’t help us, more importantly, build the tools to make decisions.

Some of the apps that we are helping people build, based off of their reporting, are invariably not visualizations. They’re rather saying, “Let’s build a tool that augments the reporting, reflects the deeper data that the report is based on, and allows people to use that tool to make a personal decision.” It’s engendering action.

A lot of the fantastic work you’ve seen from people at the Guardian and others has been about telling complex stories simply via infographics, which is a valid but very different application of data journalism.

I think that, specifically in East Africa and in Southern Africa, there’s growing recognition that the media are important stewards of historical data. In many of these societies, including industrialized societies like South Africa, the state hasn’t been a really good curator of public data and public information because of their political histories.

Nation states don’t see data as an asset? Is that because technical capacity isn’t there? Or is that because data actually contains evidence of criminality, corruption or graft?

Justin Arenstein: It’s often ineptitude and lack of resources in South Africa’s instance. In a couple of other countries, it’s systematic purging of information that is perhaps embarrassing when there’s a change of regime or political system — or in the case of South Africa and many of the colonial countries, a simple unwillingness or lack of insight as to the importance of collecting data about second-class citizens, largely the black population.

The official histories are very thin. There’s nowhere near the depth of nuance or insight into a society that you would find in the U.S. or in Europe, where there’s been very good archival record keeping. Often, the media are the only people who’ve really been keeping that kind of information, in terms of news reportage. It’s not brilliant. It’s often not primary sources — it’s secondary. But the point is that often it’s the only information that’s available.

What we’re doing is working with media companies now to help digitize and turn reportage into structured data. In a vacuum, because there is no other data, suddenly it becomes an important commercial commodity. Anyone who wants to build, for example, a tourism app or a transport app, will find that there is no other information available. This may sound like a bizarre concept to most people living in data-rich countries, like the U.S., but you simply can’t find the content. That means that you have to then go out and create the content yourself before you can build the app.

Is this a different sort of a “data divide,” where a country is “data-poor?”

Justin Arenstein: Well, maybe digitally “data poor,” because what we are doing is we’re saying that there is data. We initially also had the same reaction, saying “there is no data here,” and then realized that there’s a hell of a lot of data. Invariably, it’s locked up in deadwood format. So [we're now] liberating that data, digitizing it, structuring it, and then making sure that it’s available for people to use.

How much are media entities you work with making data, as opposed to just digitizing?

Justin Arenstein: Some are making data. We haven’t, because a lot of other actors are involved in citizen data creation. We haven’t really focused too many of our very scarce resources on that component yet.

We are funding a couple of citizen reporting apps, because there’s a lot of hype around citizen data and we’re trying to see if there are models that can really work where you create credible, sourced and actionable information. We don’t believe that you’re going to be able to do that just from text messaging. We’re looking at alternative kinds of interfaces and methods for transmitting information.

Are there companies and startups that are consuming the digital data that you’re producing? If so, what are they doing?

Justin Arenstein: Outside of the News Challenge, we are co-founding something with the World Bank called “Code for Kenya.” It’s modeled fairly closely on the Mozilla Open Use Fellowships, with a few tweaks. It’s maybe a hybrid of Code for America and the Mozilla Open Fellowships.

Where Code for America focuses on cities and Mozilla focuses on newsrooms, we’ve embedded open data strategists and evangelists into the newsrooms, backed up by an external development team at a civic tech lab. They’re structuring the data that’s available, such as turning old microfiche rolls into digital information, cleaning it up and building a data disk. They’re building news APIs and pushing the idea that rather than building websites, design an API specifically for third-party repurposing of your content. We’re starting to see the first early successes. Four months in, some of the larger media groups in Kenya are now starting to have third-party entrepreneurs developing using their content and then doing revenue-share deals.

The only investment from the data holder, which is the media company, is to actually clean up the data and then make it available for development. Now, that’s not a new concept. The Guardian in the United Kingdom has experimented with it. It’s fairly exciting for these African companies because there’s potentially — and arguably, larger — appetite for the content because there’s not as much content available. Suddenly, the unit cost of value of that data is far higher than it might be in the U.K. or in the U.S.

Media companies are seriously looking at it as one of many potential future revenue streams. It enables them to repurpose their own data, start producing books and the rest of it. There isn’t much book publishing in Africa, by Africans, for Africans. Suddenly, if the content is available in an accessible format, it gives them an opportunity to mash-up stuff and create new kinds of books.

They’ll start seeing that content itself can be a business model. The impact that we’re seeking there is to try and show media companies that investing in high-quality unique information actually gives you a long-term commodity that you can continue to reap benefits from over time. Whereas simply pulling stuff off the wire or, as many media do in Africa, simply lifting it off of the web, from the BBC or elsewhere, and crediting it, is not a good business model.

Photo via International Center for Journalists.

Related:

November 26 2012

Investigating data journalism

Great journalism has always been based on adding context, clarity and compelling storytelling to facts. While the tools have improved, the art is the same: explaining the who, what, where, when and why behind the story. The explosion of data, however, provides new opportunities to think about reporting, analysis and publishing stories.

As you may know, there’s already a Data Journalism Handbook to help journalists get started. (I contributed some commentary to it). Over the next month, I’m going to be investigating the best data journalism tools currently in use and the data-driven business models that are working for news startups. We’ll then publish a report that shares those insights and combines them with our profiles of data journalists.

Why dig deeper? Getting to the heart of what’s hype and what’s actually new and noteworthy is worth doing. I’d like to know, for instance, whether tutorials specifically designed for journalists can be useful, as Joe Brockmeier suggested at ReadWrite. On a broader scale, how many data journalists are working today? How many will be needed? What are the primary tools they rely upon now? What will they need in 2013? Who are the leaders or primary drivers in the area? What are the most notable projects? What organizations are embracing data journalism, and why?

This isn’t a new interest for me, but it’s one I’d like to found in more research. When I was offered an opportunity to give a talk at the second International Open Government Data Conference at the World Bank this July, I chose to talk about open data journalism and invited practitioners on stage to share what they do. If you watch the talk and the ensuing discussion in the video below, you’ll pick up great insight from the work of the Sunlight Foundation, the experience of Homicide Watch and why the World Bank is focused on open data journalism in developing countries.

The sites and themes that I explored in that talk will be familiar to Radar readers, focusing on the changing dynamic between the people formerly known as the audience and the editors, researchers and reporters who are charged with making sense of the data deluge for the public good. If you’ve watched one of my Ignites or my Berkman Center talk, much of this won’t be new to you, but the short talk should be a good overview of where I think this aspect of data journalism is going and why I think it’s worth paying attention to today.

For instance, at the Open Government Data Conference Bill Allison talked about how open data creates government accountability and reveals political corruption. We heard from Chris Amico, a data journalist who created a platform to help a court reporter tell the story of every homicide in a city. And we heard from Craig Hammer how the World Bank is working to build capacity in media organizations around the world to use data to show citizens how and where borrowed development dollars are being spent on their behalf.

The last point, regarding capacity, is a critical one. Just as McKinsey identified a gap between available analytic talent and the demand created by big data, there is a data science skills gap in journalism. Rapidly expanding troves of data are useless without the skills to analyze it, whatever the context. An over focus on tech skills could exclude the best candidates for these jobs — but there will need to be training to build them.

This reality hasn’t gone unnoticed by foundations or the academy. In May, the Knight Foundation gave Columbia University $2 million for research to help close the data science skills gap. (I expect to be talking to Emily Bell, Jonathan Stray and the other instructors and students.)

Media organizations must be able to put data to work, a need that was amply demonstrated during Hurricane Sandy, when public open government data feeds became critical infrastructure.

What I’d like to hear from you is what you see working around the world, from the Guardian to ProPublica, and what you’re working on, and where. To kick things off, I’d like to know which organizations are doing the most innovative work in data journalism.

Please weigh in through the comments or drop me a line at alex@oreilly.com or at @digiphile on Twitter.

September 28 2012

Four key trends changing digital journalism and society

See something or say something: Los AngelesSee something or say something: Los AngelesIt’s not just a focus on data that connects the most recent class of Knight News Challenge winners. They all are part of a distributed civic media community that works on open source code, collects and improves data, and collaborates across media organizations.

These projects are “part of an infrastructure that helps journalists better understand and serve their communities through data,” commented Chris Sopher, Knight Foundation Journalism Program Associate, in an interview last week. To apply a coding metaphor, the Knight Foundation is funding the creation of patches for the source code of society. This isn’t a new focus: in 2011, Knight chose to help build the newsroom stack, from editorial search engines to data cleaning tools.

Following are four themes that jumped out when I looked across the winners of the latest Knight News Challenge round.

Networked accountability

An intercontinental project that bridged citizen science, open data, open source hardware, civic hacking and the Internet of things to monitor, share and map radiation data? Safecast is in its own category. Adapting the system to focus on air quality in Los Angeles — a city that’s known for its smog — will be an excellent stress test for seeing if this distributed approach to networked accountability can scale.

If it does — and hacked Chumbys, LED signs, Twitter bots, smartphone apps and local media reports start featuring the results — open data is going to be baked into how residents of Los Angeles understand their own atmosphere. If this project delivers on some of its promise, the value of this approach will be clearer.

If this project delivers on all of its potential, the air itself might improve. For that to happen, the people who are looking at the realities of air pollution will need to advocate for policy makers to improve it. In the future, the success or failure of this project will inform similar efforts that seek to enlist communities in data collection, including whether governments embrace “citizensourcing” beyond natural disasters and crises. The idea of citizens as sensors continues to have legs.

Peer-to-peer collaboration, across newsrooms

As long as I’ve been reading newspapers, watching television news and following the industry, competition has always been part of the dynamic: be first to the scene, first to get the scoop, first to call the election. As the Internet has taken on a larger role in delivering the news, there have been new opportunities for competition in digital journalism: first to tweet, post or upload video, often followed by rewards from online traffic.

One (welcome) reality that jumps out in this series of Knight grants is that there are journalists from newsrooms that compete for stories who are collaborating on these projects independently. New York Times and Washington Post developers are teaming up to create an open election database. Data journalists from WNYC, the Chicago Tribune and the Spokesman-Review are collaborating on building a better interface for Census data. The same peer networks that helped build the Internet are forming around building out civic infrastructure. It’s an inspiring trend to watch.

The value of an open geo commons

The amount of consternation regarding Apple’s new mapping app for iOS 6 doesn’t seem to be dying down. It shouldn’t: David Pogue called the Apple Map app “an appalling first release,” and maybe “the most embarrassing, least usable piece of software Apple has ever unleashed.” It’s going to take a while for Apple Maps to improve — maybe even years, based upon how long it took for Google to improve maps. In the meantime, iPhone users can go to maps.google.com on Safari, along with the other third-party alternatives that Apple CEO Tim Cook recommended in his letter of apology.

In the wake of “#MAppleGate,” there’s suddenly a lot more attention being paid to the importance and value of mapping data, including how difficult it is to do maps right. And that’s where OpenStreetMap comes in. That’s also why the Knight Foundation is putting more than $500,000 behind tools from Development Seed: it will help to sustain and improve an open geo data commons that media organizations large and small can tap into to inform communities using maps.

“There are two ways the geo data space is going to evolve: 1) in closed silos of proprietary owned data or 2) in the open,” said Eric Gundersen, co-founder and CEO of Development Seed in a recent interview. “Our community does not need a fleet of cars driving millions of miles. We need good infrastructure to make it easy for people to map their surroundings and good community tools to help us garden the data and improve quality. As geo data becomes core to mobile, maps are a canvas to visualizing the ‘where’.”

As with Wikipedia, there will be people who doubt whether an open source digital map revolution enabled by MapBox, Development Seed’s open source mapping suite will come to pass. Then again, how many people believed a decade ago that Wikipedia would grow into the knowledge repository it is today?

“We are trying to radically lower the barrier of entry to map making for organizations and activists,” Gundersen told me last April. Given that they’re up against Google in mapmaking, the relatively tiny DC startup is banking on OpenStreetMap looking more like Wikipedia than Google Knol in a few years.

“Open” is in

Open data is a common thread that connects the winners — but the openness doesn’t stop there. Open maps. Open source. Open government. Open journalism. That this theme has emerged as a strong pulse isn’t a tremendous surprise, given a global movement to apply technology to open government. Moreover, no one should take this to mean that immense amounts of business, society, technology, media and government aren’t still closed. Clearly, that’s not the situation. But there’s a strong case to be made that open is the way of the day.

Data won’t save the world, on its own. However, when data is applied for the public good and put to work, there are a growing number of examples that raise optimism about data’s role in the future of journalism.

Photo Credit: Eric Fisher

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl