Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

June 14 2013

Four short links: 14 June 2014

  1. How Geeks Opened up the UK Government (Guardian) — excellent video introduction to how the UK is transforming its civil service to digital delivery. Most powerful moment for me was scrolling through various depts’ web sites and seeing consistent visual design.
  2. Tools for Working Remotely — Braid’s set of tools (Trello, Hackpad, Slingshot, etc.) for remote software teams.
  3. Git Push to Deploy on Google App EngineEnabling this feature will create a remote Git repository for your application’s source code. Pushing your application’s source code to this repository will simultaneously archive the latest the version of the code and deploy it to the App Engine platform.
  4. Amazon’s 3D Printer Store — printers and supplies. Deeply underwhelming moment of it arriving on the mainstream.

April 18 2013

Sprinting toward the future of Jamaica

Creating the conditions for startups to form is now a policy imperative for governments around the world, as Julian Jay Robinson, minister of state in Jamaica’s Ministry of Science, Technology, Energy and Mining, reminded the attendees at the “Developing the Caribbean” conference last week in Kingston, Jamaica.

photo-22photo-22

Robinson said Jamaica is working on deploying wireless broadband access, securing networks and stimulating tech entrepreneurship around the island, a set of priorities that would have sounded of the moment in Washington, Paris, Hong Kong or Bangalore. He also described open access and open data as fundamental parts of democratic governance, explicitly aligning the release of public data with economic development and anti-corruption efforts. Robinson also pledged to help ensure that Jamaica’s open data efforts would be successful, offering a key ally within government to members of civil society.

The interest in adding technical ability and capacity around the Caribbean was sparked by other efforts around the world, particularly Kenya’s open government data efforts. That’s what led the organizers to invite Paul Kukubo to speak about Kenya’s experience, which Robinson noted might be more relevant to Jamaica than that of the global north.

Kukubo, the head of Kenya’s Information, Communication and Technology Board, was a key player in getting the country’s open data initiative off the ground and evangelizing it to developers in Nairobi. At the conference, Kukubo gave Jamaicans two key pieces of advice. First, open data efforts must be aligned with national priorities, from reducing corruption to improving digital services to economic development.

“You can’t do your open data initiative outside of what you’re trying to do for your country,” said Kukubo.

Second, political leadership is essential to success. In Kenya, the president was personally involved in open data, Kukubo said. Now that a new president has been officially elected, however, there are new questions about what happens next, particularly given that pickup in Kenya’s development community hasn’t been as dynamic as officials might have hoped. There’s also a significant issue on the demand-side of open data, with respect to the absence of a Freedom of Information Law in Kenya.

When I asked Kukubo about these issues, he said he expects a Freedom of Information law will be passed this year in Kenya. He also replied that the momentum on open data wasn’t just about the supply side.

“We feel that in the usage side, especially with respect to the developer ecosystem, we haven’t necessarily gotten as much traction from developers using data and interpreting cleverly as we might have wanted to have,” he said. “We’re putting putting more into that area.”

With respect to leadership, Kukubo pointed out that newly elected Kenyan President Uhuru Kenyatta drove open data release and policy when he was the minister of finance. Kukubo expects him to be very supportive of open data in office.

The development of open data in Jamaica, by way of contrast, has been driven by academia, said professor Maurice McNaughton, director of the Center of Excellence at the Mona School of Business at the University of the West Indies (UWI). The Caribbean Open Institute, for instance, has been working closely with Jamaica’s Rural Agriculture Development Authority (RADA). There are high hopes that releases of more data from RADA and other Jamaican institutions will improve Jamaica’s economy and the effectiveness of its government.

Open data could add $35 million annually to the Jamaican economy, said Damian Cox, director of the Access to Information Unit in the Office of the Prime Minister, citing a United Nations estimate. Cox also explicitly aligned open data with measuring progress toward Millennium Development Goals, positing that increasing the availability of data will enable the civil society, government agencies and the UN to more accurately assess success.

The development of (open) data-driven journalism

Developing the Caribbean focused on the demand side of open data as well, particularly the role of intermediaries in collecting, cleaning, fact checking, and presenting data, matched with necessary narrative and context. That kind of work is precisely what data-driven journalism does, which is why it was one of the major themes of the conference. I was invited to give an overview of data-driven journalism that connected some trends and highlighted the best work in the field.

I’ve written quite a bit about how data-driven journalism is making sense of the world elsewhere, with a report yet to come. What I found in Jamaica is that media there have long since begun experimenting in the field, from the investigative journalism at Panos Caribbean to the relatively recent launch of diGJamaica by the Gleaner Company.

diGJamaica is modeled upon the Jamaican Handbook and includes more than a million pages from The Gleaner newspaper, going back to 1834. The site publishes directories of public entities and public data, including visualizations. It charges for access to the archives.

Legends and legacies

Usain Bolt in JamaicaUsain Bolt in Jamaica

Olympic champion Usain Bolt, photographed in his (fast) car at the UWI/Usain Bolt Track in Mona, Jamaica.

Normally, meeting the fastest man on earth would be the most memorable part of any trip. The moment that left the deepest impression from my journey to the Caribbean, however, came not from encountering Usain Bolt on a run but from within a seminar room on a university campus.

As a member of a panel of judges, I saw dozens of young people present after working for 30 hours at a hackathon at the University of the West Indies. While even the most mature of the working apps was still a prototype, the best of them were squarely focused on issues that affect real Jamaicans: scoring the risk of farmers that needed banking loans and collecting and sharing data about produce.

The winning team created a working mobile app that would enable government officials to collect data at farms. While none of the apps are likely to be adopted by the agricultural agency in its current form, or show up in the Google Play store this week, the experience the teams gained will help them in the future.

As I left the island, the perspective that I’d taken away from trips to Brazil, Moldova and Africa last year was further confirmed: technical talent and creativity can be found everywhere in the world, along with considerable passion to apply design thinking, data and mobile technology to improve the societies people live within. This is innovation that matters, not just clones of popular social networking apps — though the judges saw more than a couple of those ideas flow by as well.

In the years ahead, Jamaican developers will play an important role in media, commerce and government on the island. If attracting young people to engineering and teaching them to code is the long-term legacy of efforts like Developing the Caribbean, it will deserve its own thumbs up from Mr. Bolt. The track to that future looks wide open.

photo-23photo-23

Disclosure: the cost of my travel to Jamaica was paid for by the organizers of the Developing the Caribbean conference.

April 04 2013

Four short links: 4 April 2013

  1. geo-bootstrap — Twitter Bootstrap fork that looks like a classic geocities page. Because. (via Narciso Jaramillo)
  2. Digital Public Library of America — public libraries sharing full text and metadata for scans, coordinating digitisation, maximum reuse. See The Verge piece. (via Dan Cohen)
  3. Snake Robots — I don’t think this is a joke. The snake robot’s versatile abilities make it a useful tool for reaching locations or viewpoints that humans or other equipment cannot. The robots are able to climb to a high vantage point, maneuver through a variety of terrains, and fit through tight spaces like fences or pipes. These abilities can be useful for scouting and reconnaissance applications in either urban or natural environments. Watch the video, the nightmares will haunt you. (via Aaron Straup Cope)
  4. The Power of Data in Aboriginal Hands (PDF) — critique of government statistical data gathering of Aboriginal populations. That ABS [Australian Bureau of Statistics] survey is designed to assist governments, commentators or academics who want to construct policies that shape our lives or encourage a one-sided public discourse about us and our position in the Australian nation. The survey does not provide information that Indigenous people can use to advance our position because the data is aggregated at the national or state level or within the broad ABS categories of very remote, remote, regional or urban Australia. These categories are constructed in the imagination of the Australian nation state. They are not geographic, social or cultural spaces that have relevance to Aboriginal people. [...] The Australian nation’s foundation document of 1901 explicitly excluded Indigenous people from being counted in the national census. That provision in the constitution, combined with Section 51, sub section 26, which empowered the Commonwealth to make special laws for ‘the people of any race, other than the Aboriginal race in any State’ was an unambiguous and defining statement about Australian nation building. The Founding Fathers mandated the federated governments of Australia to oversee the disappearance of Aboriginal people in Australia.

January 31 2013

NASA launches second International Space Apps Challenge

From April 20 to April 21, on Earth Day, the second international Space Apps Challenge will invite developers on all seven continents to the bridge to contribute code to NASA projects.

space app challengespace app challenge

Given longstanding concerns about the sustainability of apps contests, I was curious about NASA’s thinking behind launching this challenge. When I asked NASA’s open government team about the work, I immediately heard back from Nick Skytland (@Skytland), who heads up NASA’s open innovation team.

“The International Space Apps Challenge was a different approach from other federal government ‘app contests’ held before,” replied Skytland, via email.

“Instead of incentivizing technology development through open data and a prize purse, we sought to create a unique platform for international technological cooperation though a weekend-long event hosted in multiple locations across the world. We didn’t just focus on developing software apps, but actually included open hardware, citizen science, and data visualization as well.”

Aspects of that answer will please many open data advocates, like Clay Johnson or David Eaves. When Eaves recently looked at apps contests, in the context of his work on Open Data Day (coming up on February 23rd), he emphasized the importance of events that build community and applications that meet the needs of citizens or respond to business demand.

The rest of my email interview with Skytland follows.

Why is the International Space Apps Challenge worth doing again?

Nick Skytland: We see the International Space Apps Challenge event as a valuable platform for the Agency because it:

  • Creates new technologies and approaches that can solve some of the key challenges of space exploration, as well as making current efforts more cost-effective.
  • Uses open data and technology to address global needs to improve life on Earth and in space.
  • Demonstrates our commitment to the principles of the Open Government Partnership in a concrete way.

What were the results from the first challenge?

Nick Skytland: More than 100 unique open-source solutions were developed in less then 48 hours.

There were 6 winning apps, but the real “results” of the challenge was a 2,000+ person community engaged in and excited about space exploration, ready to apply that experience to challenges identified by the agency at relatively low cost and on a short timeline.

How does this challenge contribute to NASA’s mission?

Nick Skytland: There were many direct benefits. The first International Space Apps Challenge offered seven challenges specific to satellite hardware and payloads, including submissions from at least two commercial organizations. These challenges received multiple solutions in the areas of satellite tracking, suborbital payloads, command and control systems, and leveraging commercial smartphone technology for orbital remote sensing.

Additionally, a large focus of the Space Apps Challenge is on citizen innovation in the commercial space sector, lowering the cost and barriers to space so that it becomes easier to enter the market. By focusing on citizen entrepreneurship, Space Apps enables NASA to be deeply involved with the quickly emerging space startup culture. The event was extremely helpful in encouraging the collection and dissemination of space-derived data.

As you know, we have amazing open data. Space Apps is a key opportunity for us to continue to open new data sources and invite citizens to use them. Space Apps also encouraged the development of new technologies and new industries, like the space-based 3D printing industry and open-source ROV (remote submersibles for underwater exploration.)

How much of the code from more than 200 “solutions” is still in use?

Nick Skytland: We didn’t track this last time around, but almost all (if not all) of the code is still available online, many of the projects continued on well after the event, and some teams continue to work on their projects today. The best example of this is the Pineapple Project, which participated in numerous other hackathons after the 2012 International Space Apps Challenge and just recently was accepted into the Geeks Without Borders accelerator program.

Of the 71 challenges that were offered last year, a low percentage were NASA challenges — about 13, if I recall correctly. There are many reasons for this, mostly that cultural adoption of open government philosophies within government is just slow. What last year did for us is lay the groundwork. Now we have much more buy-in and interest in what can be done. This year, our challenges from NASA are much more mission-focused and relevant to needs program managers have within the agency.

Additionally, many of the externally submitted challenges we have come from other agencies who are interested in using space apps as a platform to address needs they have. Most notably, we recently worked with the Peace Corps on the Innovation Challenge they offered at RHoK in December 2012, with great results.

The International Space Apps Challenge was a way for us not only to move forward technology development, drawing on the talents and initiative of bright-minded developers, engineers, and technologists, but also a platform to actually engage people who have a passion and desire to make an immediate impact on the world.

What’s new in 2013?

Nick Skytland: Our goal for this year is to improve the platform, create an even better engagement experience, and focus the collective talents of people around the world on develop technological solutions that are relevant and immediately useful.

We have a high level of internal buy-in at NASA and a lot of participation outside NASA, from both other government organizations and local leads in many new locations. Fortunately, this means we can focus our efforts on making this an meaningful event and we are well ahead of the curve in terms of planning to do this.

To date, 44 locations have confirmed their participation and we have six spots remaining, although four of these are reserved as placeholders for cities we are pursuing. We have 50 challenge ideas already drafted for the event, 25 of which come directly from NASA. We will be releasing the entire list of challenges around March 15th on spaceappschallenge.org.

We have 55 organizations so far that are supporting the event, including seven other U.S. government organizations, and international agencies. Embassies or consulates are either directly leading or hosting the events in Monterrey, Krakow, Sofia, Jakarta, Santa Cruz, Rome, London and Auckland.

 

December 06 2012

The United States (Code) is on Github

When Congress launched Congress.gov in beta, they didn’t open the data. This fall, a trio of open government developers took it upon themselves to do what custodians of the U.S. Code and laws in the Library of Congress could have done years ago: published data and scrapers for legislation in Congress from THOMAS.gov in the public domain. The data at github.com/unitedstates is published using an “unlicense” and updated nightly. Credit for releasing this data to the public goes to Sunlight Foundation developer Eric Mill, GovTrack.us founder Josh Tauberer and New York Times developer Derek Willis.

“It would be fantastic if the relevant bodies published this data themselves and made these datasets and scrapers unnecessary,” said Mill, in an email interview. “It would increase the information’s accuracy and timeliness, and probably its breadth. It would certainly save us a lot of work! Until that time, I hope that our approach to this data, based on the joint experience of developers who have each worked with it for years, can model to government what developers who aim to serve the public are actually looking for online.”

If the People’s House is going to become a platform for the people, it will need to release its data to the people. If Congressional leaders want THOMAS.gov to be a platform for members of Congress, legislative staff, civic developers and media, the Library of Congress will need to release structured legislative data. THOMAS is also not updated in real-time, which means that there will continue to be a lag between a bill’s introduction and the nation’s ability to read the bill before a vote.

Until that happens, however, this combination of scraping and open source data publishing offers a way forward on Congressional data to be released to the public, wrote Willis, on his personal blog:

Two years ago, there was a round of blog posts touched off by Clay Johnson that asked, “Why shouldn’t there be a GitHub for data?” My own view at the time was that availability of the data wasn’t as much an issue as smart usage and documentation of it: ‘We need to import, prune, massage, convert. It’s how we learn.’

Turns out that GitHub actually makes this easier, and I’ve had a conversion of sorts to the idea of putting data in version control systems that make it easier to view, download and report issues with data … I’m excited to see this repository grow to include not only other congressional information from THOMAS and the new Congress.gov site, but also related data from other sources. That this is already happening only shows me that for common government data this is a great way to go.

In the future, legislation data could be used to show iterations of laws and improve the ability of communities at OpenCongress, POPVOX or CrunchGov to discover and discuss proposals. As Congress incorporates more tablets on the floor during debates, such data could also be used to update legislative dashboards.

The choice to use Github as a platform for government data and scraper code is another significant milestone in a breakout year for Github’s use in government. In January, the British government committed GOV.UK code to Github. NASA, after contributing its first code in January added 11 code repositories this year. In August, the White House committed code to Github. In September, the Open Gov Foundation open sourced the MADISON crowd sourced legislation platform.

The choice to use Github for this scraper and legislative data, however, presents a new and interesting iteration in the site’s open source story.

“Github is a great fit for this because it’s neutral ground and it’s a welcoming environment for other potential contributors,” wrote Sunlight Labs director Tom Lee, in an email. “Sunlight expects to invest substantial resources in maintaining and improving this codebase, but it’s not ours: we think the data made available by this code belongs to every American. Consequently the project needed to embrace a form that ensures that it will continue to exist, and be free of encumbrances, in a way that’s not dependent on any one organization’s fortunes.”

Mill, an open government developer at Sunlight Labs, shared more perspective in the rest of our email interview, below.

Is this based on the GovTrack.us scraper?

Eric Mill: All three of us have contributed at least one code change to our new THOMAS scraper; the majority of the code was written by me. Some of the code has been taken or adapted from Josh’s work.

The scraper that currently actively populates the information on GovTrack is an older Perl-based scraper. None of that code was used directly in this project. Josh had undertaken an incomplete, experimental rewrite of these scrapers in Python about a year ago (code), but my understanding is it never got to the point of replacing GovTrack’s original Perl scripts.

We used the code from this rewrite in our new scraper, and it was extremely helpful in two ways &mddash; providing a roadmap of how THOMAS’ URLs and sitemap work, and parsing meaning out of the text of official actions.

Parsing the meaning out of action text is, I would say, about half the value and work of the project. When you look at a page on GovTrack or OpenCongress and see the timeline of a bill’s life — “Passed House,” “Signed by the President,” etc. — that information is only obtainable by analyzing the order and nature of the sentences of the official actions that THOMAS lists. Sentences are finicky, inconsistent things, and extracting meaning from them is tricky work. Just scraping them out of THOMAS.gov’s HTML is only half the battle. Josh has experience at doing this for GovTrack. The code in which this experience was encapsulated drastically reduced how long it took to create this.

How long did this take to build?

Eric Mill: Creating the whole scraper, and the accompanying dataset, was about 4 weeks of work on my part. About half of that time was spent actually scraping — reverse engineering THOMAS’ HTML — and the other half was spent creating the necessary framework, documentation, and general level of rigor for this to be a project that the community can invest in and rely on.

There will certainly be more work to come. THOMAS is shutting down in a year, to be replaced by Congress.gov. As Congress.gov grows to have the same level of data as THOMAS, we’ll gradually transition the scraper to use Congress.gov as its data source.

Was this data online before? What’s new?

Eric Mill: All of the data in this project has existed in an open way at GovTrack.us, which has provided bulk data downloads for years. The Sunlight Foundation and OpenCongress have both created applications based on this data, as have many other people and organizations.

This project was undertaken as a collaboration because Josh and I believed that the data was fundamental enough that it should exist in a public, owner-less commons, and that the code to generate it should be in the same place.

There are other benefits, too. Although the source code to GovTrack’s scrapers has been available, it depends on being embedded in GovTrack’s system, and the use of a database server. It was also written in Perl, a language less widely used today, and produced only XML. This new Python scraper has no other dependencies, runs without a database, and generates both JSON and XML. It can be easily extended to output other data formats.

Finally, everyone who worked on the project has had experience in dealing with legislative information. We were able to use that to make various improvements to how the data is structured and presented that make it easier for developers to use the data quickly and connect it to other data sources.

Searches for bills in Scout use data collected directly from this scraper. What else are people doing with the data?

Eric Mill: Right now, I only know for a fact that the Sunlight Foundation is using the data. GovTrack recently sent an email to its developer list announcing that in the near future, its existing dataset would be deprecated in favor of this new one, so the data should be used in GovTrack before long.

Pleasantly, I’ve found nearly nothing new by switching from GovTrack’s original dataset to this one. GovTrack’s data has always had a high level of quality. So far, the new dataset looks to be as good.

Is it common to host open data on Github?

Eric Mill: Not really. Github’s not designed for large-scale data hosting. This is an experiment to see whether this is a useful place to host it. The primary benefit is that no single person or organization (besides Github) is paying for download bandwidth.

The data is published as a convenience, for people to quickly download for analysis or curiosity. I expect that any person or project that intends to integrate the data into their work on an ongoing basis will do so by using the scraper, not downloading the data repeatedly from Github. It’s not our intent that anyone make their project dependent on the Github download links.

Laudably, Josh Tauberer donated his legislator dataset and converted it to YAML. What’s YAML?

Eric Mill: YAML is a lightweight data format intended to be easy for humans to both read and write. This dataset, unlike the one scraped from THOMAS, is maintained mostly through manual effort. Therefore, the data itself needs to be in source control, it needs to not be scary to look at and it needs to be obvious how to fix or improve it.

What’s in this legislator dataset? What can be done with it?

Eric Mill: The legislator dataset contains information about members of Congress from 1789 to the present day. It is a wealth of vital data for anyone doing any sort of application or analysis of members of Congress. This includes a breakdown of their name, a crosswalk of identifiers on other services, and social media accounts. Crucially, it also includes a member of Congress’ change in party, chamber, and name over time.

For example, it’s a pretty necessary companion to the dataset that our scraper gathers from THOMAS. THOMAS tells you the name of the person who sponsored this bill in 2003, and gives you a THOMAS-specific ID number. But it doesn’t tell you what that person’s party was at the time, or if the person is still a member of the same chamber now as they were in 2003 (or whether they’re in office at all). So if you want to say “how many Republicans sponsored bills in 2003,” or if you’d like to draw in information from outside sources, such as campaign finance information, you will need a dataset like the one that’s been publicly donated here.

Sunlight’s API on members of Congress is easily the most prominent API, widely used by people and organizations to build systems that involve legislators. That API’s data is a tiny subset of this new one.

You moved a legal citation and extractor into this code. What do they do here?

Eric Mill: The legal citation extractor, called “Citation,” plucks references to the US Code (and other things) out of text. Just about any system that deals with legal documents benefits from discovering links between those documents. For example, I use this project to power US Code searches on Scout, so that the site returns results that cite some piece of the law, regardless of how that citation is formatted. There’s no text-based search, simple or advanced, that would bring back results matching a variety of formats or matching subsections — something dedicated to the arcane craft of citation formats is required.

The citation extractor is built to be easy for others to invest in. It’s a stand-alone tool that can be used through the command line, HTTP, or directly through JavaScript. This makes it suitable for the front-end or back-end, and easy to integrate into a project written in any language. It’s very far from complete, but even now it’s already proven extremely useful at creating powerful features for us that weren’t possible before.

The parser for the U.S. Code itself is a dataset, written by my colleague Thom Neale. The U.S. Code is published by the government in various formats, but none of them are suitable for easy reuse. The Office of Law Revision Counsel, which publishes the U.S. Code, is planning on producing a dedicated XML version of the US Code, but they only began the procurement process recently. It could be quite some time before it appears.

Thom’s work parses the “locator code” form of the data, which is a binary format designed for telling GPO’s typesetting machines how to print documents. It is very specialized and very complicated. This parser is still in an early stage and not in use in production anywhere yet. When it’s ready, it’ll produce reliable JSON files containing the law of the United States in a sensible, reusable form.

Does Github’s organization structure makes a data commons possible?

Eric Mill: Github deliberately aligns its interests with the open source community, so it is possible to host all of our code and data there for free. Github offers unlimited public repositories, collaborators, bandwidth, and disk space to organizations and users at no charge. They do this while being an extremely successful, profitable business.

On Github, there are two types of accounts: users and organizations. Organizations are independent entities, but no one has to log in as an organization or share a password. Instead, at least one user will be marked as the “owner” of an organization. Ownership can easily change hands or be distributed amongst various users. This means that Josh, Derek, and I can all have equal ownership of the “unitedstates” repositories and data. Any of us can extend that ownership to anyone we want in a simple, secure way, without password sharing.

Github as a company has established both a space and a culture that values the commons. All software development work, from hobbyist to non-profit to corporation, from web to mobile to enterprise, benefits from a foundation of open source code. Github is the best living example of this truth, so it’s not surprising to me that it was the best fit for our work.

Why is this important to the public?

Eric Mill: The work and artifacts of our government should be available in bulk, for easy download, in accessible formats, and without license restrictions. This is a principle that may sound important and obvious to every technologist out there, but it’s rarely the case in practice. When it is, the bag is usually mixed. Not every member of the public will be able or want to interact directly with our data or scrapers. That’s fine. Developers are the force multipliers of public information. Every citizen can benefit somehow from what a developer can build with government information.

Related:

November 26 2012

Investigating data journalism

Great journalism has always been based on adding context, clarity and compelling storytelling to facts. While the tools have improved, the art is the same: explaining the who, what, where, when and why behind the story. The explosion of data, however, provides new opportunities to think about reporting, analysis and publishing stories.

As you may know, there’s already a Data Journalism Handbook to help journalists get started. (I contributed some commentary to it). Over the next month, I’m going to be investigating the best data journalism tools currently in use and the data-driven business models that are working for news startups. We’ll then publish a report that shares those insights and combines them with our profiles of data journalists.

Why dig deeper? Getting to the heart of what’s hype and what’s actually new and noteworthy is worth doing. I’d like to know, for instance, whether tutorials specifically designed for journalists can be useful, as Joe Brockmeier suggested at ReadWrite. On a broader scale, how many data journalists are working today? How many will be needed? What are the primary tools they rely upon now? What will they need in 2013? Who are the leaders or primary drivers in the area? What are the most notable projects? What organizations are embracing data journalism, and why?

This isn’t a new interest for me, but it’s one I’d like to found in more research. When I was offered an opportunity to give a talk at the second International Open Government Data Conference at the World Bank this July, I chose to talk about open data journalism and invited practitioners on stage to share what they do. If you watch the talk and the ensuing discussion in the video below, you’ll pick up great insight from the work of the Sunlight Foundation, the experience of Homicide Watch and why the World Bank is focused on open data journalism in developing countries.

The sites and themes that I explored in that talk will be familiar to Radar readers, focusing on the changing dynamic between the people formerly known as the audience and the editors, researchers and reporters who are charged with making sense of the data deluge for the public good. If you’ve watched one of my Ignites or my Berkman Center talk, much of this won’t be new to you, but the short talk should be a good overview of where I think this aspect of data journalism is going and why I think it’s worth paying attention to today.

For instance, at the Open Government Data Conference Bill Allison talked about how open data creates government accountability and reveals political corruption. We heard from Chris Amico, a data journalist who created a platform to help a court reporter tell the story of every homicide in a city. And we heard from Craig Hammer how the World Bank is working to build capacity in media organizations around the world to use data to show citizens how and where borrowed development dollars are being spent on their behalf.

The last point, regarding capacity, is a critical one. Just as McKinsey identified a gap between available analytic talent and the demand created by big data, there is a data science skills gap in journalism. Rapidly expanding troves of data are useless without the skills to analyze it, whatever the context. An over focus on tech skills could exclude the best candidates for these jobs — but there will need to be training to build them.

This reality hasn’t gone unnoticed by foundations or the academy. In May, the Knight Foundation gave Columbia University $2 million for research to help close the data science skills gap. (I expect to be talking to Emily Bell, Jonathan Stray and the other instructors and students.)

Media organizations must be able to put data to work, a need that was amply demonstrated during Hurricane Sandy, when public open government data feeds became critical infrastructure.

What I’d like to hear from you is what you see working around the world, from the Guardian to ProPublica, and what you’re working on, and where. To kick things off, I’d like to know which organizations are doing the most innovative work in data journalism.

Please weigh in through the comments or drop me a line at alex@oreilly.com or at @digiphile on Twitter.

November 02 2012

Charging up: Networking resources and recovery after Hurricane Sandy

Even though the direct danger from Hurricane Sandy has passed, lower Manhattan and many parts of Connecticut and New Jersey remain a disaster zone, with millions of people still without power, reduced access to food and gas, and widespread damage from flooding. As of yesterday, according to reports from Wall Street Journal, thousands of residents remain in high-rise buildings with no water, power or heat.

E-government services are in heavy demand, from registering for disaster aid to finding resources, like those offered by the Office of the New York City Advocate. People who need to find shelter can use the Red Cross shelter app. FEMA has set up a dedicated landing page for Hurricane Sandy and a direct means to apply for disaster assistance:

Public officials have embraced social media during the disaster as never before, sharing information about where to find help.

No power and diminished wireless capacity, however, mean that the Internet is not accessible in many homes. In the post below, learn more on what you can do on the ground to help and how you can contribute online.

For those who have lost power, using Twitter offline to stay connected to those updates is useful — along with using weather radios.

That said, for those that can get connected on mobile devices, there are digital resources emerging, from a crowdsourced Sandy coworking map in NYC to an OpenTrip Planner app for navigating affected transit options. This Google Maps mashup shows where to find food, shelter and charging stations in Hoboken, New Jersey.

In these conditions, mobile devices are even more crucial connectors to friends, family, services, resources and information. With that shift, government websites must be more mobile-friendly and offer ways to get information through text messaging.

Widespread power outages also mean that sharing the means to keep devices charged is now an act of community and charity.

Ways to to help with Sandy relief

A decade ago, if there was a disaster, you could donate money and blood. In 2012, you can also donate your time and skills. New York Times blogger Jeremy Zillar has compiled a list of hurricane recovery and disaster recovery resources. The conditions on the ground also mean that finding ways to physically help matter.

WNYC has a list of volunteer options around NYC. The Occupy Wall Street movement has shifted to “Occupy Sandy,” focusing on getting volunteers to help pick up and deliver food in neighborhoods around New York City. As Nick Judd reported for TechPresident, this “people-powered recovery” is volunteering to process incoming offers of help and requests for aid.

They’re working with Recovers.org, a new civic startup, which has now registered some 5,000 volunteers from around the New York City area. Recovers is pooling resources and supplies with community centers and churches to help in the following communities:

If you want to help but are far away from directly volunteering in New York, Connecticut or New Jersey, there are several efforts underway to volunteer online, including hackathons around the world tomorrow. Just as open government data feeds critical infrastructure during disasters, it is also integral to recovery and relief. To make that data matter to affected populations, however, the data must be put to use. That’s where the following efforts come in.

“There are a number of ways tech people can help right now,” commented Gisli Olafsson, Emergency Response Director at NetHope, reached via email. “The digital volunteer communities are coordinating many of those efforts over a Skype chat group that we established few days before Sandy arrived. I asked them for input and here are their suggestions:

  1. Sign up and participate in the crisis camps that are being organized this weekend at Geeks Without Borders and Sandy Crisis Camp.
  2. Help create visualizations and fill in the map gaps. Here is a link to all the maps we know about so far. Help people find out what map to look at for x,y,z.
  3. View damage photos to help rate damage assessments at Sandy OpenStreetMap. There are over 2000 images to identify and so far over 1000 helpers.”

Currently, there are Crisis Camps scheduled for Boston, Portland, Washington (DC), Galway (Ireland), San Francisco, Seattle, Auckland (NZ) and Denver, at RubyCon.

“If you are in any of those cities, please go the Sandy CrisisCamp blog post and sign up for the EventBrite for the CrisisCamp you want to attend in person or virtually,” writes Chad Catacchio (@chadcat), Crisis Commons communication lead.

“If you want to start a camp in your city this weekend, we are still open to the idea, but time is running short (it might be better to aim for next week),” he wrote.

UPDATE: New York-based nonprofit DataKind tweeted that they’re trying to rally the NY Tech community to pitch in real life on Saturday and linked to a new Facebook group. New York’s tech volunteers have already been at work helping city residents over the last 24 hours, with the New York Tech Meetup organizing hurricane recovery efforts.

People with technical skills in the New York area who want to help can volunteer online here and check out the NY Tech responds blog.

As Hurricane Sandy approached, hackers built tools to understand the storm. Now that it’s passed, “Hurricane Hackers” are working on projects to help with the recovery. The crisis camp in Boston will be hosted at the MIT Media Lab by Hurricane Hackers this weekend.

Sandy Crisis Camps already have several projects in the works. “We have been asked by FEMA to build and maintain a damage assessment map for the entire state of Rhode Island,” writes Catacchio. He continues:

“We will also be assisting in monitoring social media and other channels and directing reports to FEMA there. We’ll be building the map using ArcGIS and will be needing a wide range of skill sets from developers to communications to mapping. Before the weekend, we could certainly use some help from ArcGIS folks in getting the map ready for reporting, so if that is of interest, please email Pascal Schuback at pascal@crisiscommons.org. Secondly, there has been an ask by NYU and the consortium of colleges in NYC to help them determine hotel capacity/vacancy as well as gas stations that are open and serving fuel. If other official requests for aid come in, we will let the community know. Right now, we DO anticipate more official requests, and again, if you are working with the official response/recovery and need tech support assistance, please let us know: email either Pascal or David Black at david@crisiscommons.org. We are looking to have a productive weekend of tackling real needs to help the helpers on the ground serving those affected by this terrible storm.”

Related:

October 31 2012

NYC’s PLAN to alert citizens to danger during Hurricane Sandy

Starting at around 8:36 PM ET last night, as Hurricane Sandy began to flood the streets of lower Manhattan, many New Yorkers began to receive an unexpected message: a text alert on their mobile phones that strongly urged them to seek shelter. It showed up on iPhones:

…and upon Android devices:

While the message was clear enough, the way that these messages ended up on the screens may not have been clear to recipients or observers. And still other New Yorkers were left wondering why emergency alerts weren’t on their phones.

Here’s the explanation: the emergency alerts that went out last night came from New York’s Personal Localized Alerting Network, the “PLAN” the Big Apple launched in late 2011.

NYC chief digital officer Rachel Haot confirmed that the messages New Yorkers received last night were the result of a public-private partnership between the Federal Communications Commission, the Federal Emergency Management Agency, the New York City Office of Emergency Management (OEM), the CTIA and wireless carriers.

While the alerts may look quite similar to text messages, the messages themselves run in parallel, enabling them to get through txt traffic congestion. NYC’s PLAN is the local version of the Commercial Mobile Alert System (CMAS) that has been rolling out nation-wide over the last year.

“This new technology could make a tremendous difference during
disasters like the recent tornadoes in Alabama where minutes – or even seconds – of extra warning could make the difference between life and death,” said FCC chairman Julius Genachowski, speaking last May in New York City. “And we saw the difference alerting systems can make in Japan, where they have an earthquake early warning system that issued alerts that saved lives.”

NYC was the first city to have it up and running, last December, and less than a year later, the alerts showed up where and when they mattered.

The first such message I saw shared by a New Yorker actually came on October 28th, when the chief digital officer of the Columbia Journalism School, Sree Sreenivasan, tweeted about receiving the alert:

He tweeted out the second alert he received, on the night of the 29th, as well:

These PLAN alerts go out to everyone in a targeted geographic area with enabled mobile devices, enabling emergency management officials at the state and local level to get an alert to the right people at the right time. And in an emergency like a hurricane, earthquake or fire, connecting affected residents to critical information at the right time and place are essential.

While the government texting him gave national security writer Marc Ambinder some qualms about privacy, the way the data is handled looks much less disconcerting than, say, needing to opt-out of sharing location data or wireless wiretapping.

PLAN alerts are free and automatic, unlike opt-in messages from Notify NYC or signing up for email alerts from OEM.

Not all New Yorkers received an emergency alert during Sandy because not all mobile devices have the necessary hardware installed or have updated relevant software. In May 2011, new iPhones and Android devices already had the chip. (Most older phones, not so much.)

These alerts don’t go out for minor issues, either: the system is only used by authorized state, local or national officials during public safety emergencies. They send the alert to CMAS, it’s authenticated, and then the system pushes it out to all enabled devices in a geographic area.

Consumers receive only three types of messages: alerts issued by the President, Amber Alerts, and alerts involving “imminent threats to safety or life.” The last category covers the ones that went out about Hurricane Sandy in NYC last night.

According to the FCC, participating mobile carriers can allow their subscribers to block all but Presidential alerts, although it may be a little complicated to navigate a website or call center to do so. By 2014, every mobile phone sold in the United States must be CMAS-capable. (You can learn more about CMAS in this PDF). Whether such mobile phones should be subsidized for the poor is a larger question that will be left to the next administration.

As more consumers replace their devices in the years ahead, more people around the United States will also be able to receive these messages, benefiting from a public-private partnership that actually worked to deliver on improved public safety.

At least one New Yorker got the message and listened to it:

“If ‘act’ means stay put, then why yes I did,” tweeted Noreen Whysel, operations manager Information Architecture Institute. “It was enough to convince my husband from going out….”

Here’s hoping New York City doesn’t have use this PLAN to tell her and others about impending disaster again soon.

October 19 2012

San Francisco looks to tap into the open data economy

As interest in open data continues to grow around the world, cities have become laboratories for participatory democracy. They’re also ground zero for new experiments in spawning civic startups that deliver city services or enable new relationships between the people and city government. San Francisco was one of the first municipalities in the United States to embrace the city as a platform paradigm in 2009, with the launch of an open data platform.

Years later, the city government is pushing to use its open data to accelerate economic development. On Monday, San Francisco announced revised open data legislation to enable that change and highlighted civic entrepreneurs who are putting the city’s data to work in new mobile apps.

City staff have already published the revised open data legislation on GitHub. (If other cities want to “fork” it, clone away.) David Chiu, the chairman of the San Francisco Board of Supervisors, the city’s legislative body, introduced the new version on Monday and submitted it on Tuesday. A vote is expected before the end of the year.

Speaking at the offices of the Hatchery in San Francisco, Chiu observed that, by and large, the data that San Francisco has put out showed the city in a positive light. In the future, he suggested, that should change. Chiu challenged the city and the smartest citizens of San Francisco to release more data, figure out where the city could take risks, be more entrepreneurial and use data to hold the city accountable. In his remarks, he said that San Francisco is working on open budgeting but is still months away from getting the data that they need.

Rise of the CDO

This new version of the open data legislation will create a chief data officer (CDO) position, assign coordinators for open data in each city department, and make it clear in procurement language that the city owns data and retains access to it.

“Timelines, mandates and especially the part about getting them to inventory what data they collect are all really good,” said Luke Fretwell, founder of Govfresh, which covers open government in San Francisco. “It’s important that’s in place. Otherwise, there’s no way to be accountable. Previous directives didn’t do it.”

The city’s new CDO will “be responsible for sharing city data with the public, facilitating the sharing of information between City departments, and analyzing how data sets can be used to improve city decision making,” according to the revised legislation.

In creating a CDO, San Francisco is running a play from the open data playbooks of Chicago and Philadelphia. (San Francisco’s new CDO will be a member of the mayor’s staff in the budget office.) Moreover, the growth of CDOs around the country confirms the newfound importance of civic data in cities. If open government data is to be a strategic asset that can be developed for the public good, civic utility and economic value, it follows that it needs better stewards.

Assigning a coordinator in each department is also an acknowledgement that open data consumers need a point of contact and accountability. In theory, this could help create better feedback loops between the city and the cohort of civic entrepreneurs that this policy is aimed at stimulating.

Who owns the data?

San Francisco’s experience with NextBus and a conflict over NextMuni real-time data is a notable case study for other cities and states that are considering similar policies.

The revised legislation directs the Committee on Information Technology (COIT) to, within 60 days from the passage of the legislation, enact “rules for including open data requirements in applicable City contracts and standard contract provisions that promote the City’s open data policies, including, where appropriate, provisions to ensure that the City retains ownership of City data and the ability to post the data on data.sfgov.org or make it available through other means.”

That language makes it clear that it’s the city that owns city data, not a private company. That’s in line with a principle that open government data is a public good that should be available to the public, not locked up in a proprietary format or a for-pay database. There’s some nuance to the issue, in terms of thinking through what rights a private company that invests in acquiring and cleaning up government data holds, but the basic principle that the public should have access to public data is sound. The procurement practices in place will mean that any newly purchased system that captures structured data must have a public API.

Putting open data to work

Speaking at the Hatchery on Monday, Mayor Ed Lee highlighted three projects that each showcase open data put to use. The new Rec & Park app (iOS download), built by San Francisco-based startup Appallicious, enables citizens to find trails, dog parks, playgrounds and other recreational resources on a mobile device. “Outside” (iOS download), from San Francisco-based 100plus, encourages users to complete “healthy missions” in their neighborhoods. The third project, from mapping giant Esri, is a beautiful web-based visualization of San Francisco’s urban growth based upon open data from San Francisco’s planning departments.

The power of prediction

Over the past three years, transparency, accountability, cost savings and mobile apps have constituted much of the rationale for open data in cities. Now, San Francisco is renewing its pitch for the role of open data in job creation and combining increased efficiency and services.

Jon Walton, San Francisco’s chief information officer (CIO), identified two next steps for San Francisco in an interview earlier this year: working with other cities to create a federated model (now online at cities.data.gov) and using its own data internally to identify and solve issues. (San Francisco and cities everywhere will benefit from looking to New York City’s work with predictive data analytics.)

“We’re thinking about using data behind the firewalls,” said Walton. “We want to give people a graduated approach, in terms of whether they want to share data for themselves, to a department, to the city, or worldwide.”

On that count, it’s notable that Mayor Lee is now publicly encouraging more data sharing between private companies that are collecting data in San Francisco. As TechCrunch reported, the San Francisco government quietly passed a new milestone when it added to its open data platform private-sector datasets on pedestrian and traffic movement collected by Motionloft.

“This gives the city a new metric on when and where congestion happens, and how many pedestrians and vehicles indicate a slowdown will occur,” said Motionloft CEO Jon Mills, in an interview.

Mills sees opportunities ahead to apply predictive data analytics to life and death situations by providing geospatial intelligence for first responders in the city.

“We go even further when police and fire data are brought in to show the relation between emergency situations and our data,” he said. “What patterns cause emergencies in different neighborhoods or blocks? We’ll know, and the city will be able to avoid many horrible situations.”

Such data-sharing could have a real impact on department bottom lines: while “Twitter311” created a lot of buzz in the social media world, access to real-time transit data is what is estimated to have saved San Francisco more than $1 million a year by reducing the volume of San Francisco 311 calls by 21.7%.

Open data visualization can also enable public servants to understand how city residents are interacting and living in an urban area. For instance, a map of San Francisco pedestrian injuries shows high-injury corridors that merit more attention.

Open data and crowdsourcing will not solve all IT ills

While San Francisco was an early adopter of open data, that investment hasn’t changed an underlying reality: the city government remains burdened by a legacy of dysfunctional tech infrastructure, as detailed in a report issued in August 2012 by the City and County of San Francisco.

“San Francisco’s city-wide technology governing structure is ineffective and poorly organized, hampered by a hands-off Mayor, a weak Committee on Information Technology, an unreliable Department of Technology, and a departmentalized culture that only reinforces the City’s technological ineffectiveness,” state the report’s authors.

San Francisco government has embraced technologically progressive laws and rhetoric, but hasn’t always followed through on them, from setting deadlines to reforming human resources, code sharing or procurement.

“Departments with budgets in the tens of millions of dollars — including the very agency tasked with policing government ethics — still have miles to go,” commented Gov 2.0 advocate Adriel Hampton and former San Francisco government staffer in an interview earlier this year.

Hampton, who has turned his advocacy to legal standards for open data in California and to working at Nationbuilder, a campaign software startup, says that San Francisco has used technology “very poorly” over the past decade. While he credited the city’s efforts in mobile government and recent progress on open data, the larger system is plagued with problems that are endemic in government IT.

Hampton said the city’s e-government efforts largely remain in silos. “Lots of departments have e-services, but there has been no significant progress in integrating processes across departments, and some agencies are doing great while others are a mess,” commented Hampton. “Want to do business in SF? Here’s a sea of PDFs.”

The long-standing issues here go beyond policy, in his view. “San Francisco has a very fragmented IT structure, where the CIO doesn’t have real authority, and proven inability to deliver on multi-departmental IT projects,” he said. As an example, Hampton pointed to San Francisco’s Justice Information Tracking System, a $25 million, 10-year project that has made some progress, but still has not been delivered.

“The City is very good at creating feel-good requirements for its vendors that simply result in compliant companies marking up and reselling everything from hardware to IT software and services,” he commented. “This makes for not only higher costs and bureaucratic waste, but huge openings for fraud. Contracting reform was the number one issue identified in the ImproveSF employee ideation exercise in 2010, but it sure didn’t make the press release.”

Hampton sees the need for two major reforms to keep San Francisco on a path to progress: empowering the CIO position with more direct authority over departmental IT projects, and reforming how San Francisco procures technology, an issue he says affects all other parts of the IT landscape. The reason city IT is so bad, he says, its that it’s run by a 13-member council. “[The] poor CIO’s hardly got a shot.”

All that said, Hampton gives David Chiu and San Francisco city government high marks for their recent actions. “Bringing in Socrata to power the open data portal is a solid move and shows commitment to executing on the open data principle,” he said.

While catalyzing more civic entrepreneurship is important, creating enduring structural change in how San Francisco uses technology will require improving how the city government collects, stores, consumes and releases data, along with how it procures, governs and builds upon technology.

On that count, Chicago’s experience may be relevant. Efforts to open government data there have led to both progress and direction, as Chicago CTO John Tolva blogged in January:

“Open data and its analysis are the basis of our permission to interject the following questions into policy debate: How can we quantify the subject-matter underlying a given decision? How can we parse the vital signs of our city to guide our policymaking? … It isn’t just app competitions and civic altruism that prompts developers to create applications from government data. 2011 was the year when it became clear that there’s a new kind of startup ecosystem taking root on the edges of government. Open data is increasingly seen as a foundation for new businesses built using open source technologies, agile development methods, and competitive pricing. High-profile failures of enterprise technology initiatives and the acute budget and resource constraints inside government only make this more appealing.”

Open data and job creation?

While realizing internal efficiencies and cost savings are key requirements for city CIOs, they don’t hold the political cachet of new jobs and startups, particularly in an election year. San Francisco is now explicitly connecting its release of open data to jobs.

“San Francisco’s open data policies are creating jobs, improving our city and making it easier for residents and visitors to communicate with government,” commented Mayor Lee, via email.

Lee is optimistic about the future, too: “I know that, at the heart of this data, there will be a lot more jobs created,” he said on Monday at the Hatchery.

Open data’s potential for job creation is also complemented by its role as a raw material for existing businesses. “This legislation creates more opportunities for the Esri community to create data-driven decision products,” said Bronwyn Agrios, a project manager at Esri, in an interview.

Esri, however, as an established cloud mapping giant, is in a different position than startups enabled by open data. Communications strategist Brian Purchia, the former new media director for former San Francisco Mayor Gavin Newsom, points to Appallicious.

Appallicious “would not have been possible with [San Francisco's] open data efforts,” said Purchia. “They have have hired about 10 folks and are looking to expand to other cities.”

The startup’s software drives the city’s new Rec & Park app, including the potential to enable mobile transactions in the next iteration.

“Motionloft will absolutely grow from our involvement in San Francisco open data,” said Motionloft CEO Mills. “By providing some great data and tools to the city of San Francisco, it enables Motionloft to develop solutions for other cities and government agencies. We’ll be hiring developers, sales people, and data experts to keep up with our plans to grow this nationwide, and internationally.”

The next big question for these startups, as with so many others in nearby Silicon Valley, is whether their initial successes can scale. For that to happen for startups that depend upon government data, other cities will not only need to open up more data, they’ll need to standardize it.

Motionloft, at least, has already moved beyond the Bay Area, although other cities haven’t incorporated its data yet. Esri, as a major enterprise provider of proprietary software to local governments, has some skin in this game.

“City governments are typically using Esri software in some capacity,” said Agrios. “It will certainly be interesting to see how geo data standards emerge given the rapid involvement of civic startups eagerly consuming city data. Location-aware technologists on both sides of the fence, private and public, will need to work together to figure this out.”

If the marketplace for civic applications based upon open data develops further, it could help with a key issue that has dogged the results of city app contests: sustainability. It could also help with a huge problem for city governments: the cost of providing e-services to more mobile residents as budgets continue to tighten.

San Francisco CIO Walton sees an even bigger opportunity for the growth of civic apps that go far beyond the Bay Area, if cities can coordinate their efforts.

“There’s lots of potential here,” Walton said. “The challenge is replicating successes like Open311 in other verticals. If you look at the grand scale of time, we’re just getting started. For instance, I use Nextbus, an open source app that uses San Francisco’s open data … If I have Nextbus on my phone, when I get off a plane in Chicago or New York City, I want to be able to use it there, too. I think we can achieve that by working together.”

If a national movement toward open data and civic apps gathers more momentum, perhaps we’ll solve a perplexing problem, mused Walton.

“In a sense, we have transferred the intellectual property for apps to the public,” he said. “On one hand, that’s great, but I’m always concerned about what happens when an app stops working. By creating data standards and making apps portable, we will create enough users so that there’s enough community to support an application.”

Related:

October 17 2012

Data from health care reviews could power “Yelp for health care” startups

A hospital in MaineA hospital in MaineGiven where my work and health has taken me this year, I’ve been thinking much more about the relationship of the Internet and health data to accountability and patient-driven health care.

When I was looking for a place in Maine to go for care this summer, I went online to look at my options. I consulted hospital data from the government at HospitalCompare.HHS.gov and patient feedback data on Yelp, and then made a decision based upon proximity and those ratings. If I had been closer to where I live in Washington D.C., I would also have consulted friends, peers or neighbors for their recommendations of local medical establishments.

My brush with needing to find health care when I was far from home reminded me of the prism that collective intelligence can now provide for the treatment choices we make, if we have access to the Internet.

Patients today are sharing more of their health data and experiences online voluntarily, which in turn means that the Internet is shaping health care. There’s a growing phenomenon of “e-patients” and caregivers going online to find communities and information about illness and disability.

Aided by search engines and social media, newly empowered patients are discussing health conditions with others suffering from disease and sickness — and they’re taking that peer-to-peer health care knowledge into their doctors’ offices with them, frequently on mobile devices. E-patients are sharing their health data of their own volition because they have a serious health condition, want to get healthy, and are willing.

From the perspective of practicing physicians and hospitals, the trend of patients contributing to and consulting on online forums adds the potential for errors, fraud, or misunderstanding. And yet, I don’t think there’s any going back from a networked future of peer-to-peer health care, anymore than we can turn back the dial on networked politics or disaster response.

What’s needed in all three of these areas is better data that informs better data-driven decisions. Some of that data will come from industry, some from government, and some from citizens.

This fall, the Obama administration proposed a system for patients to report medical mistakes. The system would create a new “consumer reporting system for patient safety” that would enable patients to tell the federal government about unsafe practices or errors. This kind of review data, if validated by government, could be baked into the next generation of consumer “choice engines,” adding another layer for people, like me, searching for care online.

There are precedents for the collection and publishing of consumer data, including the Consumer Product Safety Commission’s public complaint database at SaferProducts.gov and the Consumer Financial Protection Bureau’s complaint database. Each met with initial resistance by industry but have successfully gone online without massive abuse or misuse, at least to date.

It will be interesting to see how medical associations, hospitals and doctors react. Given that such data could amount to government collecting data relevant to thousands of “Yelps for health care,” there’s both potential and reason for caution. Health care is a bit different than product safety or consumer finance, particularly with respect to how a patient experiences or understands his or her treatment or outcomes for a given injury or illness. For those that support or oppose this approach, there is an opportunity for public comment on proposed data collection at the Federal Register.

The power of performance data

Combining patients review data with government-collected performance data could be quite powerful in helping to drive better decisions and adding more transparency to health care.

In the United Kingdom, officials are keen to find the right balance between open data, transparency and prosperity.

“David Cameron, the Prime Minister, has made open data a top priority because of the evidence that this public asset can transform outcomes and effectiveness, as well as accountability,” said Tim Kelsey, in an interview this year. He used to head up the United Kingdom’s transparency and open data efforts and now works at its National Health Service.

“There is a good evidence base to support this,” said Kelsey. “Probably the most famous example is how, in cardiac surgery, surgeons on both sides of the Atlantic have reduced the number of patient deaths through comparative analysis of their outcomes.”

More data collected by patients, advocates, governments and industry could help to shed light on the performance of more physicians and clinics engaged in other expensive and lifesaving surgeries and associated outcomes.

Should that be extrapolated across the medical industry, it’s a safe bet that some medical practices or physicians will use whatever tools or legislative influence they have to fight or discredit websites, services or data that puts them in a poor light. This might parallel the reception that BrightScope’s profiles of financial advisors have received in industry.

When I talked recently with Dr. Atul Gawande about health data and care givers, he said more transparency in these areas is crucial:

“As long as we are not willing to open up data to let people see what the results are, we will never actually learn. The experience of what happens in fields where the data is open is that it’s the practitioners themselves that use it.”

In that context, health data will be the backbone of the disruption in health care ahead. Part of that change will necessarily have to come from health care entrepreneurs and watchdogs connecting code to research. In the future, a move to open science and perhaps establish a health data commons could accelerate that change.

The ability of caregivers and patients alike to make better data-driven decisions is limited by access to data. To make a difference, that data will also need to be meaningful to both the patient and the clinician, said Dr. Gawande. He continued:

“[Health data] needs to be able to connect the abstract world of data to the physical world of what really happens, which means it has to be timely data. A six-month turnaround on data is not great. Part of what has made Wal-Mart powerful, for example, is they took retail operations from checking their inventory once a month to checking it once a week and then once a day and then in real-time, knowing exactly what’s on the shelves and what’s not. That equivalent is what we’ll have to arrive at if we’re to make our systems work. Timeliness, I think, is one of the under-recognized but fundamentally powerful aspects because we sometimes over prioritize the comprehensiveness of data and then it’s a year old, which doesn’t make it all that useful. Having data that tells you something that happened this week, that’s transformative.”

Health data, in other words, will need to be open, interoperable, timely, higher quality, baked into the services that people use, and put at the fingertips of caregivers, as US CTO Todd Park explains in the video below:

There is more that needs to be done than simply putting “how to live better” information online or into an app. To borrow a phrase from Robert Kirkpatrick, for data to change health care, we’ll need to apply the wisdom of the crowds, the power of algorithms and the intuition of experts to find meaning in health data and help patients and caregivers alike make better decisions.

That isn’t to say that health data, once published, can’t be removed or filtered. Witness the furor over the removal of a malpractice database from the Internet last year, along with its restoration.
But as more data about doctors, services, drugs, hospitals and insurance companies goes online, the ability of those institutions to control public perception of the institutions will shift, just as it has with government and media. Given flaws in devices or poor outcomes, patients deserve such access, accountability and insight.

Enabling better health-data-driven decisions to happen across the world will be far from easy. It is, however, a future worth building toward.

Reposted byfortmyersrealty fortmyersrealty

October 03 2012

The missing ingredient from hyperwired debates: the feedback loop

PodiumPodiumWhat a difference a season makes. A few months after widespread online frustration with a tape-delayed Summer Olympics, the 2012 Presidential debates will feature the most online livestreams and wired, up-to-the-second digital coverage in history.

Given the pace of technological change, it’s inevitable that each election season will bring with it new “firsts,” as candidates and campaigns set precedents by trying new approaches and platforms. This election has been no different: the Romney and Obama campaigns have been experimenting with mobile applications, social media, live online video and big data all year.

Tonight, one of the biggest moments in the presidential campaign to date is upon us and there are several new digital precedents to acknowledge.

The biggest tech news is that YouTube, in a partnership with ABC, will stream the debates online for the first time. The stream will be on YouTube’s politics channel, and it will be embeddable.

With more and more livestreamed sports events, concerts and now debates available online, tuning in to what’s happening no longer means passively “watching TV.” The number of other ways people can tune in online in 2012 has skyrocketed, as you can see in GigaOm’s post listing debate livestreams or Mashable’s ways to watch the debates online.

This year, in fact, the biggest challenge people will have will not be finding an online alternative to broadcast or cable news but deciding which one to watch.

If you’re low on bandwidth or have a mobile device, NPR will stream the audio from the debate online and to its mobile apps. If you’re a Spanish speaker, Univision will stream the debates on YouTube with real-time translation.

The New York Times, Politico and Wall Street Journal are both livestreaming the debates at their websites or through their apps, further eroding the line between broadcast, print and online media.

While the PBS News Hour and CSPAN’s debate hub are good options, my preference is for the Sunlight Foundation’s award-winning Sunlight Live liveblog.

There are a couple of other notable firsts. The Huffington Post will deploy its HuffPost Live platform for the first time, pulling more viewers directly into participatory coverage online.

For those looking for a more… animated approach, the Guardian and Tumblr will ‘live GIF’ the presidential debates.

Microsoft is livestreaming the debates through the XBox, giving gamers an opportunity to weigh in on what they see through their Xboxes. They’ll be polled through the Xbox console during the debate, which will provide more real-time data from a youthful demographic that, according StrategyOne, still has many voters who are not firmly committed.

Social politics

The political news cycle has long since moved from the morning papers and the nightly news to real-time coverage of events. In past years, the post-debate spin by campaigns and pundits shaped public opinion. This year, direct access to online video and to the reaction of friends, family, colleagues and media through the social web means that the spin will begin as soon as any quip, policy position or rebuttal is delivered in the debate.

Beyond real-time commentary, social media will provide useful data for the campaigns to analyze. While there won’t be a “do over,” seeing what resonated directly with the public will help the campaigns tune their messages for the next debates.

Tonight, when I go on Al Jazeera’s special debate night coverage at The Stream, I’ll be looking at a number of factors. I expect the #DenverDebate and #debates hashtags to be moving too fast to follow, so I’ll be looking at which tweets are being amplified and what we can see on Twitter’s new #debates page, what images are popping online, which links are popular, how Facebook and Google+ are reacting, and what people are searching for on Google.com.

This is quite likely to be the most social political event ever, surpassing either of the 2012 political conventions or the State of the Union address. When I watch online, I’ll be looking for what resonated with the public, not just what the campaigns are saying — although that will factor into my analysis. The @mittromney account tweets 1-2 times a day. Will they tweet more? Will @barackobama’s 19 million followers be engaged? How much and how often will they update Facebook, and to what effect?

Will they live tweet open statements with links to policies? Will they link to rebuttals or fact checks in the media? Will they push people to go register or comment or share? Will they echo applause lines or attack lines? In a larger sense, will the campaigns act social, themselves? Will they reshare the people’s posts about them on social platforms or keep broadcasting?

We’ll know answers to all of these questions in a few hours.

Fact-checking in real-time

Continuing a trend from the primary season, real-time fact-checking will play a role in the debate. The difference in this historic moment is it will be the pace of it and the number of players.

As Nick Judd highlighted at techPresident, the campaign response is going to be all about mobile. Both campaigns will be trying their hands at fact checking, using new adaptive microsites at barackobama.com/debate and debates.mittromney.com, dedicated Twitter accounts at @TruthTeam2012 and and @RomneyResponse, and an associated subdomain and Tumblr.

Given the skin that campaigns have in the game, however, undecided or wavering voters are better off going with the Fourth Estate versions. Wired media organizations, like the newspapers streaming the debates I’ve listed above, will be using liveblogs and leveraging their digital readership to help fact check.

Notably, NPR senior social strategist Andy Carvin will be applying the same approach to fact checking during the debate as he has to covering the changes in the Middle East. To participate, follow @acarvin and use the #factcheck hashtag beginning at 8:30 ET.

It’s unclear whether debate moderator Jim Lehrer will tap into the fact-checking efforts online to push back on the candidates during the event. Then again, the wisdom of the crowds may be balanced by one man’s perspective. Given that he’s serving in that capacity for the 12th time, Lehrer possesses substantial experience of his own to draw upon in making his own decisions about when to press, challenge or revisit issues.

The rise of networked polities

In a larger sense, all of this interactivity falls fall short of the promise of networked politics in the Internet age. In the age of the Internet, television debates look antiquated.

When it comes to how much the people are directly involved with the presidential debates of 2012, as Micah Sifry argued earlier this week, little has changed from 2008:

“Google is going to offer some kind of interactive audience dial gadget for YouTube users, which could allow for real-time audience feedback — except it’s already clear none of that feedback is going to get anywhere near the actual debate itself. As best as I can tell, what the CPD [Commission on Presidential Debates] is doing is little more than what they did four years ago, except back then they partnered with Myspace on a site called MyDebates.org that featured video streaming, on-demand playback and archival material. Oh, but this time the partner sites will include a dynamic counter showing how many people have ‘shared their voice’.”

While everyone who has access to the Internet will be able to use multiple screens to watch, read and participate in the conversation around the debates, the public isn’t going to be directly involved in the debate. That’s a missed opportunity that won’t be revisited until the 2016 campaign.

By then, it will be an even more wired political landscape. While many politicians are still delegating the direct use of social media use to staffers, in late 2012 it ill behooves any office to be seen as technically backward and stay off them entirely.

In the years ahead, open government advocates will push politicians to use the Internet to explain their votes, not just broadcast political attacks or campaign events. After all, the United States is a constitutional republic. Executives and Congressmen are obligated to listen to the people they represent. The existing ecosystem of social media platforms may give politicians new tools to interact directly with their constituents but they’re still relatively crude.

Yes, the next generation of social media data analytics will give politicians a dashboard of what their constituents think about their positions. It’s the next generation of polling. In the years to come, however, I’m optimistic that we’re going to see much better use of the Internet to hold politicians accountable for their campaign positions and subsequent votes.

Early experiments in creating an “OKCupid for elections” will evolve. Expect sophisticated choice engines that use social and legislative data to tell voters not only whether candidates share their positions but whether they actually voted or acted upon them. Over time, opposition candidates will be able to use that accumulated data in their campaign platforms and during debates. If a member of Congress or President doesn’t follow through with the wishes of the people, he or she will have to explain why. That will be a debate worth having.

September 28 2012

Four key trends changing digital journalism and society

See something or say something: Los AngelesSee something or say something: Los AngelesIt’s not just a focus on data that connects the most recent class of Knight News Challenge winners. They all are part of a distributed civic media community that works on open source code, collects and improves data, and collaborates across media organizations.

These projects are “part of an infrastructure that helps journalists better understand and serve their communities through data,” commented Chris Sopher, Knight Foundation Journalism Program Associate, in an interview last week. To apply a coding metaphor, the Knight Foundation is funding the creation of patches for the source code of society. This isn’t a new focus: in 2011, Knight chose to help build the newsroom stack, from editorial search engines to data cleaning tools.

Following are four themes that jumped out when I looked across the winners of the latest Knight News Challenge round.

Networked accountability

An intercontinental project that bridged citizen science, open data, open source hardware, civic hacking and the Internet of things to monitor, share and map radiation data? Safecast is in its own category. Adapting the system to focus on air quality in Los Angeles — a city that’s known for its smog — will be an excellent stress test for seeing if this distributed approach to networked accountability can scale.

If it does — and hacked Chumbys, LED signs, Twitter bots, smartphone apps and local media reports start featuring the results — open data is going to be baked into how residents of Los Angeles understand their own atmosphere. If this project delivers on some of its promise, the value of this approach will be clearer.

If this project delivers on all of its potential, the air itself might improve. For that to happen, the people who are looking at the realities of air pollution will need to advocate for policy makers to improve it. In the future, the success or failure of this project will inform similar efforts that seek to enlist communities in data collection, including whether governments embrace “citizensourcing” beyond natural disasters and crises. The idea of citizens as sensors continues to have legs.

Peer-to-peer collaboration, across newsrooms

As long as I’ve been reading newspapers, watching television news and following the industry, competition has always been part of the dynamic: be first to the scene, first to get the scoop, first to call the election. As the Internet has taken on a larger role in delivering the news, there have been new opportunities for competition in digital journalism: first to tweet, post or upload video, often followed by rewards from online traffic.

One (welcome) reality that jumps out in this series of Knight grants is that there are journalists from newsrooms that compete for stories who are collaborating on these projects independently. New York Times and Washington Post developers are teaming up to create an open election database. Data journalists from WNYC, the Chicago Tribune and the Spokesman-Review are collaborating on building a better interface for Census data. The same peer networks that helped build the Internet are forming around building out civic infrastructure. It’s an inspiring trend to watch.

The value of an open geo commons

The amount of consternation regarding Apple’s new mapping app for iOS 6 doesn’t seem to be dying down. It shouldn’t: David Pogue called the Apple Map app “an appalling first release,” and maybe “the most embarrassing, least usable piece of software Apple has ever unleashed.” It’s going to take a while for Apple Maps to improve — maybe even years, based upon how long it took for Google to improve maps. In the meantime, iPhone users can go to maps.google.com on Safari, along with the other third-party alternatives that Apple CEO Tim Cook recommended in his letter of apology.

In the wake of “#MAppleGate,” there’s suddenly a lot more attention being paid to the importance and value of mapping data, including how difficult it is to do maps right. And that’s where OpenStreetMap comes in. That’s also why the Knight Foundation is putting more than $500,000 behind tools from Development Seed: it will help to sustain and improve an open geo data commons that media organizations large and small can tap into to inform communities using maps.

“There are two ways the geo data space is going to evolve: 1) in closed silos of proprietary owned data or 2) in the open,” said Eric Gundersen, co-founder and CEO of Development Seed in a recent interview. “Our community does not need a fleet of cars driving millions of miles. We need good infrastructure to make it easy for people to map their surroundings and good community tools to help us garden the data and improve quality. As geo data becomes core to mobile, maps are a canvas to visualizing the ‘where’.”

As with Wikipedia, there will be people who doubt whether an open source digital map revolution enabled by MapBox, Development Seed’s open source mapping suite will come to pass. Then again, how many people believed a decade ago that Wikipedia would grow into the knowledge repository it is today?

“We are trying to radically lower the barrier of entry to map making for organizations and activists,” Gundersen told me last April. Given that they’re up against Google in mapmaking, the relatively tiny DC startup is banking on OpenStreetMap looking more like Wikipedia than Google Knol in a few years.

“Open” is in

Open data is a common thread that connects the winners — but the openness doesn’t stop there. Open maps. Open source. Open government. Open journalism. That this theme has emerged as a strong pulse isn’t a tremendous surprise, given a global movement to apply technology to open government. Moreover, no one should take this to mean that immense amounts of business, society, technology, media and government aren’t still closed. Clearly, that’s not the situation. But there’s a strong case to be made that open is the way of the day.

Data won’t save the world, on its own. However, when data is applied for the public good and put to work, there are a growing number of examples that raise optimism about data’s role in the future of journalism.

Photo Credit: Eric Fisher

September 20 2012

Congress launches Congress.gov in beta, doesn’t open the data

The Library of Congress is now more responsive — at least when it comes to web design. Today, the nation’s repository for its laws launched a new beta website at Congress.gov and announced that it would eventually replace Thomas.gov, the 17-year-old website that represented one of the first significant forays online for Congress. The new website will educate the public looking for information on their mobile devices about the lawmaking process, but it falls short of the full promise of embracing the power of the Internet. (More on that later).

Tapping into a growing trend in government new media, the new Congress.gov features responsive design, adapting to desktop, tablet or smartphone screens. It’s also search-centric, with Boolean search and, in an acknowledgement that most of its visitors show up looking for information, puts a search field front and center in the interface. The site includes member profiles for U.S. Senators and Representatives, with associated legislative work. In a nod to a mainstay of social media and media websites, the new Congress.gov also has a “most viewed bills” list that lets visitors see at a glance what laws or proposals are gathering interest online. (You can download a fact sheet on all the changes as a PDF).

On the one hand, the new Congress.gov is a dramatic update to a site that desperately needed one, particularly in a historic moment where citizens are increasingly connecting to the Internet (and one another) through their mobile devices.

On the other hand, the new Congress.gov beta has yet to realize the potential of Congress publishing bulk open legislative data. There is no application programming interface (API) for open government developers to build upon. In many ways, the new Congress.gov replicates what was already available to the public at sites like Govtrack.us and OpenCongress.org.

In response to my tweets about the site, former law librarian Meg Lulofs Kuhagan (@librarylulu) noted on Twitter that there’s “no data whatsoever, just window dressing” in the new site — but that “it looks good on my phone. More #opengov if you have a smartphone.”

Aaron E. Myers, the director of new media for Senator Major Leader Harry Reid, commented on Twitter that legislative data is a “tough nut to crack,” with the text of amendments, SCOTUS votes and treaties missing from new Congress.gov. In reply, Chris Carlson, the creative director for the Library of Congress, tweeted that that information is coming soon and that all the data that is currently in Thomas.gov will be available on Congress.gov.

Emi Kolawole, who reviewed the new Congress.gov for the Washington Post, reported that more information, including the categories Meyers cited, will be coming to the site soon, during its beta, including the Congressional Record and Index. Here’s hoping that Congress decides to publish all of its valuable Congressional Research Reports, too. Currently, the public has to turn to OpenCRS.com to access that research.

Carlson was justifiably proud of the beta of Congress.gov: “The new site has clean URLs, powerful search, member pages, clean design,” he tweeted. “This will provide access to so many more people who only have a phone for internet.”

While the new Congress.gov is well designed and has the potential to lead to more informed citizens, the choice to build a new website versus release the data disappointed some open government advocates.

“Another hilarious/clueless misallocation of resources,” commented David Moore, co-founder of OpenCongress. “First liberate bulk open gov data; then open API; then website.”

“What’s noticeable about this evolving beta website, besides the major improvements in how people can search and understand legislative developments, is what’s still missing: public comment on the design process and computer-friendly bulk access to the underlying data,” wrote Daniel Schuman, legislative counsel for the Sunlight Foundation. “We hope that Congress will now deeply engage with the public on the design and specifications process and make sure that legislative information is available in ways that most encourage analysis and reuse.”

Kolawole asked Congressional officials about bulk data access and an API and heard that the capacity is there but the approval is not. “They said the system could handle it, but they haven’t received congressional auth. to do it yet,” she tweeted.

Vision and bipartisan support for open government on this issue does exist among Congressional leadership. There has been progress on this front in the 112th Congress: the U.S. House started publishing machine-readable legislative data at docs.house.gov this past January.

“Making legislative data easily available in machine-readable formats is a big victory for open government, and another example of the new majority keeping its pledge to make Congress more open and accountable,” said Speaker of the House John Boehner.

Last December, House Minority Whip Steny Hoyer commented upon on how technology is affecting Congress, his caucus and open government in the executive branch:

For Congress, there is still a lot of work to be done, and we have a duty to make the legislative process as open and accessible as possible. One thing we could do is make THOMAS.gov — where people go to research legislation from current and previous Congresses — easier to use, and accessible by social media. Imagine if a bill in Congress could tweet its own status.

The data available on THOMAS.gov should be expanded and made easily accessible by third-party systems. Once this happens, developers, like many of you here today, could use legislative data in innovative ways. This will usher in new public-private partnerships that will empower new entrepreneurs who will, in turn, yield benefits to the public sector.

For any of that vision of civic engagement and entrepreneurship to can happen around Web, the Library of Congress will need to fully open up the data. Why hasn’t it happened yet, given bipartisan support and a letter from the Speaker of the House?

techPresident managing editor Nick Judd asked the Library of Congress about Congress.gov. The director of the communications for the Library of Congress, Gayle Osterberg, suggested in an email in response that Congress hasn’t been clear about the manner for data release.

“Congress has said what to do on bulk access,” commented Schuman. “See the joint explanatory statement. “There is support for bulk access.”

In June 2012, the House’s leadership has issued a bipartisan statement that adopted the goal of “provid[ing] bulk access to legislative information to the American people without further delay,” putting releasing bulk data among its “top priorities in the 112th Congress” and directed a task force “to begin its important work immediately.”

The 112th Congress will come to a close soon. The Republicans swept into the House in 2010 promising a new era of innovation and transparency. If Speaker Boehner, Rep. Hoyer and their colleagues want to end these two divisive years on a high note, fully opening legislative data to the People would be an enduring legacy. Congressional leaders will need to work with the Library of Congress to make that happen.

All that being said, the new Congress.gov is in beta and looks dramatically improved. The digital infrastructure of the federal legislative system got a bit better today, moving towards a more adaptive government. Stay tuned, and give the Library of Congress (@LibraryCongress) some feedback: there’s a new button for it on every page.

This post has been updated with comments from Facebook, a link and reporting from techPresident, and a clarification from Daniel Schuman regarding the position of the House of Representatives.

August 29 2012

President Obama participates in first Presidential AMA on Reddit

Starting around 4:30 PM ET today, President Barack Obama made history by going onto Reddit to answer questions about anything for an hour. Reddit, one of the most popular social news sites on the Internet, has been hosting “Ask Me Anything” forums — or AMAs – for years, including sessions with prominent legislators like Representative Darrell Issa (R-CA), but to host a sitting President of the United States will elevate Reddit’s prominence in the intersection of technology and politics. AllThingsD has the story of Reddit got the President onto the site. Reddit co-founder Alexis Ohanian told Peter Kafka that “There are quite a few redditors at 1600 Pennsylvania Ave and at the campaign HQ — given the prominence of reddit, it’s an easy sell.”

President Obama made some news in the process, with respect to the Supreme Court decision that allowed super political action committees, or “Super PACs,” to become part of the campaign finance landscape.

“Over the longer term, I think we need to seriously consider mobilizing a constitutional amendment process to overturn Citizens United (assuming the Supreme Court doesn’t revisit it),” commented President Obama. “Even if the amendment process falls short, it can shine a spotlight of the super-PAC phenomenon and help apply pressure for change.”

President Obama announced that he’d be participating in the AMA in a tweet and provided photographic evidence that he was actually answering questions in an image posted to Reddit (above) and in a second tweet during the session.

The timing of the AMA was at least a little political, coming after a speech in Virginia and falling upon the third day of the Republic National Convention, but it is unequivocally a first, in terms of a president directly engaging with the vibrant Reddit community. Many people also tweeted that they were having trouble accessing the page during the AMA, as tens of thousands of users tried to access the forum. According to The Verge, President Obama’s AMA was the most popular post in Reddit’s history, with more than 200,000 visitors on the site concurrently. (Presidential Q&As apparently melts servers almost as much as being Biebered.)

Today’s AMA is only the latest example of presidents experimenting with online platforms, from President Clinton and President Bush posting text on WhiteHouse.gov to President Obama joining rebooting that platform on Drupal. More recently, President Obama has participated in a series of online ‘town halls’ using social media, including Twitter, Facebook, LinkedIn and the first presidential Hangout on Google+.

His use of all them deserves to be analyzed critically, in terms of whether the platforms and events were being used to shine the credential of a tech-savvy chief executive in an election year or to genuinely answer the questions and concerns of the citizens he serves.

In analyzing the success of such experiment in digital democracy, it’s worth looking at whether the questions answered were based upon the ones most citizens wanted to see asked (on Reddit, counted by upvotes) and whether the answers given were rehashed talking points or specific to the intent of the questions asked. On the first part of that rubric, President Obama scored high: he answered each of the top-voted questions in the AMA, along with a few personal ones.

 

On the rest of those counts, you can judge for yourself. The president’s answers are below:

“Hey everybody – this is barack. Just finished a great rally in Charlottesville, and am looking forward to your questions. At the top, I do want to say that our thoughts and prayers are with folks who are dealing with Hurricane Isaac in the Gulf, and to let them know that we are going to be coordinating with state and local officials to make sure that we give families everything they need to recover.”

On Internet freedom: “Internet freedom is something I know you all care passionately about; I do too. We will fight hard to make sure that the internet remains the open forum for everybody – from those who are expressing an idea to those to want to start a business. And although their will be occasional disagreements on the details of various legislative proposals, I won’t stray from that principle – and it will be reflected in the platform.”

On space exploration: “Making sure we stay at the forefront of space exploration is a big priority for my administration. The passing of Neil Armstrong this week is a reminder of the inspiration and wonder that our space program has provided in the past; the curiosity probe on mars is a reminder of what remains to be discovered. The key is to make sure that we invest in cutting edge research that can take us to the next level – so even as we continue work with the international space station, we are focused on a potential mission to a asteroid as a prelude to a manned Mars flight.”

On helping small businesses and relevant bills: “We’ve really focused on this since I came into office – 18 tax cuts for small business, easier funding from the SBA. Going forward, I want to keep taxes low for the 98 percent of small businesses that have $250,000 or less in income, make it easier for small business to access financing, and expand their opportunities to export. And we will be implementing the Jobs Act bill that I signed that will make it easier for startups to access crowd-funding and reduce their tax burden at the start-up stage.”

Most difficult decision you had to make this term? ”The decision to surge our forces in afghanistan. Any time you send our brave men and women into battle, you know that not everyone will come home safely, and that necessarily weighs heavily on you. The decision did help us blunt the taliban’s momentum, and is allowing us to transition to afghan lead – so we will have recovered that surge at the end of this month, and will end the war at the end of 2014. But knowing of the heroes that have fallen is something you never forget.”

On the influence of money in politics ”Money has always been a factor in politics, but we are seeing something new in the no-holds barred flow of seven and eight figure checks, most undisclosed, into super-PACs; they fundamentally threaten to overwhelm the political process over the long run and drown out the voices of ordinary citizens. We need to start with passing the Disclose Act that is already written and been sponsored in Congress – to at least force disclosure of who is giving to who. We should also pass legislation prohibiting the bundling of campaign contributions from lobbyists. Over the longer term, I think we need to seriously consider mobilizing a constitutional amendment process to overturn Citizens United (assuming the Supreme Court doesn’t revisit it). Even if the amendment process falls short, it can shine a spotlight of the super-PAC phenomenon and help apply pressure for change.”

On prospects for recent college grads – in this case, a law school grad: I understand how tough it is out there for recent grads. You’re right – your long term prospects are great, but that doesn’t help in the short term. Obviously some of the steps we have taken already help young people at the start of their careers. Because of the health care bill, you can stay on your parent’s plan until you’re twenty six. Because of our student loan bill, we are lowering the debt burdens that young people have to carry. But the key for your future, and all our futures, is an economy that is growing and creating solid middle class jobs – and that’s why the choice in this election is so important. The other party has two ideas for growth – more taxs cuts for the wealthy (paid for by raising tax burdens on the middle class and gutting investments like education) and getting rid of regulations we’ve put in place to control the excesses on wall street and help consumers. These ideas have been tried, they didnt work, and will make the economy worse. I want to keep promoting advanced manufacturing that will bring jobs back to America, promote all-American energy sources (including wind and solar), keep investing in education and make college more affordable, rebuild our infrastructure, invest in science, and reduce our deficit in a balanced way with prudent spending cuts and higher taxes on folks making more than $250,000/year. I don’t promise that this will solve all our immediate economic challenges, but my plans will lay the foundation for long term growth for your generation, and for generations to follow. So don’t be discouraged – we didn’t get into this fix overnight, and we won’t get out overnight, but we are making progress and with your help will make more.”

First thing he’ll do on November 7th: “Win or lose, I’ll be thanking everybody who is working so hard – especially all the volunteers in field offices all across the country, and the amazing young people in our campaign offices.”

How do you balance family life and hobbies with being POTUS? ”It’s hard – truthfully the main thing other than work is just making sure that I’m spending enough time with michelle and the girls. The big advantage I have is that I live above the store – so I have no commute! So we make sure that when I’m in DC I never miss dinner with them at 6:30 pm – even if I have to go back down to the Oval for work later in the evening. I do work out every morning as well, and try to get a basketball or golf game in on the weekends just to get out of the bubble. Speaking of balance, though, I need to get going so I’m back in DC in time for dinner. But I want to thank everybody at reddit for participating – this is an example of how technology and the internet can empower the sorts of conversations that strengthen our democracy over the long run. AND REMEMBER TO VOTE IN NOVEMBER – if you need to know how to register, go to Gottaregister.com. By the way, if you want to know what I think about this whole reddit experience – NOT BAD!”

On +The White House homebrew recipe ”It will be out soon! I can tell from first hand experience, it is tasty.”

A step forward for digital democracy?

The most interesting aspect of that Presidential Hangout was that it introduced the possibility of unscripted moments, where a citizen could ask an unexpected question, and the opportunity for followups, if an answer wasn’t specific enough.

Reddit doesn’t provide quite the same mechanism for accountability at a live Hangout, in terms of putting an elected official on the spot to answer. Unfortunately, the platform of Reddit itself falls short here: there’s no way to force a politician to circle back and give a better answer, in the way, say, Mike Wallace might have on “60 Minutes.”

Alexis Madrigal, one of the sharpest observers of technology and society currently gracing the pages of the Atlantic, is clear about the issues with a Reddit AMA: “it’s a terrible format for extracting information from a politician.”

Much as many would like to believe that the medium determines the message, a modern politician is never unmediated. Not in a pie shop in Pennsylvania, not at a basketball game, not while having dinner, not on the phone with NASA, not on TV, not doing a Reddit AMA. Reddit is not a mic accidentally left on during a private moment. The kind of intimacy and honesty that Redditors crave does not scale up to national politics, where no one ever lets down his or her guard. Instead of using the stiffness and formality of the MSM to drive his message home, Obama simply used the looseness and casual banter of Reddit to drive his message home. Here more than in almost anything else: Tech is not the answer to the problems of modern politics.

Today’s exchange, however, does hint at the tantalizing dynamic that makes it alluring: that the Internet is connecting you and your question to the most powerful man in the world, directly, and that your online community can push for him to answer it.

President Obama ended today’s AMA by thanking everyone on Reddit for participating and wrote that “this is an example of how technology and the internet can empower the sorts of conversations that strengthen our democracy over the long run.”

Well, it’s a start. Thank you for logging on today, Mr. President. Please come back online and answer some more follow up questions.

Reposted byRK RK

August 03 2012

Palo Alto looks to use open data to embrace ‘city as a platform’

In the 21st century, one of the strategies cities around the world are embracing to improve services, increase accountability and stimulate economic activity is to publish open data online. The vision for New York City as a data platform earned wider attention last year, when the Big Apple’s first chief digital officer, Rachel Sterne, pitched the idea to the public.

This week, the city of Palo Alto in California joined over a dozen cities around the United States and globe when it launched its own open data platform. The platform includes an application programming interface (API) which enables direct access through a RESTful interface to open government data published in a JSON format. Datasets can also be embedded like YouTube videos, as below:

“We’re excited to bring the value of Open Data to our community. It is a natural complement to our goal of becoming a leading digital city and a connected community,” said James Keene, Palo Alto City Manager, in a prepared statement. “By making valuable datasets easily available to our residents, we’re further removing the barriers to a more inclusive and transparent local government here in Palo Alto.”

The city initially published open datasets that include the 2010 census data, pavement condition, city tree locations, park locations, bicycle paths and hiking trails, creek water level, rainfall and utility data. Open data about Palo Alto budgets, campaign finance, government salaries, regulations, licensing, or performance — which would all offer more insight into traditional metrics for government accountability — were not part of this first release.

“We are delighted to work with a local, innovative Silicon Valley start-up,” said Dr. Jonathan Reichental, Palo Alto’s chief information officer, in a prepared statement. (Junar’s U.S. offices are in Palo Alto.) “Rather than just publishing lists of datasets, the cloud-based Junar platform has enhancement and visualization capabilities that make the data useful even before it is downloaded or consumed by a software application.”

Notably, the city chose to use Junar, a Chilean software company that raised $1.2 million dollars in funding in May 2012. Junar provides data access in the cloud through the software-as-a-service model. There’s now a more competitive marketplace for open data platforms than has existed in years past, with a new venture-backed startup joining the space.

“The City of Palo Alto joins a group of forward-thinking organizations that are using Open Data as a foundation for more efficient delivery of services, information, and enabling innovation,” said Diego May, CEO and co-founder of Junar, in a prepared statement. “By opening data with the Junar Platform, the City of Palo Alto is exposing and sharing valuable data assets and is also empowering citizens to use and create new applications and services.”

The success or failure of Palo Alto’s push to become a more digital city might be more fairly judged in a year, when measuring downstream consumption of its open data in applications and services by citizens — or by government in increasing productivity — will be possible.

In the meantime, Reichental (who may be familiar to Radar readers as O’Reilly Media’s former CIO) provided more perspective via email on what he’s up to in Palo Alto.

What does it mean for a “city to be a platform?”

Reichental: We think of this as both a broad metaphor and a practicality. Not only do our citizens want to be plugged in to our government operations — open data being one way to achieve this among others — but we want our community and other interested parties to build capability on top of our existing data and services. Recognizing the increasing limitations of local government means you have to find creative ways to extend it and engage with those that have the skills and resources to build a rich and seamless public-private partnership.

Why launch an open data initiative now? What success stories convinced you to make the investment?

Reichental: It’s a response to our community’s desire to easily access their data and our want as a City to unleash the data for better community decision-making and solution development.

We also believe that over time an open data portal will become a standard government offering. Palo Alto wants to be ahead of the curve and create a positive model for other communities.

Seldom does a week pass when a software engineer in our community doesn’t ask me for access to a large dataset to build an app. Earlier this year, the City participated in a hackathon at Stanford University that produced a prototype web application in less than 24 hours. We provided the data. They provided the skills. The results were so impressive, we were convinced then that we should scale this model.

How much work did it take to make your data more open? Is it machine-readable? What format? What cost was involved?

Reichental: We’re experimenting with running our IT department like a start-up, so we’re moving fast. We went from vendor selection to live in just a few weeks. The data in our platform can be exported as a CSV or to a Google Spreadsheet. In addition, we provide an API for direct access to the data. The bulk of the cost was internal staff time. The actual software, which is cloud-based, was under $5000 for the first year.

What are the best examples of open data initiatives delivering sustainable services to citizens?

Reichental: Too many to mention. I really like what they’re doing in San Francisco (http://apps.sfgov.org/showcase/) but there are amazing things happening on data.gov and in New York City. Lots of other cities in the US doing neat things. The UK has done some high-quality budget accountability work.

Are you consuming your own open data?

Reichental: You bet we are.

Why does having an API matter?

Reichental: We believe the main advantage of having an API is for app development. Of course, there will be other use cases that we can’t even think of right now.

Why did you choose Junar instead of Socrata, CKAN or the OGPL from the U.S. federal government?

Reichental: We did review most of the products in the marketplace including some open source solutions. Each had merits. We ultimately decided on Junar for a 1-year commitment, as it seemed to strike the right balance of features, cost, and vision alignment.

Palo Alto has a couple developers in it. How are you engaging them to work with your data?

Reichental: That’s quite the understatement! The buzz already in the developer community is palpable. We’ve been swamped with requests and ideas already. We think one of the first places we’ll see good usage is in the myriad of hackathons/code jams held in the area.

What are the conditions for using your data or making apps?

Reichental: Our terms and conditions are straightforward. The data can be freely used by anyone for almost any purpose, but the condition of use is that the City has no liability or relationship with the use of the data or any derivative.

You told Mashable that you’re trying to acting like a “lean startup.” What does that mean, in practice?

Reichental: This initiative is a good example. Rather than spend time making the go-live product perfect, we went for speed-to-market with the minimally viable solution to get community feedback. We’ll use that feedback to quickly improve on the solution.

With the recent go-live of our redesigned public website, we launched it initially as a beta site; warts and all. We received lots of valuable feedback, made many of the suggested changes, and then cutover from the beta to production. We ended up with a better product.

Our intent is to get more useful capability out to our community and City staff in shorter time. We want to function as close as we can with the community that we serve. And that’s a lot of amazing start-ups.

July 30 2012

Mobile participatory budgeting helps raise tax revenues in Congo

In a world awash in data, connected by social networks and focused on the next big thing, stories about genuine innovation get buried behind the newest shiny app or global development initiative. For billions of people around the world, the reality is that inequality in resources, access to education or clean water, or functional local government remain serious concerns.

South Kivu, located near the border of the Democratic Republic of Congo, has been devastated by the wars that have ravaged the region over the past decade.

Despite that grim context, a pilot program has born unexpected fruit. Mobile technology, civic participation, smarter governance and systems thinking combined to not only give citizens more of a voice in their government but have increased tax revenues as well. Sometimes, positive change happens where one might reasonably least expect it. The video below tells the story. After the jump, World Bank experts talk about story behind the story.

“Beyond creating a more inclusive environment, the beauty of the project in South Kivu is that citizen participation translates into demonstrated and measurable results on mobilizing more public funds for services for the poor,” said Boris Weber, team leader for ICT4Gov at the World Bank Institute for Open Government

, in an interview in Washington. “This makes a strong case when we ask ourselves where the return of investment of open government approaches is.”

Gathering support

The World Bank acted as a convener, in this context, said Tiago Peixoto, an open government specialist at the World Bank, in an interview. The Bank brought together the provincial government and local government to identify the governance issues and propose strategies to address them.

The challenge was straightforward: the South Kivu provincial government needed to relay revenues to the lower level of government to fund services but wasn’t doing so, both because lack of incentives and concerns about how the funds would be spent.

What came out of a four day meeting was a request for a feasibility study on participatory budgeting from the World Bank, said Peixoto.

Initially, the Bank found good conditions with respect to strong civil society, despite years of war. They found a participatory budgeting expert in Cameroon, who came and did workshops with local governments in how the process would work. They chose some cities as control groups, to introduce some scientific rigor.

They shared scholarship on participatory budgeting with all the stakeholders, emphasizing that research shows participation is more effective than penalties in taxation compliance.

“It’s like the process of ownership,” said Peixoto in our interview. “Once you see where money is going, you see how government can work. When you see a wish list, where some things happen and others do not because people aren’t paying, it changes perspectives.”

Hitting the books

When asked to provide more context on the scholarship in this area, Peixoto obliged, via email.

“As shown in a cross-national analysis by Torgler & Schneider (2009), citizens are more willing to pay taxes when they perceive that their preferences are properly taken into account by public institutions,” he wrote.

“Along these lines, the existing evidence suggests the existence of a causal relationship between citizen participation processes and levels of tax compliance. For instance, studies show that Swiss cantons with higher levels of democratic participation present lower tax evasion rates (Pommerehne & Weck-Hannemann 1996, Pommerehne & Frey 1992, Frey 1997). This effect is particularly strong when it comes to direct citizen participation in budgetary decisions, i.e. fiscal referendum (Frey & Feld 2002, Frey et al. 2004, Torgler 2005):

“The fiscal exchange relationship between taxpayers and the state also depends on the politico-economic framework within which the government acts. It has, in particular, been argued that the extent of citizens’ political participation rights systematically affects the kind of tax policy pursued by the government and its tax authority. (…) The more direct democratic the political decision-making procedures of a canton are, the lower is tax evasion according to these studies” (Feld & Frey 2005:29)

“According to his (Torgler) estimates, tax morale is significantly higher in direct democratic cantons. Distinguishing between different instruments of direct democracy, he finds that the fiscal referendum has the highest positive influence on tax morale” (Feld & Frey 2005:19)

Participatory budgeting, which has been gaining more attention in cities in the United States as more governments implement open government initiatives, has had particular success in Brazil, pointed out Peixoto, who is native to that country.

“In the Latin American context, a number of authors have observed a similar relationship with regard to participatory budgeting processes,” wrote Peixoto.

“In the municipality of Porto Alegre (BR) for instance, Schneider and Baquero (2006) show that the adoption of PB led to a substantive increase in tax revenues. In another study Zamboni (2007) compares the performance of similar Brazilian municipalities with and without PB processes: even when controlling for other factors, the study finds a significant relationship between the existence of PB and the increase in tax revenues. Another comparative study of 25 municipalities in Latin America and Europe also finds a significant reduction in levels of tax delinquency after the adoption of PB (Cabannes 2004):

“What is the relationship between the PB process and the municipality’s tax revenues? Most respondent cities indicated that the PB process entailed an increase in tax revenues and a decrease in delinquency. In Campinas, Recife and Cuenca, tax revenues increased significantly in a very few years; in Porto Alegre, property tax delinquency dropped from 20 per cent to 15 per cent and, in less than ten years, property taxes grew from 6 per cent to almost 12 per cent of the municipality’s revenues. Mundo Novo, in Brazil,also emphasized the drop in tax delinquency and relates it to the transparency of public administration entailed by PB. The immediate visibility of the work and services that result from PB also tends to change the citizenry’s taxpaying habits.” (Cabannes 2004:36)

Presenting the mayors with the results of that research provided them a strong incentive to try participatory budgeting, emphasized Peixoto. The results from the pilot, however, provided evidence of the efficacy of the practice:

The World Bank found that tax compliance in Kabare went from 7% to 12% in Kabare. In Ibanda, the impact of the pilot was even greater, with 16-fold increase in tax compliance. After the pilot, the provincial government decided to start transferring money to local areas but only if cities used participatory budgeting in the process.

“This was an eye-opening process,” said Jean Bunani, senior counsel in the Ministry of Budget in South Kivu

, in a prepared statement provided by the World Bank. Bunani was one of the beneficiaries of the project. “As a result, the province started transferring funds to local governments to start providing basic services to citizens,” he said. “This had been mandated by law for years – and for years the law had been ignored.”

Peixoto insisted that the results in South Kivu be interpreted with caution. “It is difficult to confirm a causal relationship participatory budgeting and the increase in tax compliance at scientific levels thus far,” he said, “but the evidence collected thus and testimonials of local officials suggests the existence of this causality.”

Mobile technology helped increase civic participation

“The way the citizens and the provincial Government of South Kivu took ownership of this project shows that technology can help build more inclusive decision making processes even in fragile and low-tech environments,” said Weber.

The vast majority of people in the area don’t have computers. Mobile phones may be one of the most important technologies to enter the region in decades – and people value them, walking long distances to generators to keep them charged.

“This is a place where they don’t have electricity for houses but they charge phones,” said Weber.

The mobile technology initiative was coordinated with the cooperation of the local mobile operators and funded by the World Bank. 1 million text messages cost $10,000, when purchased in bulk. Over 250,000 messages have been sent in support of the project, as of February 2012. Whenever there was a region meeting to deliberate about where to spend budgets funds, every handset under the local cellular towers would get a text message about it. And after the meeting, everyone would get a message with the results.

“The benefits from the participatory budgeting outweigh those costs enormously,” said Peixoto. “For more people to pay taxes, they need to know participatory budgeting exists. That’s the mass mobilization. There was already a substantial increase in the year 2010, when the local government started consulting grassroots organizations on an informal basis. Nevertheless, the process gains steam in 2011, in which the full methodology of participatory budgeting is really in place, with direct participation of the citizens and with the support of mobile telephony. Please note that these results are evaluated with control groups. In other words, in cities without the participatory budgeting process, the same behavior is not identified.”

Peixoto followed up with a review of research on public participation. “Some evidence suggests that participation may be even more effective at curbing tax evasion than traditional deterrence measures, such as fines and controls,” he wrote. “At odds with conventional economic reasoning, the literature in the field of ‘tax morale’ suggests that citizen participation actually comes across as a better remedy for tax evasion than commonly adopted deterrence policies (e.g. Torgler 2005, Feld & Frey 2007, Feld & Torgler 2007).

Weber noted that it’s hard to know exactly what the mobile penetration is in the South Kivu area. Unreliable research estimates put handset ownership at 14%, he said, but people also share devices, which in turn means that it’s not the most accurate estimate for research purposes.

“The new global focus on results-based aid is creating strong demand for better feedback data,” said Weber.

“This type of initiative can provide it and help crowdsource the monitoring of development impact. As more governments start to make commitments as part of the Open Government Partnership, they now need to stand up to the challenge on how to do engage citizens in a meaningful manner,” said Weber. “This type of project provides us with valuable lessons how donors can support governments in this effort.”

Finding the ROI for open government

This project clearly shows some of benefits associated with open government, said Peixoto, but only when citizens are involved in the process. Technology, in that context, is an enabler but is not sufficient by itself.

“These benefits are only generated when there is real engagement of the citizens,” said Peixoto. “Both the politicians and those working on the ground very convinced that the transparency of the budget in itself would not suffice to generate the results that we now observe. Nevertheless, to ensure that real engagement happens is an extremely long process, which cannot take place without having all the stakeholders involved.”

In that context, making participatory budgeting work using mobile devices isn’t just about working with the mayors, regional government, development officials, citizens or telecommunications companies: it’s about systems thinking and collaboration between all of the stakeholders.

“While we spend 10% of our time to convince governments to make their budgets transparency, the other 90% is convincing them to let citizens having a real say on where the money is going,” said Peixoto. “The road from transparency to accountability is neither obvious, nor an easy one.”

He also noted that technology complements what he describes a core components of citizen engagement, participatory design and institutional reform. Which is to say, rethinking processes and institutions come first, followed by figuring out how to architect technology to support them. In the Congo, mobile phones supported participatory budgeting by reducing associated costs, avoiding elite capture, maintaining public engagement and raising awareness of the process, which helped gather popular support.

Exploring mobile government

South Kivu is also experimenting with mobile phone voting, including some beta tests during the budget meetings.

“Mobile voting is expected to be implemented in full-scale, enabling a large number of inhabitants to remotely participate in the process of budget allocation,” said Peixoto. “The program is now going beyond the pilots, aiming to institutionalize ICT facilitated participatory budgeting in other provinces in the DR Congo and beyond.”

Weber said that after the pilots, 100% of citizens asked preferred to vote by mobile. “With ballots, there were huge lines,” he said. “It’s really about the costs of participation. This spoke to them.”

An important next step will be finding more ways to bridge the digital divide to engage the community in an ongoing dialogue about governance, including more analog methods like “data murals.”

“We are interested in inverting the logics of innovation,” wrote Peixoto. “The replication of contents and processes from the offline world into the online world has been the focus of people working in this field for a long time. Now we are looking how we can bring elements of the online world to the offline reality of South-Kivu, where access to the Internet is extremely scarce. Hence, one of our priorities for the beginning of next year is the creation of ‘data murals’ in which budget visualizations often available in online environments will be painted on the walls of the cities. If the logic is that of ‘going where citizens already are,’ in South-Kivu we will bring data and data visualization to the streets: budget data in citizen readable format.”

For instance, in Brazil, they’re painting the numbers for the budget on the wall, said Peixoto. “So, you could paint data visualizations on street walls. Fancy online data visualizations are very nice – but what if there’s no Internet? You need to get creative. We want to bring data vizualization to the streets, with a physical version of a dashboard where you go back and do updates as steps happen.”

Peixoto expects that the experience in South Kivu will also help inform how upcoming participatory budgeting initiatives mediated with mobile technology will be implemented elsewhere in the world, from Brazil to Cameroon to the Dominican Republic.

“Organizations working with participatory budgeting in the United States and Europe have already demonstrated interest in learning from the experience of South-Kivu and the use of mobile phones,” observed Peixoto, who said that the approach is already being replicated in Cameroon, the Dominican Republic and (back) in Brazil.

“In Cameroon, we are making a randomized experiment to assess the impact of SMS as a means to mobilize citizens and avoid elite capture,” he said.

The World Bank as set up a beta website for the Cameroon project and is working on another one with the Open Knowledge Foundation, similar to “Where does my money go?” The visualizations will be used to inform the participatory budgeting process, providing citizens with the means to use those visualizations to indicate where they’d like money to be allocated.

“This all goes to show that innovations in open government go both ways, from developing to developed countries,” said Peixoto. “The fact that people are not blogging about it in English does not mean that it does not exist. Sometimes people are just too busy making it happen. “

July 25 2012

Rethinking regulatory reform in the Internet age

As the cover story of a February issue of The Economist highlighted, concerns about an over-regulated America are cresting in this election year, with headlines from that same magazine decrying “excessive environmental regulation” and calling for more accurate measurement of the cost of regulations. Deleting regulations is far from easy to do but there does appear to be a political tailwind behind doing so.

As a legislator and chairman of the Government Oversight and Reform Committee, it’s fair to say that Representative Darrell Issa (D-CA) been quite active in publicly discussing the issue of regulations and regulatory burdens upon business. As a former technology entrepreneur, and a successful one at that (he’s the wealthiest member of Congress) Rep. Issa does have first-hand knowledge of what it takes to run a business, to bring products to market, and to deal with the various regulations.

In a wide-ranging interview earlier this summer, Rep. Issa commented on a number of issues related to open government and the work of the committee. When we talked about smart disclosure and the reforming the Freedom of Information Act, I posed several questions about regulatory data, in the context of its role in the marketplace for products and services. Our interview on regulation is below, followed by a look at how his office and the White House are trying to use the Web to improve regulatory reform and involve citizens in the debate.

What role does the release of regulatory data from the various agencies, in the form of smart disclosure or other directions, have in creating market transparency, bringing products to market or enabling citizens to understand the quality of said products? What is the baseline for regulation? For instance, after flying a lot recently, I’ve felt grateful the FAA had regulations that meant my flights would be safes when I flew back and forth across the country or ocean. There’s some baseline for the appropriate amount of regulation but it’s never entirely clear what that might be.

Rep. Issa: I’ll give you a good example of why regulations that you believe in, you don’t believe in. Do you believe it’s dangerous to have your cell phone on as you’re going across country?

My understanding is that it is extremely likely that many people’s cellphones have been, in fact, left on while they fly cross country or while they take off and land. The probability of people not having switched them off is high. To date, I have not heard a documented case where a switched on cellphone interfered with the navatronics of the plane. [See Nick Bilton's reporting on the FAA and gadgets in the New York Times.] That logically suggests to me that it’s not as much of a risk as has been posited, but I haven’t seen the data.

Rep Issa: So, on a regulatory basis, your country is lying to you. I’m making the statement as I’m asking the question. Of course your country’s lying to you about the risk. Of course there’s a valid reason to turn off your cell phone: it’s so you won’t be distracted while they’re telling you where the exit is. So rather than say, “Look, we have the right to have you shut off your cellphone and we believe that for safety purposes you should do it, but let’s not kid each other: If you’ve got it on final so you can get your emails a little earlier by 30 seconds and you don’t mind your battery going dead a little faster, it probably has no real risk.’

The fact is your government has regulatory power to regulate an action for which they don’t actually have a good faith belief it’s causing damage. Just the opposite: they have the knowledge that these units are on all the time by accident, in people’s luggage, and our planes still don’t crash.

My problem with regulations is they need to have a cost benefit. And that cost benefit, the burden has to be against the regulator, not for the regulator. So when the EPA says, “You need to take the arsenic out of water,” as they did a number of years ago, and it sounded great, but the number was arbitrary and they had no science. And what ended up happening in New Mexico was that people’s small water districts went out of business. In some cases, people went back to taking what was ever in their well and you go, “Well, why didn’t they have a number that they could justify you absolutely had to have otherwise it was hurting you?” Well, the answer is because they never did the science, they just did the regulations.

So where does the balance lie, in your opinion?

Rep Issa: When it comes to individual liberty, I try to be as absolute as possible. When it comes to regulatory needs, I tend to be as limited as possible, both because of people’s liberty, but also because government has a tendency to want to grow itself. And if you let it grow itself, one day you wake up like the frogs that were slowly boiled because they were put in the water and didn’t notice it getting warm until they were cooked.

When I’ve traveled abroad, I’ve heard from citizens of other countries, particularly in the developing world, that one of the things that they admire about the U.S. is that we have an FDA, an EPA, an FTC and other regulatory bodies which they see holding our quite powerful corporations to some level of account. What do role those institutions have in the 21st Century to hold private interests, which have incredible amounts of power in our world, accountable for the people?

Issa: I gave you the EPA example because there was a debate that ultimately the EPA won on arsenic to the detriment of whole communities who disagreed, who said, you haven’t made the case as to why you picked a particular level. They all supported the idea that water should be clean. The question is at what point of the cost-benefit was it the right level of clean. And I remember that one.

Let me give you one in closing that’s probably perfect. Today, the FDA is unable to ensure that generic cancer and antibiotics are in sufficient supply, which was one of its mandates. And as a result, there’s a whole bootleg market developing — and the left and the right are both concerned about it — for both cancer and antibiotics because there’s a shortage. But the FDA had a regulatory responsibility to ensure that the shortage didn’t occur and they’re failing it. So the FDA has a job it’s not doing.

Additionally, people are traveling to Europe and other places to get drugs which are saving lives because they’re getting approved in those countries quicker. These are western countries with the equivalent of FDA, but they’re getting approved quicker and clinical trials are going better and moving over there.

So when we look at the FDA, you’re not attacking them because you think you shouldn’t have the Food and Drug Administration dealing with particularly the efficacy of medicines, but because the FDA is falling short in the speed to market, getting longer and longer, meaning people are being denied innovative drugs.

Can the Web help with regulatory reform and e-rulemaking?

Representative Issa, whose committee heard testimony on regulatory impediments to job creation last week, is not alone in the U.S. House in his interest in streamlining regulations. This week, Speaker Boehner and his caucus have been pushing to “cut the red tape” limiting or loosening regulations on small businesses until unemployment falls to 6%.
The administration has not been inactive on this front, although it’s fair to say that House Republicans have made clear that its progress towards regulatory reform to date has been unsatisfactory. One early case study can be found in FCC open Internet rules and net neutrality, where OpenInternet.gov was used to collect public feedback for proposed rules. Public comments on OpenInternet.gov were officially entered as official comment, which was something of a watershed in e-rulemaking. The full version of the final rules, however, were not shared with the public until days after they were voted upon.

In January 2011, President Barack Obama issued an executive order focused on reforming regulation regulatory review. One element of the order was particularly notable for observers who watch to see whether citizen engagement is part of open government efforts by this administration: its focus upon public participation in the regulatory process.
As I’ve
written elsewhere, this order is part of a larger effort towards e-rulemaking by the administration. In February 2012, Regulations.gov relaunched with an API and some social media features, with an eye towards gaining more public participation. This electronic infrastructure will almost certainly be carried over into future administrations, regardless of the political persuasion of the incumbent of the Oval Office.

This summer, Cass Sunstein, the administrator of the Office for Information and Regulatory Affairs in the White House, asked the American people for more ideas on how the federal government could “streamline, simplify or eliminate federal regulations to help businesses and individuals.”

As the Wall Street Journal reported last year, the ongoing regulatory review by OIRA is a nod to serious, long-standing concerns in the business community about excessive regulation hampering investment and job creation as citizens struggle to recover from the effects of the Great Recession.

It’s not clear yet if an upgraded Regulations.gov will makes any difference in the quality of regulatory outcomes. Rulemaking and regulatory review are, virtually by their nature, wonky and involve esoteric processes that rely upon knowledge of existing laws and regulations.

In the future, better outcomes might come from smart government approaches, through adopting what Tim O’Reilly has described “algorithmic regulation,” applying the dynamic feedback loops that Web giants use to police their systems against malware and spam in government agencies entrusted with protecting the public interest.

In the present, however, while the Internet could involve many more people in the process, improved outcomes will depend upon an digitally literate populace that’s willing to spend some of its civic surplus on public participation in identifying problematic regulations. That would mean legislators and staff, regulators and agency workers to use the dynamic social Web of 2012 to listen as well as to broadcast.

To put it another way, getting to “Regulations 2.0″ will require “Citizen 2.0″ — and we’ll need the combined efforts of all our schools, universities, libraries, non-profits and open government advocates to have a hope of successfully making that upgrade.

Do citizens have a ‘right to record’ in the digital age?

When Representative Darrell Issa (R-CA) and I talked this summer about his proposal for a digital Bill of Rights, I followed up by asking him about whether it might be more productive to focus on the rights that we already have in the digital context.

That conversation naturally led to a question about freedom of assembly and freedom of the press, both of which came under some pressure in the United States during the Occupy protests of the past year. Our interview follows.

How can we make sure that the ‘inalienable rights’ that we are endowed with already are receiving oversight and enforcement from our representatives?

Rep Issa: I think that when we’re aghast at what China’s doing to Google, it helps us say we’re so upset about that. I think we need to take examples, we need to see what we don’t like and what the American people want us to protect. Sometimes you look abroad for really bad behavior and then you look internally to find similar behavior, maybe a quantum leap lower, but it’s still there.

You mentioned Washington D.C. and Occupy. I think that’s a classic example where free speech was turned into free camping. The rights of the public broadly to enjoy an asset that was set aside for public use [were involved], where the Mayor — who doesn’t happen to be from my party or even my ideology — comes to us and says, “We’ve got rats, we’ve got crud. We’ve got all of these things that are spreading into the rest of the city. These people are not exercising their free rights for most of the day, what they’re doing is camping on grounds that were not designed or built or prepared for that.”

As it went on week after week after week, it wasn’t camping overnight, between the day you arrive and the protest the next day: it was effectively ‘living in.’ That’s a good example where the rules are pretty well understood, the history is pretty understood, and the enforcement at the end was pretty consistent with what it’s been over the years. Your presence can be a protest, but, at some point, your presence becomes simply an impediment to other people’s rights.

Are you concerned that there have been dozens of journalists arrested at Occupy protests? And I don’t mean just citizens livestreaming what’s happening, although one could make a case that they’re committing ‘acts of journalism.’ I’m referring to credentialed journalists arrested while they’re actively chronicling what law enforcement is doing to their fellow citizens.

Rep Issa: There’s a separate question, which is whether law enforcement is entitled to the cloak of secrecy as they pull you over for a DWI and their camera isn’t on when they rough you up but is on when you resist arrest. Those are areas of personal liberty.

We’re not dealing ‘digitally,’ but we are dealing with an era in which a policeman or other individuals demand the rights to video you involuntarily, when it suits them, and then object if you want to video their doing the same event but from an independent perspective.

Do I think the court has to rule on that? Absolutely. Do I think you have to find the anecdotal examples of most egregious behavior in order to prove the point? Probably. But I think there have been a number of them.

When they talk about arresting journalists, whether credentialed or not, the court has to weigh in and say the police should not be afraid of a camera. If they’re afraid of a camera, they might be afraid of a witness. And if they’re afraid of you and I watching for some valid reason, great. But if you and I watching and the equivalent, digitally capturing it, then they’ve crossed a line.

I want to be careful. Some of the arrests of journalists, some of those arrest examples include, basically, misbehavior of journalists getting in the face of people, shoving cameras at them and asking questions designed to be less than what you would call passive. Those are not necessarily the best example. It’s sort of like you look at the paparazzi and Princess Di dying: it wasn’t the finest day to claim that paparazzi had rights.

On the other hand, if it’s somebody who from a distance who is observing and video recording an actual arrest of or holding of some individual who is simply walking down the street and says, “Hey, don’t stop me, you haven’t got a right” — the two are very different. And passive observation by the press is a better one to take to the court because it’s a slam dunk First Amendment [case].

Aggressive behavior by press who get in the face and blocks somebody trying to move is always a little bit more of a call where you and I could probably find a point in which we would say that the First Amendment line has been crossed in that somebody else’s rights have been infringed.

This is important, because this is where the left and the right should come to a common ground. Strict adherence to rights, even when it’s inconvenient, is part of what makes America a better country.

Addendum

This week, the Chief of the Metropolitan Police in the District of Columbia issued an order [PDF] affirming the public’s right to photograph and film police officers who are performing official business. The MPD’s action came as part of a court-mandated settlement of a lawsuit brought by Jerome Vorus, who claimed he was wrongly detained by the police department after photographing police activity.

The order “recognizes that members of the general public have a First Amendment right to video record, photograph, and/or audio record MPD members while MPD members are conducting official business or while acting in an official capacity in any public space, unless such recordings interfere with police activity.”

Given instances of documented interference with credentialed media in the city of New York, such guidance might be useful in the five borough as well.

Does the Open Government Partnership merit more oversight and attention?

Brazilian President Dilma Roussef speaks at the 2012 annual Open Government Partnership conference

Brazilian President Dilma Roussef speaks at the 2012 annual Open Government Partnership conference

There are any number of responsibilities and challenges inherent in moving forward with the historic Open Government Partnership (OGP) that officially launched last September. Global Integrity’s recent assessment of the National Action plans submitted to the Open Government Partnership by participating countries found cause for both concern and optimism, As I’ve highlighted elsewhere previously.

The National Action Plan commits the United States to 18 different open government initiatives, including implementing the Extractive Industries Transparency Initiative (EITI). One of the primary functions of the committee that Representative Darrell Issa (R-CA) chairs in the U.S. House is to provide oversight of what’s happening in the Executive Branch of government. In that context, the Government Oversight and Reform has an important role in overseeing not just what the proposals are but how they’re actually executed by agencies. In March 2011, the committee held a hearing on open government initiatives in the United States.

Earlier this summer, I interviewed Rep. Issa about a number of issues related to open government at the federal level including the involvement of the United States in OGP. Here’s what he had to say on the topic:

There always will be people who only see the negative of the United Nations or before that, the League of Nations. There will be people who find the World Trade Organization a group that needs to be struck down, because they view the access by the developed nations to assets of the developing nations works to their detriment.

Using those as backdrops, any time lawful representatives of governments come together to see if, in fact, there’s a win-win, I applaud it. The question that I have with this formation is will they come back to their people and stand the test of the traditional question of what is sovereign and what isn’t? And more importantly, see if they have the will of their people broadly through actual new statutes. A lot of what we’re seeing in agreement abroad right now is that individuals from our government go over. They agree to agree, but they never come back and make the circle, of do the American people agree. Do their representatives have the information, and an intervening election, so that when they vote for it, they’re voting for something akin to a treaty?

I think you see it in TPP [The Trans Pacific Partnership], and other things, that sometimes what you do is you say, “Well, we’re bound internationally for that which has not been bought into by the country itself, the people of the country.” I’m broadly for these kinds of talks. I’m decisively against finding out that you’re bound to something that wasn’t approved, not just by legislative representatives but by the American people, because I can give somebody authority to go have a conversation. I can’t give them authority to make a deal on behalf of the American people that the American people don’t know until after the deal has been made.

With respect to the concerns Rep. Issa raised about whether the American people have been consulted, each one of these national action plans for the Open Government Partnership was arrived at with a public consultation with the people of the countries in question. (I was present at the third White House open government partnership consultation as a member of civil society and posted my notes online.) There has been criticism about whether those public consultations are good enough or not, including the one held by our neighbor to the north, up in Canada. (Full disclosure: I was asked to sit on Canada’s open government advisory board and made a series of recommendations for Canada.) Once agreed to, it will be up to civil society and Congress to hold the government of a country accountable for implementing the plans.

There will be inevitable diplomatic challenges for OGP, from South Africa’s proposed secrecy law to Russia’s membership. Given that context, all of the stakeholders in the Open Government Partnership — from the government co-chairs in Brazil and the United Kingdom to the leaders of participating countries to the members of civil society that have been given a seat at the table — will need to keep pressure on other stakeholders if significant progress is going to be made on all of these fronts. If the next President of the United States doesn’t directly support the partnership and its principles on the campaign trail and in actions, it will leave considerable room for other countries to score diplomatic points for joining without delivering upon the promise of its requirements for their people. If OGP is to be judged more than a PR opportunity for politicians and diplomats to make bold framing statements, government and civil society leaders will need to do more to hold countries accountable to the commitments required for participation: they must submit Action Plans after a bonafide public consultation. Moreover, they’ll need to define the metrics by which progress should be judged and be clear with citizens about the timelines for change.

How will “open government” play into Election 2012?

It remains to be seen if open government or OGP comes up as a significant issue in the presidential campaign or in the context of this year’s Congressional election. While the Obama and Romney campaigns are heavily criticizing one another on the issue of “transparency,” from the White House’s mixed record to the former Massachusetts governor’s records in office or work in the Winter Olympics, the future of U.S. involvement in the partnership or its commitments in the plan isn’t making the campaign stump. For that matter, neither is open innovation in the public sector, including the use of prizes and challenges, or lean government.

That’s unfortunate. While there may be a strong rationale for both candidates for the presidency to focus on other issues than the emerging, often nebulous field of “open government,” including fundamental concerns like the economy, foreign policy, energy, education or healthcare, more open policies stand to benefit each of those areas. For instance, at the launch of OGP last September in New York, World Wide Web inventor Tim Berners-Lee argued that more transparency in aid and financial markets attracts more investment in developing countries. The party that would stand to benefit the most from competition on open government would be the American people.

And, while the ambiguity of open government and open data has been driving discussions online for months now, there’s just enough traction behind initiatives around open health data, energy data, and smart disclosure for policy makers, legislators and the electorate to pay a bit more attention to what’s happening in those areas.

Image Credit: DL Photo/CGU at the 2012 Open Government Partnership Conference

Uncertain prospects for the DATA Act in the Senate

The old adage that “you can’t manage what you can’t measure” is often applied to organizations in today’s data-drenched world. Given the enormity of the United States federal government, breaking down the estimated $3.7 trillion dollars in the 2012 budget into its individual allocations, much less drilling down to individual outlays to specific programs and subsequent performance, is no easy task. There are several sources for policy wonks to turn use for applying open data to journalism, but the flagship database of federal government spending at USASpending.gov simply isn’t anywhere near as accurate as it needs to be to source stories. The issues with USASpending.gov have been extensively chronicled by the Sunlight Foundation in its ClearSpending project, which found that nearly $1.3 trillion of federal spending as reported on the open data website was inaccurate.

If the people are to gain more insight into how their taxes are being spent, Congress will need to send President Obama a bill to sign to improve the quality of federal spending data. In the spring of 2012, the U.S. House passed by unanimous voice vote the DATA Act, a signature piece of legislation from Representative Darrell Issa (R-CA). H.R. 2146 requires every United States federal government agency to report its spending data in a standardized way and establish uniform reporting standards for recipients of federal funds.

“The DATA Act will transform how we are able to monitor government spending online,” said Ellen Miller, co-founder and executive director of the Sunlight Foundation, in a prepared statement. “We’ve said time and time again that transparency is not a partisan issue, and we are proud to see there was broad support across the aisle for the bill. The DATA Act will increase transparency for federal spending data and expand when, where and how it is available online,” said Ellen Miller, co-founder and executive director of the Sunlight Foundation. The DATA Act also received support from a broad range of other open government stalwarts, from OMB Watch to Citizens for Responsibility and Ethics in Washington (CREW):

Orgs in Support of DATA Act

Discussing DATA

I spoke with Rep. Issa, who serves as the chairman of the U.S. House Government Reform and Oversight Committee, about the DATA Act and the broader issues around open government data at the Strata Conference in New York City.

Daniel Schuman, the Sunlight Foundation’s legislative counsel, summarized our conversation on open government data over at the Sunlight Foundation’s blog. Video of our discussion is embedded below.

Rep. Issa: …when I work with [Inspector Generals], they would love to have access to predictive [data analytics tools]. Today, they only have forensic. And in many cases, they have like stove pipe forensic. They only know after the fact, a portion of the data, and it frustrates them. We’re going to change that.

The DATA Act is bipartisan, which here in Washington is very unusual. One of the reasons is that people who want to know from the left and the right want to be in the know. We believe that by mandating standard reporting and a process of greater transparency and, of course, the tools created to make this easy and inexpensive for the private sector to participate in will give us an opportunity which will at some time be used by the left or the right or often used by simply people who have a vested interested in advising the private sector accurately on what is, has and will become events in government or for that matter, events in the private sector that are being aggregated through the government.

Your industry is going to be essential because if we give you more accurate, more easily compiled data, unless you turn it into information that’s valuable, we haven’t really accomplished what we want to. The same is true, though, unless you do it, my IGs won’t have private sector solutions that allow them to pick up COTS or near COTS solutions that are affordable and valuable and use them in evaluating government to drive out waste and fraud in government.

What’s next for the DATA Act?

The Senate version of the DATA Act, which is sponsored by Senator Mark Warner (D-VA) remains “pending” in the Homeland Security and Government Affairs Committee after a hearing last week, despite the considerable efforts of a new Data Transparency Coalition to move the bill. The hearing came one week after the coalition held a public DATA Demo Day that featured technology companies demonstrating different uses of standardized federal spending data, including claims that it could have prevented the scandal over excessive conference spending in the General Services Agency.

At the hearing, Senator Warner proposed an amended version of the DATA Act that would drop the independent board modeled on the Recovery Accountability and Transparency Board that oversaw spending from the American Recovery and Reinvestment Act of 2009, as Joseph Marks reported for Next Gov.

The DATA Act, however, received a hearing but not a markup, as Daniel Schuman, the legislative counsel of the Sunlight Foundation, wrote at the transparency advocate’s blog. For those who aren’t well versed in the legislative process, “markups” are when amendments are considered. The DATA Act will have to pass through the HSGAC committee to get to the floor of the Senate.) In his summary of the hearing, Schuman highlighted the opposition of the Office of Management and Budget and U.S. Treasury Department to the Act’s provisions:

Gene Dodaro, the Comptroller General, testified about a newly-issued GAO report on federal spending transparency, which alternatively praised and criticized OMB’s efforts to comply with legislation to improve information availability. During the Q&A, Dodaro explained that it may be helpful for Congress to enact legislation declaring what spending information it wants to have available to the public, as a way of establishing priorities and direction.

OMB Controller Daniel Werfel’s testimony [PDF] focused on OMB’s efforts to improve the accuracy and availability of spending information, largely defending the administration’s record. During the Q&A, Werfel emphasized that new legislation is not necessary to implement spending transparency as the administration already has the necessary authority. While his testimony highlighted the administration’s claims of what it has accomplished, it did not engage the concerns that OMB has dragged its feet over the last 4 years, or that OMB — as an arm of the president — may have mixed incentives about releasing potentially politically damaging information. He did explain that OMB has not released a statement of administration policy on the DATA Act, but that OMB (unsurprisingly) is less than enthusiastic about shifting responsibility over standard-setting and implementation to an independent body.

Treasury Department Assistant Fiscal Secretary Richard Gregg testified [PDF] about ongoing internal efforts at Treasury to improve data quality and projects that will yield results in the future. During the Q&A, Gregg explained that legislation isn’t needed for financial transparency, leadership in the executive branch would be sufficient. This raises the question of whether sufficient leadership is being exercised.

The question of leadership that Schuman raised is a good one, as is one regarding incentives. During July’s International Open Government Data Conference in DC, Kaitlin Bline, the senior developer working on the Sunlight Foundation’s Clearspending project, said that the problems with USASpending.gov government spending data come from oversight, not technology.

Bline was blunter in her post on aGeneral Accounting Office, Congressional committees performing oversight of federal agencies, or special commission, notably the Truman committee during World War II. In the decades since, the work of inspector generals and Congressional staffers has been augmented by fraud detection technology, a critical innovation given the estimated $70 billion dollars in improper payments made by the federal government within the Medicare and Medicaid programs alone. (The fraud detection technology that was developed at PayPal and spun out into Palantir Technologies, in fact, has been deployed to that end.)

The promise of standardizing federal spending data, grant data — or performance data — is that those entrusted with oversight could be empowered with predictive data analytics tools and teams to discover patterns and shift policy to address them.

While the huge budget deficit in the United States is highly unlikely to be closed by cutting fraud and waste alone, making federal spending machine-readable and putting it online clearly holds promise to save taxpayers dollars. First, however, the quality of government spending data must be improved.

Important questions about the DATA Act remain, from the cost of its implementation for cities and states, which would have to report federal grants, to the overall cost of the bill to federal government. The Congressional Budget Office estimated that the DATA Act would cost the government $575 million to implement over 5 years. In response to the CBO, House Oversight staff have estimated that annual savings from standards and centralized spending database that would more than offset that outlay, including:

  • $41 million in funds recovered from questionable recipients
  • $63 million in funds withheld from questionable recipients
  • $5 billion in savings recommended by inspectors general
  • unknown savings resulting from better internal spending control and better oversight by Congressional appropriators.

No formal subsequent action on the DATA Act has been scheduled in the Senate and, with the August recess looming and many eyes turning to cybersecurity legislation, there are uncertain prospects for its passage in this election year’s legislative calendar.

The need for the federal government, watchdogs and the people to be able to accurately track the spending of taxpayer dollars through high quality open government data, however, remains acute.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl