Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

June 29 2011

Citizen science, civic media and radiation data hint at what's to come

SafecastNatural disasters and wars bring people together in unanticipated ways, as they use the tools and technologies easily at hand to help. From crisis response to situational awareness, free or low cost online tools are empowering citizens to do more than donate money or blood: now they can donate, time, expertise or, increasingly, act as sensors. In the United States, we saw a leading edge of this phenomenon in the Gulf of Mexico, where open source oil spill reporting provided a prototype for data collection via smartphone. In Japan, an analogous effort has grown and matured in the wake of the nuclear disaster that resulted from a massive earthquake and subsequent tsunami this spring.

The story of the RDTN project, which has grown into Safecast, a crowdsourced radiation detection network, isn't new, exactly, but it's important.

Radiation monitoring and grassroots mapping in Japan has been going on since April, as Emily Gertz reported at I recently heard more about the Safecast project from Joi Ito at this year's Civic Media conference at the MIT Media Lab, where Ito described his involvement. Ethan Zuckerman blogged Ito's presentation, capturing his thoughts on how the Internet helped cover the Japanese earthquake (Twitter "beat the pants" off the mainstream media on the first day) and the Safecast project's evolution from a Skype chat.

According to Gertz' reporting, Safecast now includes data from a variety of sources, including feeds from the U.S. Environmental Protection Agency, Greenpeace, a volunteer crowdsourcing network in Russia, and the Japanese Ministry of Education, Culture, Sports, Science and Technology. Radiation data that's put into Safecast is made available for others to use via Pachube, an open-source platform for monitoring sensor data.

Ito said that a lot of radiation data that the Japanese government had indicated would be opened up has not been released, prompting the insight that crises, natural or otherwise, are an excellent opportunity to examine how effective an open government data implementation has been. Initially, the RDTN project entered an environment where there was nearly no radiation data available to the public.

"They were releasing data, it was just not very specific," said Sean Bonner, via Skype Interview. Bonner has served as the communications lead for Safecast since the project began. The Japanese government "would release data for some areas and not for others — or rather they didn't have it," he said. "I don't think they had data they weren't releasing. Our point is that the sensors to detect the data were not in place at all. So we decided to help with that."

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 20% on registration with the code STN11RAD

A KickStarter campaign in April raised nearly $37,000 to purchase Geiger counters to gather radiation data. Normally, that might be sufficient to obtain dozens of devices, given costs that range from $100 to nearly $1,000 for a professional-grade unit. The challenge is that if Geiger counters weren't easy to get before the Japanese nuclear meltdown, they became nearly impossible to obtain afterwards.

The Safecast project has also hacked together iGeigie, an iPhone-connected Geiger counter that can detect beta and gamma radiation. "The iGeigie is just a concept product, it's not a focus or a main solution," cautioned Bonner. "So a lot of what we've been doing it trying to help cover more ground with single sensors."

Even if they were in broader circulation, Geiger counters are unlikely to detect radiation in food or water. That's where open source hardware and hackerspaces become more relevant, specifically the Arduino boards that Radar and Make readers know well.

"We have Arduinos in the static devices that we are building and connecting to the web," said Bonner. "We're putting those around and they report data back to us." In other words, the Internet of Things is growing.

The sensors Safecast is deploying will capture alpha, beta and gamma radiation. "It's very important to track all three," said Bonner. "The very sensitive devices we are using are commercially produced. [They are] Inspector Alerts, made by International Medcom. Those use the industry standard 2-inch pancake sensor, which we are using in our other devices as well. We are using the same sensors everywhere. "

Citizen science and open data

Open source software and citizens acting as sensors have steadily been integrated into journalism over the past few years, most dramatically in the videos and pictures uploaded after the 2009 Iran election and during this year's Arab Spring. Citizen science looks like the new frontier. "I think the real value of citizen media will be collecting data," said Rich Jones, founder of OpenWatch, a counter-surveillance project that aims to "police the police." Apps like Open Watch can make "analyzing data a revolutionary act," said Justin Jacoby Smith. The development of Oil Reporter, grassroots mapping, Safecast, social networks, powerful connected smartphones and massive online processing power have put us into new territory. In the context of environmental or man-made disasters, collecting or sharing data can also be a civic act.

Crowdsourcing radiation data on Japan does raise legitimate questions about data quality and reporting, as Safecast's own project leads acknowledge.

"We make it very clear on the site that yes, there could most definitely be inaccuracies in crowd-sourced data," Safecast's Marcelino Alvarez told Public Radio International. "And yes, there could be contamination of a particular Geiger counter so the readings could be off," Alvarez said. "But our hope is that with more centers and more data being reported that those points that are outliers can be eliminated, and that trends can be discerned from the data."

The thinking here is that while some data may be inaccurate or some sensors misconfigured, over time the aggregate will skew toward accuracy. "More data is always better than less data," said Bonner. "Data from several sources is more reliable than from one source, by default. Without commenting on the reliability of any specific source, all the other sources help improve the overall data. Open data helps with that."

Safecast is combining open data collected by citizen science with academic, NGO and open government data, where available, and then making it widely available. It's similar to other projects, where public data and experimental data are percolating.

Citizen science can create information orders of magnitude better than Google Maps, said Brian Boyer, news application developer at the Chicago Tribune, referencing the grassroots mapping work of Jeffrey Warren and others. "It's also fun," Boyer said. "You can get lots of people involved who wouldn't otherwise be involved doing a mapping project."

As news of these experiments spreads, the code and policies used to build them will also move with them. The spread of open source software is now being accompanied by open source hardware and maker culture. That will likely have unexpected effects.

When you can't meet demand for a device like a Geiger counter, people will start building their own, said Ito at the MIT Civic Media conference. He's seeing open hardware design spread globally. While there's an embargo on the export of many technologies, "we argue — and win — that open source software is free speech," said Ito. "Open source hardware is the same." If open source software now plays a fundamental role in new media, as evidenced by the 2011 winners of the Knight News Challenge, open source hardware may be supporting democracy in journalism too, says Ito.

Given Ito's success in anticipating (and funding) other technological changes, that's one prediction to watch.


April 14 2011

Personal data is the future, but does anybody care?

Like most people, I tend to surround myself with like-minded folks. Most of my dinner party conversations turn into rousing debates on the future of web standards, or which company will unlock the true power of personal data on the web, or how can we mark our bits with emotional cues to make our web experiences more human. That sort of thing.

But every now and then, I reconnect with old friends and even meet new people who don't find a conversation on data rousing at all. They have other things on their minds and they haven't thought about cookies or the amount of data Facebook is collecting on us. The mere utterance of the phrase "silos of data" kills a perfectly lovely conversation.

The problem is that understanding our personal data is important for everyone — not just geeks. People spend an incredible amount of time on Facebook, Google, Amazon, Twitter and other websites, creating content and telling the world how we feel, what we consume, how we think, and what we care about. And none of this belongs to us. I usually rile up a bit of a reaction when I mention that all of that time and energy spent is sold to advertisers, but the reaction boils down to privacy issues rather understanding the value of that information.

As I've been building my own personal data collection startup, I've thought a great deal about how I could communicate the value of knowing and owning your own data to non-geeks. The answer came to me after making a list of all of the personal data collection applications I have signed up for. I looked at those I use religiously versus those I've abandoned. Those I use religiously include: RunKeeper, TripIt, Foursquare, Gowalla, Fitbit, Mint, Hashable, OKCupid, and Foodspotting. Those that I love the idea of, but have since left behind, include: Hunch, Blippy, 23andMe, GoodReads, Plancast and Dopplr.

I know that others' lists will be different, but the point is that this process allowed me to step back and really think about what sort of real-time value I was getting out of gathering my own data. I was able to boil the results down to three categories that, I believe, could be used to incentivize personal data collection for just about anybody. These categories are:

  1. Utility
  2. Serendipity
  3. Self-expression

In order to incentivize the continued use of any personal data collection application, you either have to really excel in one of these areas or cover all three. Let me explain.

(Disclosure: O'Reilly AlphaTech Ventures is an investor in Foursquare, RunKeeper, and TripIt.)

Where 2.0: 2011, being held April 19-21 in Santa Clara, Calif., will explore the intersection of location technologies and trends in software development, business strategies, and marketing.

Save 25% on registration with the code WHR11RAD

1. Utility

Mint.comProbably the best example of utility from collecting personal data is By merely hooking up your online bank accounts, you get a snapshot of where you're spending your money, how much you have left, and you're given suggestions on where you can improve your financial situation. was so handy for so many people that they had to do very little marketing. Their users became rabid fans and told stories to everyone who would listen about how Mint saved them all kinds of money, exposed fees that they didn't know they were paying, and helped them get savvier about their finances.

Utility in itself isn't sexy, but if you make it incredibly beneficial and impossible to live without, people will pay for it. Utility includes things like:

  • Tracking — How much you spend, where you've been, how much you've consumed, when you did that thing you used to do last, etc.
  • Augmentation — Anything that can extrapolate from and add to the raw data is helpful.
  • Organization — Ways to sort and make sense of the raw data.
  • Visualization — A way to present the data so it is easy to interpret.

Utility is where TripIt proved more useful to me than Dopplr. I thought Dopplr's lovely design and serendipity-driven features were going to win me over, but at the end of the day, it was the usefulness of TripIt's easy itineraries, flight tracking, and augmentation through things like weather and maps, that led me to use it religiously while I almost completely forgot about Dopplr.

2. Serendipity

OKCupidOKCupid does the same basic thing that Hunch does: it asks the user to answer endless questions about their personal tastes and preferences. However, OKCupid has something over Hunch to incentivize users to actually spend the time to answer those questions: serendipity. And it's not just any serendipity, it's serendipity at its finest: the promise of finding love.

Answering questions about myself is kind of a fun notion ... once. People take personality quizzes all of the time online, but when we get the results, what is the first thing we do? We share it with friends. And once our friends have done the test and we all compare notes, that's it. We don't really go back. Where it gets interesting is if these results lead us to discover new friends, potential mates, cool stuff and ideas that could change our lives. And it isn't enough to have this happen once. It has to uncover serendipitous moments over and over again.

Serendipity is also the core element that drives my usage of geo-location applications. A couple of months ago, I was in New York and checked into a pizza place near Union Square. I looked to see who had checked in recently and Mark Suster's smiling face appeared. I had never met Mark, but I'd always wanted to and Foursquare allowed me to connect with him serendipitously in a place I never expected to. It's moments like these that drive me to continue to check in even though it takes time and effort to do so.

Serendipity is, ultimately, how you use the data to connect people to people, people to things that may interest them, and people to opportunities.

3. Self-expression

RunkeeperThere is definitely a utility in using an application like RunKeeper, but that's not primarily why I use it. I'm pretty proud of my commitment to training and my progress with running. RunKeeper gives me that tool I need to strongly signal to everyone who follows me that I'm a runner. The more I log my runs, the more people express how impressed they are, and so the more I log my runs. It's cyclical.

Other applications signal personal tastes as well. Foodspotting signals that you are cultured (if you take shots of a variety of ethnic foods), healthy (if you post organic, vegetarian or the like), indulgent (posting desserts, expensive meals, decadent burgers, etc.) or the like. Hashable signals you are a mover and a shaker without being accused of namedropping. signals whether you have hipster or hip-hop leanings. I have to admit that I've been known turn off the scrobbler when in a pop music mood. Why make the effort to stop the scrobbler and start the scrobbler again? Because I'm aware of the signals I'm sending.

The self-expression or taste signaling dimension of our personal data collection has the strongest potential for creating the ultimate personalized web experience. It's yet to be completely explored. We are practically screaming who we are and what we like as we post our activity on social applications, yet most recommendation engines and data mining engines continue to put us in traditional demographic and psychographic boxes. The potential of all this will be unlocked when emotional/taste data is mapped to products, check-ins, and our activity across social applications.

I'm looking forward to the day that personal data collection is part of the popular vernacular. Until then, it is up to us — the geeks and developers of theses applications — to help people collect these moments so they receive real-time value.


Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
No Soup for you

Don't be the product, buy the product!

YES, I want to SOUP ●UP for ...