Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

February 10 2012

Preview of HIMSS 2012

I am very happy to be attending the Healthcare Information and Management Systems Society (HIMSS) conference this year. We are at a pivotal moment in the history of healthcare in this country and health IT is playing a very prominent role. This will be one of the most important healthcare conferences of the year. If you can't make it to Las Vegas in person, there are opportunities to attend virtually. Just go to himssvirtual.org for more information.

I will be moderating panel presentations at the HIMSS Social Media Center on Tuesday and Wednesday. This year I expect social media to play a much larger presence in the conference, and the new location for the pavilion will put it front and center. Since the keynote this year is from one of the founders of Twitter, Biz Stone, I'm sure there will be a social media flavor throughout the event.

I will also be participating in the brand new eCollaboration Forum at HIMSS on Thursday. The Collaborative Health Consortium has partnered with HIMSS to sponsor a new, exclusive event focused on the shift to collaborative care platforms to take place at the conference. The event will focus on collaborative platforms as foundations for transformation to accountable care. Attendees will be able to learn what a collaborative healthcare platform is and why the healthcare industry needs it, discover paths to take to effectively implement collaborative technologies, and get further resources to help evaluate the solutions available in the shift toward an accountable care health model.

I am honored to be moderating a panel with David C. Kibbe, MD MBA, senior advisor at the American Academy of Family Physicians; Jonathan Hare, chairman of Resilient Network Systems; and Scott Rea, vice president GOV/EDU Relations and senior PKI Architect at DigiCert.

Our session, "Developing Trust in the Health Internet as a Platform," will focus on the tools, technologies and rules we must decide upon to establish trust in the Internet as the platform for healthcare. Effective health information exchange of any resource requires deep trust, following from the right architecture and the right rules. We will discuss efforts like DirectTrust.org and the EHR/HIE Interoperability Workgroup as conveners that are creating a community to move us forward.

My fellow Radar blogger Andy Oram will also be on hand to provide context and his own unique perspective (as well as keep me focused on what matters).

Related:

December 19 2011

Big crime meets big data

Marc Goodman (@futurecrimes) is a former Los Angeles police officer who started that department's first Internet crime unit in the mid-1990s. After two decades spent working with Interpol, the United Nations, and NATO, Goodman founded the Future Crimes Institute to track how criminals use technology.

Malicious types of software, like viruses, worms, and trojans, are the main tools used to harvest personal data. Cyber criminals also use social engineering techniques, such as phishing emails populated with data gleaned from social networks, to trick people into providing further details. In the interview below, Goodman outlines some of the other ways organized criminals and terrorists are harnessing data for nefarious ends.

What motivates data criminals?

Marc GoodmanMarc Goodman: Anything that would motivate someone to join a startup would motivate a criminal. They want money, shares in the business, a challenge. They don't want a 9-to-5 environment. They also want the respect of their peers. They have an us-against-them attitude; they're highly innovative and adaptive, and they never take the head-on approach. They always find clever and imaginative ways to go about something that a good person would never have considered.

What type of personal data is most valuable to criminals?

Marc Goodman: The best value is a bank account takeover. A standard credit card might cost a criminal only $10, but for $700 they could buy details of a bank account with $50,000 in it, money that could be stolen in just one transaction.

European credit cards tend to cost more than American credit cards since Europeans are much better at guarding their data. There's also a universal identifier for Americans — the social security number — but the same thing doesn't exist from a pan-European perspective.

How is data crime more scalable than traditional crime?

Marc Goodman: Data crime can be scripted and automated. If you were to take a gun or a knife and stand on a street corner, there are only so many people you can rob. You have to do the crime, run away from the scene, worry about the police, etc. You can't walk into Wembley Stadium with a gun and say, "Everybody, put your hands up," but you can do the equivalent from a cyber-crime perspective.

One of the reasons why cyber crime thrives is that it's totally international whereas law enforcement is totally national. Now, the person attacking you can be sitting in New York or Tokyo or Botswana. The ability to conduct business without getting on a plane is an awesome advantage for international organized crime.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

How has cyber crime evolved?

Marc Goodman: In the 1970s, you had to be a clever hacker and create your own scripts. Now all of that stuff can be bought off the shelf. You can buy a package of crimeware and put in the email addresses or the domain that you want to attack via a nice user interface. It's really plug-and-play criminality.

You claim that the 2008 Mumbai attackers used real-time data gathering from social networks and other media. How do terrorists use data?

Marc Goodman: Since the Internet arrived, terrorists have been advertising, doing PR, recruiting, and fundraising, all online. But this was the first time that we had seen terrorists use technology to the full extent that this group did during the incident. They had mobile phones and satellite phones. The terrorist war room they set up to monitor the media and feed back information in real time to the attackers was a really significant innovation.

They re-engineered the attack mid-incident to kill more people. They were constantly looking for new hostages. Organizations like the BBC and CNN were tweeting to ask people on the ground in Mumbai to contact a producer. People trapped in hotels called the TV stations. All of that information was being tracked by the terrorist war room. There was an Indian minister who was doing a live interview on the Indian Broadcast Network (IBN) while hiding in the kitchen of the ballroom of the Taj Mahal hotel. The war room picked this up and directed the attackers to that part of the hotel where they could find the minister.

What can be done to combat cyber crime?

Marc Goodman: The terrorism problem is very different from the cyber crime problem. Most terrorism tends to have a basis in the real world whereas cyber crime tends to be purely online. Governments are pretty good at tracking the terrorists in their own countries, and there is decent international cooperation on terrorism.

What is making things more difficult for governments is that, in the old days, if you tapped somebody's home phone, you had a good picture of what was going on. Now you don't know where to look. Are they communicating on Facebook, on Twitter, or having a meeting in World of Warcraft?

Law enforcement needs to develop better systems to deal with the madness of social media in terrorist attacks. The public is getting involved in ways that are, frankly, unhealthy. There was a hostage situation in the U.S. a couple of months ago where a man took a hostage and was sexually assaulting her. He was trapped in a hotel room with guns and was posting live on Facebook and Twitter. Then the public started to interact with the hostage-taker, tweeting things like, "You wouldn't kill her. You are not brave enough to do it." In the past, police could close off several blocks, put up yellow crime scene tape, close the airspace over the scene, and bring in a trained negotiator. How does law enforcement intervene when there can be a completely disintermediated conversation between the criminal or terrorist and the general public?


Marc Goodman discussed the business of illegal data at Strata New York 2011. His full presentation is available in the following video:

This interview was edited and condensed.

Related:

December 12 2011

Can the People's House become a social platform for the people?

Congressional hackathon
InSourceCode developers work on "Madison" with volunteers.

There wasn't a great deal of hacking, at least in the traditional sense, at the "first congressional hackathon." Given the general shiver that the word still evokes in many a Washingtonian in 2011, that might be for the best. The attendees gathered together in the halls of the United States House of Representatives didn't create a more interactive visualization of how laws are made or a mobile health app. As open government advocate Carl Malamud observed, the "hack" felt like something even rarer in the "Age of the App for That:"

In a time when partisanship and legislative gridlock have defined Congress for many citizens, seeing the leadership of the United States House of Representatives agree on the importance of using the power of data and social networking to open government was an early Christmas present.

"Increased access, increased connection with our constituents, transparency, openness is not a partisan issue," said House Majority Leader Eric Cantor.

"The Republican leader and I may debate vigorously on many issues, but one area where we strongly agree is on making Congress more transparent and accessible," said House Democratic Whip Steny Hoyer in his remarks. "First, Congress took steps to open up the Capitol building so citizens can meet with their representatives and see the home of their legislature. In the same way, Congress is now taking steps to update how it connects with the American people online."

An open House

While the event was branded as a "Congressional Facebook Developer Hackathon," what emerged more closely resembled a loosely organized conference or camp.

Facebook executives and developers shared the stage with members of Congress to give keynotes to the 200 or so attendees before everyone broke into discussion groups to talk about constituent communications, press relations and legislative data. The event might be more aptly described as a "wonk-a-thon," as Sunlight Foundation's Daniel Schuman put it last week.

This "hackathon" was organized to have some of the feel of an unconference, in the view of Matt Lira, digital director for the House Majority Leader. Lira sat down for a follow-up interview last Thursday.

"There's a real model to CityCamp," he said. "We had 'curators' for the breakout. Next time, depending on how we structure it, we might break out events that are designed specifically for programming, with others clustered around topics. We want to keep it experimental."

Why? "When Aneesh Chopra and I did that session at SXSW, that personally for me was what tripped my thinking here," said Lira. "We came down from the stage and formed a circle. I was thinking the whole time that it would have been a waste of intellectual talent to have Tim O'Reilly and Clay Shirky in the audience instead of engaging in the conversation. I was thinking I never want to do a panel again. I want it to be like this."

Part of the challenge, so to speak, of Congress hosting a hackathon in the traditional sense, with judging and prizes, lies in procurement rules, said Lira."There are legal issues around challenges or prizes for Congress," he explained. "They're allowed in the executive branch, under DARPA, and now every agency under the COMPETES Act. We can't choose winners or losers, or give out prizes under procurement rules."

Whatever you call it, at the end of the event, discussion leaders from the groups came back and presented on the ideas and concepts that had been hashed out. You can watch a short video that EngageDC produced for the House Majority Leader's office below:

What came out of this unprecedented event, in other words, won't necessarily be measured in lines of code. It's that Congress got geekier. It's that the House is opening its doors to transparency through technology.

Given the focus on Facebook, it's not surprising that social media took center stage in many of the discussions. The idea for it came from a trip to Silicon Valley, where Representative Cantor said he met with Facebook founder Mark Zuckerberg and COO Sheryl Sandberg, and discussed how to make the House more social. After that conversation, Lira and Steve Dwyer, director of online communications and technology for the House Democratic Whip, organized the event.

For a sense of the ideas shared by the working groups, read the story of the first congressional "hackathon" on Storify.

"For government, I don't think we could have done anything more purposeful than this as a first meeting," said Lira in our interview. "Next, we'll focus on building this group of people, strengthening the trust, which will prove instrumental when we get into the pure coding space. I have 100% confidence that we could do a programming-only event now and would have attendance."

A Likeocracy in alpha

As the Sunlight Foundation's John Wonderlich observed earlier this year, access to legislative data brings citizens closer to their representatives.

"When developers and programmers have better access to the data of Congress, they can better build the databases and tools that let the rest of us connect with the legislature," he wrote.

If more open legislative data goes online, when we talk about what's trending in Congress, those conversations will be based upon insight into how the nation is reacting to them on social networks, including Facebook, Twitter, and Google+.

Facebook developers Roddy Lindsay, Tyler Brock, Eric Chaves, Porter Bayne, and Blaise DiPersia coded up a simple proof of concept of what making legislative data might look like. "LikeOcracy" pulls legislation from a House XML feed and makes it more social. The first version added Facebook's ubiquitous "Like" buttons to bill elements. A second version of the app adds more opportunities for reaction by integrating ReadrBoard, which enables users to rate sections or individual lines as "Unnecessary, Problematic, Great Idea or Confusing." You can try it out on three sample bills, including the Stop Online Piracy Act.

Would "social legislation" in a Facebook app catch on? The growth of civic startups like PopVox, OpenCongress and Votizen suggests that the idea has legs. [Disclosure: Tim O'Reilly was an early angel investor in PopVox.]

Likeocracy doesn't tap into Facebook's Open Graph, but it does hint at what integration might look like in the future. Justin Osofsky, Facebook's director of platform partnerships, described how the interests of constituents could be integrated with congressional data under Facebook's new Timeline. Citizens might potentially be able to simply "subscribe" to a bill, much like they can now for any web page, if Facebook's "Subscribe" plug-in was applied to the legislative process.

Opening bill markup online

The other app presented at the hackathon came not from the attendees but from the efforts of InSourceCode, a software development firm that's also coded for Congressman Mike Pence and the Republican National Committee.

Rep. Darrell Issa, chairman of the House Committee on Oversight and Government Reform, introduced the beta version of MADISON on Wednesday, a new online tool to crowdsource legislative markup. The vision is that MADISON will work as a real-time markup engine to let the public comment on bills as they move through the legislative process. "The assumption is that legislation should be open in Congress," said Issa. "It should be posted, interoperable and commented upon."

As Nick Judd reported at techPresident, the first use of MADISON is to host Issa and Sen. Ron Wyden's "OPEN bill," which debuted on the app. Last week, the congressmen released the Online Protection and Enforcement of Digital Trade Act (OPEN) at Keepthewebopen.com. The OPEN legislation removes one of the most controversial aspects of SOPA, using the domain name system for enforcement, and instead places authority with the International Trade Commission to address enforcement of IP rights on websites that are primarily infringing upon copyright.

Issa said that his team had looked at the use of wikis by Rep. John Culberson, who put the healthcare reform bill online in a wiki. "There are some problems with editors who are not transparent to all of us," said Issa. "That's one of the challenges. We want to make sure that if you're an editor, you're a known editor."

MADISON includes two levels of authentication: email for simple commenting and a more thorough vetting process for organizations or advocacy groups that wish to comment. "Like most things that are a 1.0 or beta, our assumption is that we'll learn from this," said Issa. "Some members may choose to have an active dialog. Others may choose to have it be part of pre-markup record."

Issa fielded a number of questions on Wednesday, including one from web developer Brett Stubbs: "Will there be open access or an API? What we really want is just data." Issa indicated that future versions might include that.

Jayson Manship, the "chief nerd" at InSourceCode, said that MADISON was built in four days. According to Manship, the idea came from conversations with Issa and Seamus Kraft, director of digital strategy for the House Committee on Oversight and Government Reform. MADISON is built with PHP and MySQL, and hosted in RackSpace's cloud so it can scale with demand, said Manship.

"It's important to be entrepreneurial," said Lira in our interview. "There are partners throughout institutions that would be willing to do projects of different sizes and scopes. MADISON is something that Issa and Seamus wanted to do. They took it upon themselves to get the ball rolling. That's the attitude we need."

"We're working to hold the executive accountable to taxpayers," said Kraft last week. "Opening up what we do here in these two halls of Congress is equally important. MADISON is our first shot at it. We're going to need a lot of help to make it better."

Kraft invited the remaining developers present to come to the Rayburn Office Building, where Manship and his team had brought in half a dozen machines, to help get MADISON ready for launch. While I was there, there were conversations about decisions, plug-ins and ideas about improving the interface or functionality, representing a bona fide collaboration to make the app better.

There's a larger philosophical issue relating to open government that Nick Judd touched upon over at techPresident in a follow-up post on MADISON:

The terms for the site warn the user that anything they write on it will become public domain — but the code itself is proprietary. Meanwhile, OpenCongress' David Moore points out that the code that powers his organization's website, which also allows users to comment on individual provisions of bill text, is open source and has been available for some time. In theory, this means the Oversight staff could have started from that code and built on it instead of beginning from scratch. The code being proprietary means that while people like Moore might be able to make suggestions, they can't just download it, make their own changes and submit them for community review — which they'd happily do at little or no cost for a project released under an open-source license.

As Moore put it, "Get that code on GitHub, we'll do OpenID, fix the design."

When asked about whether the team had considered making MADISON code open source, Manship said that "he didn't know, although they weren't opposed to it."

While Moore welcomed MADISON, he also observed that Open Congress has had open-source code for bill text commenting for years.

The decision by Issa's office to fund the creation of an app that was already available as open-source software is one that's worth noting, so I asked Kraft why they didn't fork OpenCongress' code, as Judd suggests. "While there was no specific budget expense for MADISON, it was developed by the Oversight Committee," said Kraft.

"While we like and support OpenCongress' code, it didn't fit the needs for MADISON," Kraft wrote in an emailed statement.

What's next is, so to speak, an "OPEN" question, both in terms of the proposed SOPA alternative and the planned markup of SOPA itself on December 15. The designers of OPEN are actively looking for feedback from the civic software development community, both in terms of what functionality exists now and what could be built in future iterations.

THOMAS.gov as a platform

What Moore and long-time open-government advocates like Carl Malamud want to see from Congress is more structural change:

They're not alone. Dan Schuman listed many other ways the House has yet to catch up with 21st century technology:

We have yet to see bulk access to THOMAS or public access to CRS reports, important legislative and ethics documents are still unavailable in digital format, many committee hearings still are not online, and so on.

As Schuman highlighted, the Sunlight Foundation has been focused on opening up Congress through technology since the organization was founded. To whit: "There have been several previous collaborative efforts by members of the transparency community to outline how the House of Representatives can be more open and accountable, of which an enduring touchstone is the Open House Project Report, issued in May 2007," wrote Schuman.

The notion of making THOMAS.gov into a platform received high-level endorsement from a congressional leader when House Minority Whip Steny Hoyer remarked on how technology is affecting Congress, his caucus and open government in the executive branch:

For Congress, there is still a lot of work to be done, and we have a duty to make the legislative process as open and accessible as possible. One thing we could do is make THOMAS.gov — where people go to research legislation from current and previous Congresses — easier to use, and accessible by social media. Imagine if a bill in Congress could tweet its own status.

The data available on THOMAS.gov should be expanded and made easily accessible by third-party systems. Once this happens, developers, like many of you here today, could use legislative data in innovative ways. This will usher in new public-private partnerships that will empower new entrepreneurs who will, in turn, yield benefits to the public sector.

One successful example is how cities have made public transit data accessible so developers can use it in apps and websites. The end result has been commuters saving time every day and seeing more punctual trains and buses as a result of the transparency. Legislative data is far more complex, but the same principles apply. If we make the information available, I am confident that smart people like you will use it in inventive ways.

Hoyer's specific citation of the growth of open data in cities and an ecosystem of civic applications based upon it is further confirmation that the Gov 2.0 meme is moving into the mainstream.

Making THOMAS.gov into a platform for bulk data would change what's possible for all civic developers. What I really want is "data on everything," Stubbs told me last week. "THOMAS is just a visual viewer of the internal stuff. If we could have all of this, we could do something with it. What I would like is a data broker. I'd like a RESTful API with all of the data that I could just query. That's what the government could learn from Facebook. From my point of view, I just want to pull information and compile it."

If Hoyer and the House leadership would like to see THOMAS.gov act as a platform, several attendees at the hackathon suggested to me that Congress could take a specific action: collaborate with the Senate and send the Library of Congress a letter instructing it to provide bulk legislative data access to THOMAS.gov in structured formats so that developers, designers and citizens around the nation can co-create a better civic experience for everyone.

"The House administration is working on standards called for by the rule and the letter sent earlier this year," said Lira. "We think they will be satisfactory to people. The institutions of the House have been following through since the day they were issued. The first step was issuing an XML feed daily. Next year, there will be a steady series of incremental process improvements. When the House Administrative Committee issues standards, the House Clerk will work on them. "

Despite the abysmal public perception of Congress, genuine institutional changes in the House of Representatives driven by the GOP embracing innovation and transparency are incrementally happening. As Tim O'Reilly observed earlier this year, the current leadership of the House on transparency is doing a better job than their predecessors.

In April, Speaker Boehner and Majority Leader Cantor sent a letter to the House Clerk regarding legislative data release. Then, in September, a live XML feed for the House floor went online. Yes, there's a long way to go on open legislative data quality in Congress. That said, there's support for open-government data from both the White House and the House.

"My personal view is that what's important right now is that the House create the right precedents," said Lira. "If we create or adopt a data standard, it's important that it be the right standard."

Even if open government is in beta, there needs to be more tolerance for experiments and risks, said Lira. "I made a mistake in attacking We the People as insufficient. I still believe it is, but it's important to realize that the precedent is as important as the product in government. In technology in general, you'll never reach an end. We The People is a really good precedent, and I look forward to seeing what they do. They've shown a real commitment, and it's steadily improving."

A social Congress

While Sean Parker may predict that social media will determine the outcome of the 2012 election, governance is another story entirely. Meaningful use of social media by Congress remains challenged by a number of factors, not least an online identity ecosystem that has not provided Congress with ideal means to identify constituents online. The reality remains that when it comes to which channels influence Congress, in-person visits and individual emails or phone calls are far more influential with congressional staffers.

As with any set of tools, success shouldn't be measured solely by media reports or press releases but by the outcomes from their use. The hard work of bipartisan compromise between the White House and Congress, to the extent it occurs, might seem unlikely to be publicly visible in 140 characters or less.

"People think it's always an argument in Washington," said Lira in our interview. "Social media can change that. We're seeing a decentralization of audiences that is built around their interests rather than the interests of editors. Imagine when you start streaming every hearing and making information more digestible. All of a sudden, you get these niche audiences. They're not enough to sustain a network, but you'll get enough of an audience to sustain the topic. I believe we will have a more engaged citizenry as a result."

Lira is optimistic. "Technology enables our republic to function better. In ancient Greece, you could only sustain a democracy in the size of city. Transportation technology limited that scope. In the U.S., new technologies enabled global democracy. As we entered the age of mass communication, we lost mass participation. Now with the Internet, we can have people more engaged again."

There may be a 30-year cycle at play here. Lira suggested looking back to radio in the 1920s, television in the 1950s, and cable in the 1980s. "It hasn't changed much since; we're essentially using the same rulebook since the '80s. The changes made in those periods of modernization were unique."

Thirty years on from the introduction of cable news, will the Internet help reinvigorate the founders' vision of a nation of, by and with the people? "I do think that this is a transformational moment," said Lira. "It will be for the next couple of years. When you talk to people — both Republicans and Democrats — you sense we're on the cusp of some kind of change, where it's not just communicating about projects but making projects better. Hearings, legislative government and executive government will all be much more participatory a decade from now. "

In that sweep of history, the "People's House" may prove to be a fulcrum of change. "If any place in government is going to do it, it's the House" said Lira. "It's our job to be close to the public in a way that no other part of government is. In the Federalist Papers, that's the role of the House. We have an obligation to lead the way in terms of incorporating technology into real processes. We're not replacing our system of representative government. We're augmenting it with what's now possible, like when the telegraph let people know what the votes were faster."

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

November 18 2011

The future of social media at the National Archives

In November 2011, conversations about connection technologies have shifted from whether governments should use social media to how governments should use social media. Facebook, Flickr, Twitter and YouTube are part of the default template for the websites of newly elected officials.

As the year comes to an end, the risks and rewards of Web 2.0 are better known for both citizens and government alike. People from every walk of life naturally have questions about what the explosion of social media will mean for the future of society, including difficult questions about what this new landscape will mean for privacy, security, freedom of expression and online identity. Predictions about what the future of social media will mean for an increasingly networked society range from dystopian autocracies with pervasive surveillance to stronger, data-driven digital democracies. Or both.

It's in that context that the National Archives recently convened a conversation about "What's Next?" at the McGowan Theater in its headquarters in Washington, D.C.

David Ferriero (@dferriero), the Archivist of the United States of America (AOTUS), introduced the forum on social media:

Access to records in this century means digital access. For many people, if it is not online, it doesn't exist. The use of social media to increase access is the new norm. NARA [National Archives and Records Administration] has been going after innovative tools and projects that increase digital access to our records, including projects that invite public participation. We are developing a Citizen Archivist Dashboard that will encourage the public to pitch in via social media tools on a number of our projects.

Ferriero introduced the idea of a "citizen archivist" after he joined the National Archives in 2009. Now, the National Archives is moving forward with enlisting the help of the people in identifying, digitizing and archiving the nation's history with the use of the Internet.

Prior to the "What's Next?" forum, National Archives staff hosted a social media fair where they showed how they were using Twitter, Facebook, Flickr and location-aware social networks to share the nation's history and connect to citizens online. To learn more about how this "Citizen Archivist Dashboard" will work, you can watch the video embedded below and download a PDF of the presentation that Pamela Wright, chief digital access strategist at NARA, gave at the forum. Joseph Marks attended the forum and wrote about the dashboard at NextGov afterward.

If you watch the video, you'll see Ferriero and Wright discuss how the National Archives is thinking about the work of preservation in the age of social media. As more records become digital, the institution faces huge challenges in how it approaches retaining the historical records.

At the forum, I gave a short presentation in which I talked about the past, present and future for social media. The talk framed a subsequent panel discussion about social media where Wright and I joined David Weinberger, senior researcher at the Harvard Berkman Center for Internet and Society, and Sarah Bernard, deputy director for the Office of Digital Strategy at the White House.

Weinberger shared some thoughts about "what's new in social media" prior to the forum on his blog:

1. The Internet began as an open "address space" that enabled networks to be created within it. So, we got the Web, which networked pages. We got social networks, which networked people. We are well on our way to networking data, through the Semantic Web and Linked Open Data. We are getting an Internet of Things. The DPLA [Digital Public Library of America] will, I hope, help create a network of cultural objects.

2. The Internet and the web have always been social, but the rise of networks particularly tuned to social needs is of vast importance because the social determines all the rest. Indeed, the Internet is a medium only because we are in fact that through which messages pass. We pass them along because they matter to us, and we stake a bit of ourselves on them. We are the medium.

3. Of all of the major and transformative networks that have emerged, only the social networks are closed and owned. I don't know how or if we will get open social networks, but it is a danger that as of now we do not have them.

Over the course of the evening, I moderated an engaging conversation about what the rapidly expanding universe of collaborative technologies means for the society, government and media. There was a lively question and answer period, with the people "formerly known as the audience" joining us in the theater and through an active online backchannel via Twitter. It was an honor to join this discussion.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

Related:

November 17 2011

Strata Week: Why ThinkUp matters

Here are a few of the data stories that caught my attention this week.

ThinkUp hits 1.0

ThinkUpThinkUp, a tool out of Expert Labs, enables users to archive, search and export their Twitter, Facebook and Google+ history — both posts and post replies. It also allows users to see their network activity, including new followers, and to map that information. Originally created by Gina Trapani, ThinkUp is free and open source, and will run on a user's own web server.

That's crucial, says Expert Labs' founder Anil Dash, who describes ThinkUp's launch as "software that matters." He writes that "ThinkUp's launch matters to me because of what it represents: The web we were promised we would have. The web that I fell in love with, and that has given me so much. A web that we can hack, and tweak, and own." Imagine everything you've ever written on Twitter, every status update on Facebook, every message on Google+ and every response you've had to those posts — imagine them wiped out by the companies that control those social networks.

Why would I ascribe such awful behavior to the nice people who run these social networks? Because history shows us that it happens. Over and over and over. The clips uploaded to Google Videos, the sites published to Geocities, the entire relationships that began and ended on Friendster: They're all gone. Some kind-hearted folks are trying to archive those things for the record, and that's wonderful. But what about the record for your life, a private version that's not for sharing with the world, but that preserves the information or ideas or moments that you care about?

It's in light of this, no doubt, that ReadWriteWeb's Jon Mitchell calls ThinkUp "the social media management tool that matters most." Indeed, as we pour more of our lives into these social sites, tools like ThinkUp, along with endeavors like the Locker Project, mark important efforts to help people own, control and utilize their own data.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

DataSift opens up its Twitter firehose

DataSiftDataSift, one of only two companies licensed by Twitter to syndicate its firehose (the other being Gnip), officially opened to the public this week. That means that those using DataSift can in turn mine all the social data that comes from Twitter — data that comes at a rate of some 250 million tweets per day. DataSift's customers can analyze this data for more than just keyword searches and can apply various filters, including demographic information, sentiment, gender, and even Klout score. The company also offers data from MySpace and plans to add Google+ and Facebook data soon.

DataSift, which was founded by Tweetmeme's Nick Halstead and raised $6 million earlier this year, is available as a pay-as-you-go subscription model.

Google's BigQuery service opens to more developers

Google announced this week that it was letting more companies have access to its piloting of BigQuery, its big data analytics service. The tool was initially developed for internal use at Google, and it was opened to a limited number of developers and companies at Google I/O earlier this year. Now, Google is allowing a few more companies into the fold (you can indicate your interest here), offering them the service for free — with the promise to notify them in 30 days if it plans to charge — as well as adding some user interface improvements.

In addition to a GUI for the web-based version, Google has improved the REST API for BigQuery as well. The new API offers granular control over permissions and lets you run multiple jobs in the background.

BigQuery is based on the Google tool formerly known as Dremel, which the company discussed in a research paper published last year:

[Dremel] is a scalable, interactive ad-hoc query system for analysis of read-only nested data. By combining multi-level execution trees and columnar data layout, it is capable of running aggregation queries over trillion-row tables in seconds. The system scales to thousands of CPUs and petabytes of data.

In the blog post announcing the changes to BigQuery, Google cites Michael J. Franklin, Professor of Computer Science at UC Berkeley, who calls BigQuery's ability to process big data "jaw-dropping."


Got data news?

Feel free to email me.

Related:

September 29 2011

From crowdsourcing to crime-sourcing: The rise of distributed criminality

Crowdsourcing began as a legitimate tool to leverage the wisdom of the crowds to solve complex business and scientific challenges. Unfortunately, these very same techniques are increasingly being adopted by the criminal underground for nefarious purposes.

The concept of crowdsourcing first gained widespread attention in an article written in 2006 by Jeff Howe for Wired Magazine. Howe defined crowdsourcing as the act of outsourcing a task to a large, undefined group of people through an open call.

The increasing application of crowdsourcing is changing "business as usual" in a wide variety of industries. In a noted example, Don Tapscott, in his book Wikinomics, described how one Canadian gold mining company facing a looming shutdown desperately turned to the general public to help solve a critical business problem. The company, Goldcorp, was so frustrated with the inability of its own geologists to locate any gold that it did something unheard of at the time: it offered $500,000 to anyone who could find and map the location of the company's own gold in its own mines. To facilitate the effort, Goldcorp posted their full datasets online. After receiving submissions from more than a thousand people in 50 different countries, Goldcorp achieved the success that had so eluded the firm previously. A member of the public used Goldcorp's data to make an incredible discovery and to locate more than $3 billion worth of gold using techniques never previously employed in the mining industry.

While numerous productive examples of crowdsourcing such as the Goldcorp case have been documented over time, these very same techniques increasingly are being exploited for criminal purposes as well.

Crime and the crowd

The growing popularity of crowdsourcing has not gone unnoticed, either, by international organized crime groups and local neighborhood thugs, each of which is quickly updating its tactics to drive operational efficiencies. Welcome to the world of "crime-sourcing." Borrowing from Howe's concept, crime-sourcing can be defined as the act of taking the whole or part of a criminal act and outsourcing it to a crowd of either witting or unwitting individuals.

The growth in crime-sourcing is shaking up long-standing business models and traditions within the criminal underground and is leading to innovations in crime. For example, all organized crime groups have historically looked upon outsiders with great suspicion: don't trust somebody you don't know and who has not been vetted. Elaborate processes were established, such as the Mafia's Omertà, to ensure newcomers to the criminal enterprise were neither rats nor cops. It would often take years of robberies, loan sharking and murder to gain the trust and confidence of the "boss."

The distributed crime network

As the world turned to globalization, so too did organized crime. Their initial attempts were limited, but generally effective. Drug cartels in Latin American began to work with organized crime groups in Eastern Europe. The Japanese Yakuza and Chinese Triads developed ties and turned to one another for very specific tasks, such as carrying out a particular "hit" or laundering a large sum of money in a different jurisdiction. Though these disparate crime groups were located in different parts of the world, they found ways to build trust and work together in their joint illicit pursuits.

Eventually, specialties emerged and criminal enterprises learned to outsource all tasks not within their specific areas of expertise. For example, in a standard phishing operation, an organized crime group might commission the creation of a scam web page and contact a secondary broker to get a list of thousands of email addresses. Using another intermediary, the crime group would get access to a compromised computer and rent a botnet to distribute the spam emails for a period of agreed upon time, such as 12 or 24 hours.

As hapless victims readily provided their banking and credit card information, the data would be culled and forwarded to the contracting criminals. The crime group would likely rent a distributed proxy network to obfuscate their true locations and to run transactions against the compromised accounts.

Of course, all this money needs to be received, processed and laundered in a way that protects the criminal enterprise, and there are numerous illicit techniques for hiring unsuspecting participants to take on the task. The most common is to place an ad in a print newspaper or an online publication offering opportunities to "work from home" and make "quick money" as an "importer/exporter."

Using this ruse, organized crime groups have duped thousands into receiving stolen property at their homes and opening shell bank accounts in their own names. After the funds are received in the account of the unwary pawn, he is instructed to immediately send them overseas via Western Union exchange for a small fee or commission. In doing so, the crime groups have crowdsourced the most dangerous part of their business, leaving behind a trail of false leads for law enforcement to find.

Web 2.0 Summit, being held October 17-19 in San Francisco, will examine "The Data Frame" — focusing on the impact of data in today's networked economy.

Save $300 on registration with the code RADAR

Strangers with a common criminal cause

One of the more interesting developments in crowdsourced offenses has been the birth of the crime "flash mob." The practice of crime flash mobs has become so common that the media have now coined a term "flash robs" to describe the ensuing theft and violence. In these cases groups of individual criminals, who may or may not even know each other, are organizing themselves online and suddenly descending into unsuspecting stores to steal all that they can in a flash. The unsuspecting merchant has little he can do when 40 unruly strangers suddenly run into his shop and run off with all his merchandise. Dozens of these cases have occurred, including one in which co-conspirators planned an attack via Facebook and Twitter that lead to the pillaging of a Victoria's Secret store in London.

Sadly, flash mobs are increasingly turning violent as innocent bystanders are being attacked and assaulted in broad daylight. In Chicago in June 2011, dozens descended on a neighborhood street and began assaulting and robbing law-abiding citizens. In the Chicago incident, 15-20 youths dragged a man off his motor scooter and severely beat him. A mere two months later in Philadelphia, a similar incident occurred.

Flash mobs are an advantageous way of crowdsourcing a robbery for the criminals involved. Using the power of the Internet, they are able to assemble an overwhelming force of unrelated strangers. Thus, if any of the participants involved are arrested, they are unlikely to be able to "rat" on their co-conspirators, whom they met for the first time at the scene of the crime.

The crime request hotline

Crime-sourcing reached new heights earlier this year when noted hacking group LulzSec opened up a hacking request hotline for the general public. The group advertised the 614 area code phone number on its Twitter feed and allowed the crowd to select LulzSec's next hacking victim. This new modus operandi in crime-sourcing allows the public to vote, "American Idol"-style, on who shall be the next victim of a crime. The group later released a statement noting that it had successfully launched distributed denial of service attacks (DDoS) against eight sites suggested by callers.

Crime-sourcing's unwitting accomplices

Not all of those who participate in a crowdsourced crime do so knowingly. In fact, employing crowdsourcing techniques, it is increasingly possible for organized crime groups to get hapless innocents to carry out key elements of a crime on their behalf. In one example, the unsuspecting (and the lustful) were enticed to solve a CAPTCHA word puzzle in order to get access to free online pornography.

It seemed like a good deal for the end-user: for each CAPTCHA they solved, a person using the name Melissa would provide access to more and more pornographic images. What the end-user did not know is that, in fact, the CAPTCHAs being solved were being used to break into Yahoo email accounts and steal information. By tapping the public appetite for pornography, organized crime groups were able to create a useful crowdsourced method of automating CAPTCHA solving in order to give them unauthorized access to email accounts.

Crowdsourcing a criminal casting call

In perhaps one of the most ingenious uses of crime-sourcing seen to date, a bank robber in Seattle utilized Craigslist to recruit a crowd of unwitting participants to facilitate his escape. In the days leading up to the robbery, the perpetrator placed an ad on Craigslist seeking workers for a purported road-maintenance project paying $28.50 an hour. He instructed his "contractors" to show up at a street location at the exact place and time an armored car was to be delivering cash to a local Bank of America.

The robber instructed all those showing up for the promise of work to wear their own yellow vest, safety goggles, respirator mask and blue shirt — the criminal's exact outfit the day of the robbery. After overpowering the armored car driver with pepper spray, the suspect grabbed a duffel bag filled with cash, ran past a dozen or so similarly dressed innocents and made his escape 100 yards away to a local creek where he floated away in a pre-positioned inner tube. 911 calls reporting the robbery described the suspect as being a construction worker in a yellow vest. When police arrived on seen, they had numerous robbery suspects from which to choose.

Crime-sourcing meets "investigation-sourcing"

While crime-sourcing has allowed organized crime groups to commit more crimes with less risk, law enforcement officials are now leveraging the power of crowdsourcing to fight crime as well.

The NYPD has already launched a social media unit to track criminals on Facebook and Twitter. More recently, as the streets of the UK burned in the aftermath of violent protests, citizens of London banded together online to identify looters.

In one of the most impressive uses of "investigation-sourcing" to date, the Canadian public came together to identify the thousands of protesters who caused millions of dollars of damage as a result of the Vancouver Canucks losing the NHL championship in June 2011. Using a variety of image processing techniques, the firm Gigapixel was able to assemble 216 publicly submitted photographs and assemble them into one seamless high-resolution image. The phenomenal resolution of the resultant picture allowed the faces of tens of thousands of riot participants to be viewed in high resolution. The identification of more than 10,000 participants by name was completed by tagging individuals in Facebook, breaking a record for the number of tags in a given image to date. Many of those identified in the photos have now been successfully arrested and prosecuted by Canadian authorities.

The future of crime-sourcing

The technology involved in various crowdsourcing techniques is, of course, neither good nor bad. What started as a legitimate methodology to tap the wisdom of crowds for the betterment of business and science has unfortunately been adopted by the criminal underground. As demonstrated in the numerous examples listed above, organized crime groups clearly understand how to employ these techniques to commit more crime with less risk.

Undoubtedly, criminals will continue to innovate and develop new tactics to grow their profits from crime-sourcing. Whether this crime trend continues unabated depends on the ability of the police and the law-abiding members of our society to organize themselves as an effective countermeasure. In the looming clash between cops and robbers to crowdsource good versus evil, victory will belong to whichever group proves itself capable of mobilizing the larger crowd.

Thanks to Tarun Wadhwa for his helpful comments on earlier drafts of this post.


Below you'll find video from Marc Goodman's Strata Summit presentation, "The Business of Illegal Data":

Photo on home and category pages: Crime Scene by alancleaver_2000, on Flickr

Related:

September 15 2011

Strata Week: Investors circle big data

This was a busy week for data stories. Here are a few that caught my attention:

Big money for big data

Opera SolutonsThere's recently been a steady stream of funding news for big data, database, and data mining companies. Last Thursday, Hadoop-based data analytics startup Platfora raised $5.7 million from Andreessen Horowitz. On Monday, 10gen announced it had raised $20 million for MongoDB, its open-source, NoSQL database. On Tuesday, Xignite said it had raised $10 million to build big data repositories for financial organizations; data storage provider Zetta announced a $9 million round; and Walmart announced it had acquired the ad targeting and data mining startup OneRiot (the terms of the deal were not disclosed). Finally, yesterday, big data analytics company Opera Solutions announced that it had raised a whopping $84 million in its first round of funding.

GigaOm's Derrick Harris offers the story behind Opera Solution's massive round of funding, noting that the company was already growing fast and doing more than $100 million per year in revenue. He also points to the company's penchant for hiring PhDs (90 so far), "something that makes it more akin to blue-chipper IBM than to many of today's big data startups pushing Hadoop or NoSQL technologies." Harris also notes that at a half-billion-dollar valuation and with 600-plus employees, Opera Solutions isn't a great acquisitions target for other big companies, even those wanting to beef up their analytics offerings. He contends this could allow Opera Solutions to remain independent and perhaps make some acquisitions of its own.

Ushahidi and Wikipedia team up for WikiSweeper

Wikipedia and UshahidiThe crisis-mapping platform Ushahidi unveiled a new tool this week to help Wikipedia editors track changes and verify sources on articles. The project, called WikiSweeper, is aimed at those highly- and rapidly-edited articles that are associated with major events.

As Ushahidi writes on its blog:

When a globally-relevant news story breaks, relevant Wikipedia pages are the subject of hundreds of edits as events unfold. As each editor looks to editing and maintaining the quality and credibility of the page, they need to manually track the news cycle, each using their own spheres of reference. The decisions that are made to accept one source while rejecting others remains opaque, as are the strategies that editors develop to alert and keep track of the latest information coming in from a variety of different sources.

WikiSweeper is based on Ushahidi's own open-source Sweeper tool, and its application to Wikipedia will help Ushahidi in turn build out its own project. After all, during major events, information comes in from multiple sources at a breakneck pace, and in crisis response, the accuracy and trustworthiness of the sources need to be quickly and transparently identified. As Ushahidi points out, this makes it a "win-win" for both organizations as they gain better tools for dealing with real-time news and social data.

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code ORM30

Angry Birds take down pigs and the economy

Invoking the seasonal declarations come March about the amount of time Americans waste at work watching the NCAA college basketball tournament, The Atlantic's Alexis Madrigal has pointed to a far more insidious and year-round problem: the amount of hours American workers lose by playing Angry Birds.

Drawing on data about the number of minutes people spend playing Angry Birds per day — 200 million — Madrigal has calculated the resulting lost hours and lost wages. He estimates about 43,333,333 on-the-clock hours are spent playing Angry Birds each year, accounting for $1.5 billion in lost wages per year.

Obviously there are some really big assumptions in this calculation. The first is that five percent of the total Angry Bird hours are played by Americans at work ... we don't know the international breakdown, nor do we know how often people play at work. But, five percent seemed like a reasonable assumption. Second, the Pew income data for smartphone ownership is not that precise, particularly on the upper ($75,000+) and lower (less than $30,000) ends. I had to pick numbers, so I basically split Americans up into four categories: people earning $30,000, $50,000, $75,000, and $100,000, then I calculated simple hourly wages for those groups (income/52/40) and did a weighted average based on smartphone adoption in those categories. The $35 per hour number I used is comparable with the $38 that Challenger, Gray, and Christmas used for fantasy sports players. But this is certainly a rough approximation. Put it this way: I bet this estimate is right to the order of magnitude, if not in the details.

Take that, Gladwell

Malcolm Gladwell raised the ire of many social-media-savvy activists last year by claiming that "the revolution will not be tweeted." Writing in The New Yorker, Gladwell dismissed social media as a tool for change. He argued that bonds formed online are "weak" and unable to withstand the sorts of demands necessary for social change.

Gladwell's assertions have been countered in many places, and a new article analyzing social media's role in the Arab Spring takes the rebuttals to a new level.

"After analyzing over 3 million tweets, gigabytes of YouTube content and thousands of blog posts, a new study finds that social media played a central role in shaping political debates in the Arab Spring. Conversations about revolution often preceded major events on the ground, and social media carried inspiring stories of protest across international borders," the authors write.

The authors describe their research methodology for extracting and analyzing the texts from blogs and tweets, but also lamented some of the problems they faced, particularly with access to the Twitter archive.

Got data news?

Feel free to email me.

Related:

August 26 2011

Visualization of the Week: Social media and the UK riots

The recent violence in the U.K. has led some in the British government to propose banning people from social networks during times of civic unrest. Prime Minister David Cameron told an emergency session of Parliament:

Everyone watching these horrific actions will be struck by how they were organised via social media. We are working with the police, the intelligence services and industry to look at whether it would be right to stop people communicating via these websites and services when we know they are plotting violence, disorder and criminality.

But The Guardian has analyzed some 2.5 million tweets relating to the events in the U.K., and the newspaper's findings challenge the government's contention that Twitter and other social networks were used to organize violence.

More than 206,000 tweets — about 8% of the total — focused on coordinating clean-up efforts following the rioting and looting.

In the interactive visualization, you can see the relationship in the various communities where outbreaks of violence occurred between the events themselves and the surge of Twitter activity. In the majority of cases, Twitter activity increased after, not before, the violence.

Screenshot from the Guardian's Twitter riot visualization
Twitter traffic during U.K. riots. Click to see the full interactive visualization

The Guardian says it will continue to examine this database in the coming weeks, just as the British government continues its inquiries into the relationship between social media and violence.

Found a great visualization? Tell us about it

This post is part of an ongoing series exploring visualizations. We're always looking for leads, so please drop a line if there's a visualization you think we should know about.

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code STN11RAD



More Visualizations:


August 18 2011

Developer Week in Review: Google Goes Yardsaling

This is the city: Los Angeles, Calif. Every year, millions of tourists flock to this Mecca of stardom and glamour, hoping that some of it will rub off on them. Sometimes they're geeks. My name is Turner. I carry a MacBook.

This is your somewhat delayed Developer Week in Review, coming this week from the Mondrian Hotel in Hollywood, a place where the laws of reality have become so distorted that paying $6 for a can of soda has actually begun to seem reasonable. There was no WIR last week, as I was trapped in an alternate universe full of hotels with Wi-Fi connections slower than dial-up. 20kb/sec, swear to God!

We're wrapping up our West Coast trip this week, a vacation that has been warped somewhat by the presence of my 16-year-old son. Certainly, if my wife and I had been traveling alone, we would not have taken a ride on a Nike Missile elevator in the Marin Headlands, or toured a WW2 submarine. Not that I'm complaining, our side-trip to the LA Gun Club this week to shoot semi-auto AK-47s and AR-15s was definitely a blast (pun intended).

All your patents are belong to us!

Google and Motorola MobilityContinuing the massive arms buildup of patent portfolios being waged among all the smartphone makers (with the exception of RIM, which seems content to take the role of Switzerland in this war), Google has assimilated Motorola. In addition to super-sizing Google's intellectual property assets in the mobile space, it also places Google in the role of a direct competitor to the other Android licensees. Until now, Google produced what were essentially engineering development platforms, but no real consumer products. Now that Google owns the DROID (the Motorola version), they're suddenly in the position of having a strong pre-existing consumer channel.

On one hand, the acquisition makes a lot of sense. Motorola is a pioneer in the mobile space, and the purchase gives Google a lot of ammo to fend off the increasing spate of patent lawsuits being lobbed its way. On the other hand, Google is now trying to sell the Android operating system to companies that it will be selling against. While it's great to talk about how Android will remain open, the reality is that once Google is fighting for market share with companies like HTC, you have to believe the relationship will become strained at best.

Will a Kzinti invasion be next?

In another case of fiction predicting reality, the last few weeks have been host to a series of social-media-organized protests, which at least in England quickly transformed into riots. Philadelphia and Cleveland fell victim to less widespread but still serious incidents of violence, and San Francisco shut down cell phone service in one BART station after word of a planned protest emerged.

None of this should be surprising to aficionados of classic science fiction, who will recognize the flash mobs now appearing as an eerie echo of the flash crowds described by author Larry Niven in his "Known Space" series. Niven used cheap teleportation as the mechanism that brought large groups of people together at the site of interesting events, but social media is proving to have an equally powerful, if more localized, affect.

Niven also predicted that the presence of a crowd would attract people whose only reason to be there is to take advantage of the chaos to loot and cause mayhem. The big question now is, how much restriction will we accept in this new medium to prevent future occurrences? We are already seeing draconian censorship and invasion of privacy as a result of the battles against child pornography and music piracy, will this be the next battlefront?

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science -- from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code STN11RAD

Pimp my language

While it seems there's a new emerging language every week, lots of developers are still being productive members of society with the old programming warhorses. But that doesn't mean a language can't get an "Extreme Makeover: ISO Editor"! Case in point, C++ moved into the new decade with the acceptance of the C++ 11 specification.

The new standard brings O-O concepts such as lambda functions and improved type coercion into the language, and it should make the lives of developers still maintaining existing C++ code much more bearable in the future. One must wonder which old-school language will be the next to get a fresh coat of paint. As the old joke goes, if they ever add O-O to COBOL, they'll have to call it add one to COBOL.

Got news?

Please send tips and leads here.

Related:

July 21 2011

If it's important, the news will find me

This is part of an ongoing series related to Peter Meyers' project "Breaking the Page, Saving the Reader: A Buyer & Builder's Guide to Digital Books." We'll be featuring additional material in the weeks ahead. (Note: This post originally appeared on A New Kind of Book. It's republished with permission.)


I don't necessarily want to give up Twitter, Google+, blogs, Zite, and so on. But it's clear: they're too much for my single core, ADD-prone brain to manage. Gone are the days when I try to consume, or even just scan, all the social media I've signed up for. I've tried drinking from the Internet's firehose and what I've ended up with is a wet face and a headache. After a couple recent experiments in offline living I'm sold on the idea that, for me, less is more. I think more clearly, and more creatively, when I unplug. It seems kinda obvious but it's taken me a decade or so to figure out: info-gorging leaves me feeling fat-headed and logy. Now, for example, rather than web surfing and info snacking in the morning, I get up early and read whatever book I'm into for an hour or two.

I do try to carve out an hour or two each day for some focused exploration — digital publishing and design are my main areas of interest — but even there I've given up stressing that I might miss some Seriously Important Item. There's just no way I can see everything that comes out. I figure if something's really important I'll catch it somewhere; I'm sure I miss plenty using this system, but all in all I feel less scatterbrained.

But here's the thing: for some topics — what's happening in Afghanistan, the debt crisis, toddler management — I'd like to figure out a way to stay informed without getting overwhelmed. I don't want a newsfeed on any of these topics. What I want is something I've started to think of as News Gems: the best of what's out there, updated only when something new has happened or when something notable has been written. And I want it presented in a way that's more compelling than a list of links. I want something that's more like the cover of a magazine, where typography, titling, and visuals all combine to say: "Hey, Pete! Look at this, it's slightly more important than that. And over here, you might also be interested in this." A sporadically delivered email or text alerting me to new stuff would also be nice; it's gotten to the point where I more or less ignore email alerts that show up each day.

What are you doing to stay on top of the topics you care about? Leave a comment or shoot me an email [peter DOT meyers; gmail].

TOC Frankfurt 2011 — Being held on Tuesday, Oct. 11, 2011, TOC Frankfurt will feature a full day of cutting-edge keynotes and panel discussions by key figures in the worlds of publishing and technology.

Save 100€ off the regular admission price with code TOC2011OR



Related:



July 20 2011

Four short links: 20 July 2011

  1. Random Khan Exercises -- elegant hack to ensure repeatability for a user but difference across users. Note that they need these features of exercises so that they can perform meaningful statistical analyses on the results.
  2. Float, the Netflix of Reading (Wired) -- an interesting Instapaper variant with a stab at an advertising business model. I would like to stab at the advertising business model, too. What I do like is that it's trying to do something with the links that friends tweet, an unsolved problem for your humble correspondent. (via Steven Levy
  3. JSON Parser Online -- nifty web app for showing JSON parses. (via Hilary Mason)
  4. Facebook and the Epiphanator (NY Magazine) -- Paul Ford has a lovely frame through which to see the relationship between traditional and social media. So it would be easy to think that the Whole Earthers are winning and the Epiphinators are losing. But this isn't a war as much as a trade dispute. Most people never chose a side; they just chose to participate. No one joined Facebook in the hope of destroying the publishing industry.

July 05 2011

Search Notes: Why Google's Social Analytics tools matter

The big search news over the past week has been the launch of Google Plus, but lots of other stuff has been going on as well. Read on for the run dow.

Google social analytics

Plus isn't the only social launch Google had recently. The company also pushed out social analytics features in both Google Analytics and Google Webmaster Tools.

If you use the new version of Google Analytics, you'll now see a social engagement report. Use the social plugin to configure your site for different social media platforms to monitor the behavior of visitors coming from those platforms. Do those coming from Twitter convert better than those coming from Facebook? Do those who "+1" a page spend more time on it? Those are the sorts of questions the new social reports aim to answer.

You can also use Google Webmaster Tools to see how +1 activity is impacting how searchers interact with your pages in search results. In particular, you can see if the click-through rate of a result improves when it includes +1 annotations.

This is just one example of how the silos of the web are integrating. You shouldn't think of "social" users and "search" users when you are doing audience analysis for your site. You instead have one audience who many be coming to your site any number of ways. Engaging in social media can help your site be more visible in search, as results become more personalized and pages that our friends have shared, liked, and "plussed" show up more often for us.

Some may wonder if integrations like this mean that Google is weighting social signals more strongly in search. But those kinds of questions miss the point. The specific signals will continue to change, but the important thing is to engage your audiences wherever they are. The lines will continue to blur.

Google Realtime Search goes offline "temporarily"

A few day ago, Google's realtime search mysteriously disappeared. The reason: Google's agreement with Twitter expired and Google is now working on a new system to display realtime information. While this has temporarily impacted a number of results pages (such as top shared links and top tweets on Google News), it has not impacted Google's social results, which show results that your friends have shared.

Google social results

New Google UI

Google launched the first of many user interface updates last week, with the promise of many more changes to follow throughout the summer.

Google, Twitter and the FTC

But the Google world is not just about launches. The FTC formally notified Google that they are reviewing the business. Google says that they are "unclear exactly what the FTC's concerns are" but that they "focus on the user [and] all else will follow."

The Wall Street Journal reports that the investigation focuses on Google's core search advertising business, including "whether Google searches unfairly steer users to the company's own growing network of services at the expense of rival providers."

The FTC may also being investigating Twitter, due to how Twitter may be acquiring applications.

Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders.

Save 20% on registration with the code AN11RAD



Google Plus (or is it +?)


Google PlusAnd of course we have to dig into that well-chronicled launch. As you're no doubt aware, Google launched their latest social effort last week: Google+. Or Google Plus. Or Plus. Or +. I don't know. But it's different from Plus One (+1?). Also it's not Wave, Buzz, Social Circles. Or Facebook.

I've just started using it, so I don't have a verdict on it yet, although I don't know that I buy intoGoogle's premise that "online sharing is awkward. Even broken." And that Google Plus will fix that. It doesn't mean I won't like the product, either. Google is of course under more scrutiny than usual since earlier social launches haven't gone over as well as they'd have liked. What do you all think of it?

Lots of sites have done comprehensive run downs, including:

(Google's Joseph Smarr, a member of the Google+ team, will discuss the future of the social web at OSCON. Save 20% on registration with the code OS11RAD.)



Yahoo search BOSS updates


Yahoo launched updates to their BOSS (Build your own search service) program. If you're a developer who uses Yahoo BOSS, you might be interested in the changes.

Schema.org and rel=author

A few weeks ago, Google, Microsoft, and Yahoo launched the schema.org alliance, which provides joint support for 100+ microdata formats. At the same time, Google announced support for rel=author, which enables site owners to provide structured markup on a page that specifies the author of the content.

The schema.org announcement seems to be a foundational announcement to encourage platform providers, such as content management system creators, to build in support of microdata formats for future use by the search engines.

On the other hand, Google has already launched integration of rel=author with search results. You can see examples of how this looks with results for the initial set of authors Google is working with.



Related:


June 14 2011

The blurring line between speech and text

As we watch the sordid cavalcade of media gaffes — from Anthony Weiner's Twitter photos to Chrysler's "slip of the tongue" (someone tweeting on behalf of the car maker mistakenly thought they were using their personal account when they declared that Detroit was full of terrible drivers) — we are seeing a society that is coming to terms with the blurring line between text and speech. That is, the ephemeral nature of all speech is being given the permanence of text.

We will spend the next generation coming to terms with the consequences.

Our apologies - our account was compromised earlier today. We are taking steps to resolve it.less than a minute ago via web Favorite Retweet Reply


Once something is said it cannot be unsaid. True. But historically it couldn't be shared to a wider circle of listeners. Speech is not permanent. Speech gives way to time and passes into the fog of memory. Therefore the social norms governing speech are more forgiving. We are expected, allowed even, to say things we without due consideration, in close company, knowing that we will regret some portion of what we say. We are able to use the full context of a conversation (who is there, what has been said before etc.) to nuance our speech and say things that wouldn't look good when reduced to text. And yet on social networks we speechify, we talk and we are saying plenty of things we might regret. Such speech isn't meant to be a permanent record. But it is. As Meghan Garber writes in a Nieman Lab post:

As a culture... we tend to insist on categorizing our communication, drawing thick lines between words that are spoken and words that are written. So libel is, legally, a different offense than slander; the written word, we assume, carries the heft of both deliberation and proliferation and therefore a moral weight that the spoken word does not. Text, we figure, is: conclusive, in that its words are the deliberate products of discourse; inclusive, in that it is available equally to anyone who happens to read it; exclusive, in that it filters those words selectively; archival, in that it preserves information for posterity; and static, in that, once published, its words are final.

We are hurtling toward a world of total information capture where email, texting, instant message and mobile video are documenting our everyday speech and action — in effect rendering all speech as text. There will be few places to "talk" without that talk being given weight and permanence.

We are then faced with two options: Either give up the liberties that speech allows — thinking "out loud," using the context of the conversation to add meaning to a comment and so on — or become more lenient with speech that happens to become text. In the case of Weiner, his behavior is unacceptable in any context. As a society we understand his transgression and he is being punished for it. Fair enough. In the case of Chrysler, a mistake was punished through Chrysler firing both the Tweeter and the entire social media agency that person worked for. I hope in the future we are able to see the distinction and dole out our punishments accordingly. We all say things we regret. Now we all write things we regret. Perhaps as a result of this shared reality we will learn a bit more forgiveness for each other.

Joshua-Michéle Ross will discuss social media architecture at Web 2.0 Expo New York 2011, being held Oct. 10-13 in New York City.

Save 20% on registration with code WEBNY11RAD



Related:


June 01 2011

Will your business survive the digital revolution?

Over the last few years we've watched in giddy disbelief as a web-based social network launched from a dorm room at Harvard University unexpectedly found its way to be an enabler of a Middle East uprising. We've seen how new types of media have propelled people and events into the spotlight and even helped elect a U.S. president. We've looked in awe as mobile devices connected to a ubiquitous network have brought global commerce to the most remote parts of the developing world. We've seen 100-year-old businesses vanish as cocky upstarts replace their once unshaken dominance. We've delighted as citizens have been empowered by a new ease in which to leverage recently liberated stores of data held by governments.

With just these few observations it's clear to all of us that technology is no longer just in support of our lives and organizations; it's taking a commanding and empowering position. And it's vital that we all fully understand just how profound these changes truly are (and will be). The very survival of your organization likely depends on it.

Are we at the start or the end of this technology revolution?

We observe these incredible events unfold and this may lead us to believe we've reached a new pinnacle of technological innovation. Many of us might believe that we're peaking in our capacity to make amazing things happen. To them I say: we've barely even started.

From economics to democracy, from health to entertainment, from retail to education, and everything else in-between, something remarkable is happening.

In my view the events described here are just the beginning of a seismic shift in our human experience. Indeed, these innovations are not reserved for a single nation or continent. This technology-based revolution is the first to quickly reach and impact every corner of the planet.

Every generation believes it lives through remarkable and changing times. And that is probably true. But the large transformations, most recently like those of both the agricultural and industrial revolutions, don't happen that often. These changes are a railroad switch that shifts the course of human destiny. Some have coined our era as the information revolution. But the emergence of the information age has merely been the precursor and a glimmer of things to come.

The true revolution is the convergence of many things. Revolutions require more than just a few elements to be in place. Historically they have required a unique alignment of qualities such as economic and political conditions, readiness for change, demographics and a catalyst.

We see much of that today. Of course, today the catalyst is the Internet. It's also the ease in which so many of us can now produce digital innovation (creating new value through electronic, non-analog means). It's also about the availability of low-cost, ubiquitous global communication networks with an abundance of devices connected. It's close to zero-cost cloud-based storage. With low cost storage comes the easy retention of massive volumes of data and when it's coupled with the fact there are so many opportunities to collect that data; new uses and value can be derived from it.

There is a new world order that is unique to our time that is also enabling this change. Not least the emergence of prosperity in many part of the world and the breathtaking rise of the BRIC nations and others. This prosperity is creating a new class of educated, global participants. This means more competition and it means more innovation. It's all these things and more converging to produce a significant technology-based social and business disruption.

As this technology revolution unfolds, does your business have a survival plan?


Web 2.0 Summit, being held October 17-19 in San Francisco, will examine "The Data Frame" — focusing on the impact of data in today's networked economy.

Save $300 on registration with the code RADAR

The evidence is clear

The signals are in both the destruction of existing paradigms and in the creation of completely new ones. We're watching entire industries disappear or be reinvented through digital transformation: newspapers, books, movies, music, travel agents, photography, telecommunication companies, healthcare, fund-raising, stock-trading, retail, real estate, and on and on.

Digital innovation has few geographic boundaries, so the disruptor can emerge from almost any place on earth.

Completely new models are emerging: location-based services, mobile apps, gamification, payment systems and new forms of payment, cloud computing, big data analysis and visualization, recommendation engines, near-field communications, real-time knowledge, tablets and other new form factors, augmented reality, gesture-based computing, personal medicine, large scale global social networks, microblogging and more. Many of these did not exist five years ago and many more will exist in the next five years. In fact, the next major disruptor is probably already underway. This kind of change is equally exciting and terrifying for organizations.

Why it is different this time

When the Walkman became the Discman, the music industry flourished. But when the digital MP3 player was introduced, the music industry was fundamentally and forever reinvented. Digital transformations are not subtle or calm. They are equal measure painful, chaotic, and exciting.

When mobile phones were introduced they enabled people to untether themselves from a fixed wire and talk almost anywhere. That was useful and convenient. But when smartphones freely enable the coordination of people and events that facilitates the overthrow of a corrupt government, this is not business as usual. That's a fundamental shift in how humans communicate and coordinate their activities.

It will be a rough ride

Sure it won't all be rosy and bad people will do bad things using more of this technology. But that's certainly not news. The vulnerabilities will grow but so will our ability to fight attacks. Opportunities in security will remain in high demand.

There will also be booms, bubbles, and busts. That's a normal part of the economic lifecycle. In fact, outside of the obvious pain it causes, a bust can be a valuable response to irrationality in the market. We will see many of these cycles through this transformation, but I believe we will net out with a continued exponential growth in digital innovation.

The big stuff is yet to come

When you observe how digitization causes significant economic restructuring and the emergence of completely new forms of business, and you factor in an entirely new level of social connectedness, it's hard not to conclude that big things are ahead.

It's also easy to be unfazed by the digital change underway, particularly if you're working deep within it. In addition, it's equally easy to become fatigued and even cynical about further change. But stop, elevate yourself above the chaos and noise, and the digital transformation is a palpable societal disruption.

At the heart of this blog is not a regurgitation of change that many of us already recognize and embrace; moreover, it's about urging each one of us not to underestimate this transformational shift. It's also neutral on the subject — but recognizes — the social and economic negatives that may result. Big shifts like these do evoke, for example, strong feelings of nationalism (somewhat ironically). But I'll steer away from this subject for now.

Failure to anticipate, prepare and respond sufficiently is a significant organizational risk. In other words, delivering your product or service to the market of yesterday and today without constantly exploring reinvention for the market of tomorrow may be certain business suicide. And while that's largely always been true, it's seldom been so necessary and urgent.

Once we recognize the magnitude of change that digital innovation is causing and may bring in the months and years ahead, it will help us to think bigger and to think in ways that may previously have seemed absurd.

As inventors and facilitators of the future we would do ourselves a great injustice to underestimate the change.

The digital revolution: my own personal experiences

Let's just take a quick look at my world for a moment. In many areas of my life it's fascinating comparing how I did things in 2001 vs. how I do them now in 2011. By the way, it's worth noting that while I immerse myself in technology and innovation through my work, I'm not particularly unique in the way I use technology outside of work.

So let's take a look at some of the changes over the course of 10 years: I no longer wear a watch. No need, I get time from my smartphone. I got rid of my landline phone. My phone is my smartphone. I never go to a bank. Done online. I don't know anyone's phone number by heart. I select a name and my phone dials the number. Outside of a radius of a few miles, I don't know how to get anywhere anymore without my GPS. I never use a map. I barely mail a letter. My use for stamps is diminishing. I seldom print anything. Everything that can be reserved, I do online. I don't watch scheduled TV. I watch shows off my digital video recorder or computer when I want (in HD, no less). I use my smartphone for less and less voice calls. I text. I read, take classes, post photos, write, research, play, watch movies, listen to music, comparison shop, order insurance, complain and more all online.

I'm pretty sure your experiences are fairly similar.

Perhaps it is a little bit of an exaggeration, but I mostly only emailed and consumed static content online in 2001.

Almost every one of these areas represents an industry. And as a result of these enabled behavioral changes over the course of a mere 10 years, within these industries many organizations have been created and destroyed.

If this kind of transformation can happen in the past 10 years, with everything we know about how things are trending, what might our lives look like in 10 years from now? While not necessarily a novel question, I'm simply suggesting each of us are being forced to think bigger and more innovatively than ever before about the realities and possibilities of the future.

So what should organizations do?

I'm confident most enlightened organizations have some form of a strategy in place. That's good news. For those that don't or are hesitant, it's time to act. In either case, the following are just a few fundamentals worth considering:

  • Recognize the magnitude of the digital revolution in acceptance and in action.
  • Invest in understanding how your organization can anticipate and respond quickly to change.
  • Monitor and interpret trends and new technology entrants.
  • Audit your vulnerabilities and score progress and risk on a regular basis.
  • Prepare by taking greater risks.
  • Innovate as standard practice (this doesn't just happen, you need a strategy).
  • Make bold changes in order to continue to succeed when disruption is a certainty.

Technology used to be the domain of a few. Now it's the fabric woven into how we all live, work, and play. Today it has the power to create and destroy value in an unprecedented manner. That's a big deal for every organization.

It's likely a very big deal for you, too.



Related:


March 22 2011

For election info, the Internet reaches a new high-water mark

The Pew Internet and Life Project recently released new research on the Internet and Campaign 2010 that 73% of adult Internet users went online to get news, information or otherwise be involved in last year's elections. That represents some 54% of all US adults, or a majority of the population, that are now are turning to the Internet when election season comes around. Expect that to grow further in the presidential season next year.

"As the Internet has developed as a tool for political engagement and information-seeking, the audience for online political content has also changed," said Aaron Smith, Pew Internet senior research specialist in a prepared statement. Smith authored the report. "These online spaces are a meeting place where politically engaged Americans of all stripes — young and old, conservative and liberal — can come to catch up on the latest events, share their thoughts on the political news of the day, and see what their friends have to say about the issues that are important to them."

Online

Mainstream media websites occupy the top 5 spots in the list of the main sources of news cited by respondents, next to Yahoo and Google. Only 2% of those surveyed said that they visited a candidate's website, setting a low bar for that number to explode in the 2012 cycle as both incumbents and those wishing to oust them turn to the web to "go direct" to citizens.

For some, where they're visiting is a little less clear: 29% of those surveyed chose "other" for their main sources of news, which could mean any number of sources in the blogosphere or the rest of the Internet.

Chart: Pew_Internet and Campaign 2010_Source

Source: The Pew Research Center's Internet & American Life Project, November 3-24, 2010 Post-Election Tracking Survey. n=2,257 national adults ages 18 and older, including 755 cell phone interviews. Interviews were conducted in English and Spanish. Note: totals may exceed 100% due to multiple responses. This chart is based on data from "22% of online Americans used social networking or Twitter for politics in 2010 campaign,"a report on politics and social media by the Pew Research Center’s Internet & American Life Project. This report is available in full on our website at http://pewInternet.org/Reports/2011/The-Internet-and-Campaign-2010.aspx. The Pew Internet & American Life Project is one of seven projects that make up the Pew Research Center, a nonpartisan, nonprofit "fact tank" that provides information on the issues, attitudes and trends shaping America and the world. The Project produces reports exploring the impact of the Internet on families, communities, work and home, daily life, education, health care, and civic and political life. For more information about the Project, please visit http://pewInternet.org/About-Us.aspx.
Tags:
Pew_Internet and Campaign 2010_Source
Powered By: iCharts | create, share, and embed interactive charts online
 

Social

Social networking is an increasingly important factor in American consumption of political news. According to eMarketer, in 2011 half of all US Internet users log in monthly to Facebook.

There are now tens of million American Twitter users, though a small percentage of those users account for the bulk of the tweets. According to Pew, one in five online adults (22%) used Twitter or a social networking site for political purposes in 2010. Twitter has some 200 million users worldwide, approximately 60 million of them are in the United States.

Video

With more broadband access, Internet-connected flatscreen televisions, set-top boxes and an explosion of video-capable smartphones and tablets, people are also watching in more places, spaces and times than ever before. Timeshifting stopped being a science fiction phenomenon years ago with the introduction of digital video recorders, familiar now as "DVRs", but on-demand video from Apple, Amazon, YouTube, Netflix and a host of other sites are available to those able to pay the toll for broadband Internet access.

Chart: Pew_Internet and Campaign 2010_Online Video

Source: The Pew Research Center's Internet & American Life Project, November 3-24, 2010 Post-Election Tracking Survey. n=2,257 adult Internet users ages 18 and older, including 755 cell phone interviews; interviews were conducted in English and Spanish. n=1,628 based on Internet users. This chart is based on data from "22% of online Americans used social networking or Twitter for politics in 2010 campaign,"a report on politics and social media by the Pew Research Center’s Internet & American Life Project. This report is available in full on our website at http://pewInternet.org/Reports/2011/The-Internet-and-Campaign-2010.aspx. The Pew Internet & American Life Project is one of seven projects that make up the Pew Research Center, a nonpartisan, nonprofit "fact tank" that provides information on the issues, attitudes and trends shaping America and the world. The Project produces reports exploring the impact of the Internet on families, communities, work and home, daily life, education, health care, and civic and political life. For more information about the Project, please visit http://pewInternet.org/About-Us.aspx.
Tags:
Pew_Internet and Campaign 2010_Online Video
Powered By: iCharts | create, share, and embed interactive charts online
 

Polarized?

According to the report, some "55% of all Internet users feel that the Internet increases the influence of those with extreme political views, compared with 30% who say that the Internet reduces the influence of those with extreme views by giving ordinary citizens a chance to be heard."

Is the Internet polarizing? Micah Sifry of techPresident wasn't so sure:

This could be true, or it could be a false positive. What if people are conflating things? Arguably politics in America is more polarized, but cable TV and talk radio and paid negative political advertising are driving that shift, while the Internet is just an overall disruptive force that is enabling lots of more people to speak up and connect with the like-minded and unlike-minded alike.

Groups

Polarization can express itself in how people group online and offline. As with so many activities online, political information gathering online requires news consumers to be more digitally literate.

That may mean recognizing the potential for digital echo chambers, where unaware citizens become trapped in a filter bubble created by rapidly increasing personalization in search, commercial and social utilities like Google, Amazon and Facebook.

Pew's research found that some of the people surveyed at least recognized the complexity of the political landscape online. With a few clicks of the mouse, keystrokes or finger taps, a news consumer can find the best and worst of humanity is mirrored online. The open platform of the Internet allows extremist views to co-exists alongside moderate perspectives. It also provides means for like-minded citizens to find one another, using the Internet as a platform for collective action.

Chart: Pew Internet_Internet and Campaign 2010_Point of View

Source: The Pew Research Center's Internet & American Life Project, November 3-24, 2010 Post-Election Tracking Survey. N=2,257 adult Internet users ages 18 and older, including 755 cell phone interviews; Interviews were conducted in English and Spanish. n=1,167 based on online political users. This chart is based on data from "22% of online Americans used social networking or Twitter for politics in 2010 campaign,"a report on politics and social media by the Pew Research Center’s Internet & American Life Project. This report is available in full on our website at http://pewInternet.org/Reports/2011/The-Internet-and-Campaign-2010.aspx. The Pew Internet & American Life Project is one of seven projects that make up the Pew Research Center, a nonpartisan, nonprofit "fact tank" that provides information on the issues, attitudes and trends shaping America and the world. The Project produces reports exploring the impact of the Internet on families, communities, work and home, daily life, education, health care, and civic and political life. For more information about the Project, please visit http://pewInternet.org/About-Us.aspx.
Tags:
Pew Internet_Internet and Campaign 2010_Point of View
Powered By: iCharts | create, share, and embed interactive charts online
 

While the diversity of sources may have radically expanded and the delivery systems for them may have multiplied, finding and establishing the truth of what's out there can be be challenging.

Caveat lector.

You can download the full report as a PDF here. For digital politicos, it's a must read.

Four short links: 22 March 2011

  1. EveryBlock Redesigned -- EB has been defined for a while now as "that site that makes my city's statistics useful and relevant". Now they're getting more into the user-reporting: As valuable as automated updates of crime, media mentions, and other EveryBlock news are, contributions from your fellow neighbors are significantly more meaningful and useful. While we're not removing our existing aggregation of public records and other neighborhood information (more on this in a bit), we've come to realize that human participation is essential, not only as a layer on top but as the bedrock of the site. They have a new mission: our goal is to help you make your block a better place. That's a bold goal, and quite a big change from where they were at. Will they manage any aspect of journalism, or will this become a GroupOn-ad-filled geo-portal for MSNBC? Looking forward to finding out.
  2. Typography in 8 Bits: System Fonts -- nifty rundown of fonts from the microcomputer days. I still go a bit weak-kneed at the sight of the C64 fonts. Which aspect of the system you're building will be remembered with weak knees in (gulp) thirty years' time? (via Joshua Schachter)
  3. Twitter in the Christchurch Earthquake -- analysis of the tweets around the quake, including words and retweets. (via Richard Wood)
  4. ChumbyCV -- computer vision framework for Chumby. CV on low-power ubiquitous hardware makes devices smarter and be higher-level sensors of activity and objects. (via BERG London)

March 18 2011

Social media design should start with human behavior

Businesses are under pressure to crack the social media code. There's all those tools and platforms to harness, and all those best practices to adopt. Staying on top of it is exhausting. Staying ahead of it is almost impossible.

Fortunately, there's a better way.

In the following interview, Paul Adams (@Padday), global brand experience manager at Facebook and a speaker at the upcoming Web 2.0 Expo, explains how a simple commitment to value can unravel the complications of social media. The key, Adams says, is to understand and serve basic human behavior.


How is social media design lacking? How can it be improved?

Paul AdamsPaul Adams: I'm not sure we should even start with the concept of "social media design." Social behavior in humans is as old as our species, so the emergence of an Internet based on social behavior is simply our rudimentary technology catching up with offline life. Thinking about "social design" should be embedded in everything we do, and not thought of in isolation. We should think about it the same way designers of electronic appliances think of electricity — it's just there, it's the hub, powering other things.

It's problematic that many businesses focus on existing and emerging technology, and not on social behavior. Thinking about platform integration first, like Twitter or Facebook, or technologies first, like what could be enabled by "mobile location" or "real-time updates," is the wrong place to start. Often, businesses need to step back and consider what will motivate people to use what they are developing, above and beyond what exists today. Something that I've been saying for a while is that human behavior changes slowly, much slower than technology. By focusing on human behavior, not only are you much more likely to create something that people value and use, but you're more likely to protect yourself from sudden changes in technology.

Interestingly, even this may not be enough. When it comes to designing around social behavior, it's not just about meeting a need that people currently struggle with, it's about understanding why people would change their current behavior. When things don't work well, people develop workarounds and form habits. These habits are hard to shift. Why try something new, even if it looks a little better, when what you currently use works fine? This is why basic technologies such as SMS remain popular with people, and advanced technologies like Google Wave didn't catch on. SMS works, so why try something else?

People also take the path of least resistance, and trying something new involves change. This is compounded by the fact that many of these opportunities for design are latent needs. In other words, people can't see the problems they have because they have developed workarounds. So simply saying that what you developed is better won't cut it.

Web 2.0 Expo San Francisco 2011, being held March 28-31, will examine key pieces of the digital economy and the ways you can use important ideas for your own success.

Save 20% on registration with the code WEBSF11RAD

Search engines are integrating social media into their products. How do you see this area evolving over the next few years?

Paul Adams: Integrating content created on social media platforms into search engines raises big questions around data usage and privacy. How this evolves will be very interesting because it highlights the difference between being public and being publicized.

At SXSW last year, danah boyd spoke articulately about this — her talk is worth checking out. My take, heavily influenced by the work of danah and others, is that when most people think of the public, they think of the public as they have experienced it offline in the past, usually being outside and being bounded by space and time. The most public setting for many would be something like a large music festival, where many thousands of people are gathered, and at least hundreds can observe your behavior. But only those people who are there, at that time, can observe what you say and do. Search and discovery platforms online, however, are very different. They are bounded by neither space nor time.

The problem is that many people don't realize or understand that. They have no idea what it means to "index the web." They act in the moment, in a specific context, and don't think about how that content might look in the future, in a different context. Some technology pundits say that people should be more careful with what they post, but I strongly disagree with that. It's up to us — the people designing and building the technology — to design the right thing in the first place. Our tools should respect the context in which the content was first created.

This raises a really important question: Does the fact that a post or update was public on a blog, social media site, or review site make it permissible for anyone to take that content and publish it wherever they choose? I don't think it does. Yet, that's what search engines are starting to do. It's interesting to me to see billboard ads with tweets on them. Do those people know that what they said is plastered across town? Is that what they expected when they created the content? The unfortunate fact is that many people will probably come to understand what it means to post publicly online by exposure of something they thought had a limited audience. And I don't think that's a good thing for anybody.

How important is reputation in social media interactions?

Paul Adams: Reputation, or the broader concept of "Identity," is the cornerstone for all other interactions. People need to know who they are interacting with in order to act appropriately, and they constantly scan for cues. As with influence, this is really complex. For example, it's possible to take the view that no one has a single reputation. We are uniquely viewed by every single person, based on our previous interactions and their previous experiences. In this light, the current trend in representing reputation online with people being assigned a single score, makes no sense.

We're also undervaluing the influence of strong ties. The people closest to us are often the ones who influence us most. To heavily generalize, people are influenced by five different groups in the following descending order: closest friends and family, groups of people they have a strong affiliation with, groups of people who are similar to them, very large groups of people, and finally, by random individuals they don't recognize.

Is there too much focus on the total number of followers or "likes"?

Paul Adams: We're still seeing the fans and followers arms race — businesses trying to gather as many fans as possible. But I think that's fundamentally wrong. It's more important to focus on quality, not quantity, of connections.

For example, many brands run competitions on social media platforms. You have to "Like" or "Follow" that business to enter. So the question is whether they are making connections with advocates of their brand, or with people who simply love competitions. If it's the latter, then they're filling their social media interactions and data with noise.

As I mentioned earlier, people are often most influenced by their closest friends. So only make connections with true advocates of your brand, and market to the friends of those fans.

Will we get to a point where "social media" is not an online thing, but a bridge between the digital and real worlds?

QR codePaul Adams: I think we're already seeing it happening. We see Facebook, Twitter and Google Maps stickers on business windows all over town. I do think this is where it's headed. As I mentioned earlier, social media should be like electricity. It's there, powering everything, but we don't really think about it.

Our phone, or whatever we carry around with us, will probably be our primary source and producer of social media data, so it's important that when we use it, we're not burdened by its place in the ecosystem — for example, by seeing constant privacy controls or too many invasive alerts.

Fundamentally, the phone collects a number of datasets that other devices don't. It knows who we communicate with the most, who we care about the most — because it knows who we call and text most often — and it also knows where we are, where we've been, and probably where we're going. And in the near future, it will know the things we buy.

Mobile is going to be a very disruptive space, and I'm not sure how it will evolve. Rather than try and predict which technologies will be dominant, I think the safer bet for businesses is to understand how these technologies will support human behavior and how they will help people do things they are struggling to do today.

Photo: QR code by Projeto Sticker Map on Flickr



Related:




March 16 2011

Knowledge management in the age of social media

Twitter and office buildingKnowledge management, which is broadly defined as the identification, retention, effective use, and retirement of institutional insight, has been an elusive goal for most large organizations. It is motivated by practical business intent, such as the distribution of knowledge to avoid relearning the same best practices over and over. However, in reality it is a requirement that is remarkably difficult to attain. Some of the smartest people I have worked with have been frustrated by their efforts, not through lack of trying or ability but by the inherent challenges it presents. The emergence and impact of social media in the enterprise forces us to rethink knowledge management and creates completely new challenges.

Today, some of the core issues with existing knowledge management approaches can be categorized as behavioral and technical (I recognize the complexity of the subject and acknowledge there are many more qualities to examine; using the following two should be sufficient to support the points in this blog).

1. Behavioral

In order for a knowledge management system (KMS) to have value, employees must enter insight on a regular basis and they must keep the knowledge current (we can all agree that out-of-date information, which has reference value, is much less useful as the general desired state of actionable knowledge). Seldom are either of these behaviors adequately incentivized. By sharing tacit knowledge, many employees believe they are reducing their own value to the organization. In addition, updating the information requires effort, which is rarely a priority against the core responsibilities of the employee.

Psychic income earned on your own time might provide incentive for Wikipedia updates, but it doesn't often translate to well spent effort on company time.

2. Technical

When presenting to audiences I often ask if it is easier for them to use a public search engine to find information about their organizations or use their own organizations' websites for a search. As you might guess, the majority of the room goes with the public engine.

It's remarkably difficult to organize information in the right manner, make it searchable, and then present it so the most relevant responses are at the top of the search results. In addition, organizational information is hardly the example of pristine structure. While public search engines benefit from counting the number of links between items (a good measure of popularity), internal systems have no such equivalent. Unstructured content is the king of the public web, whereas it is the bane of the enterprise. (Things will change in the future as new technologies, such as those that support the semantic web, are broadly adopted and implemented).

The situation is compounded when employees are disillusioned by the effectiveness and effort to use the KMS and resort to old habits, like asking colleagues or improvising in the absence of guidance (thus repeating mistakes or missing best practices). The system often fails to be adopted — or at best is used by a small proportion of the organization — and no amount of resuscitation is enough to bring it back to life.


Web 2.0 Expo San Francisco 2011, being held March 28-31, will examine key pieces of the digital economy and the ways you can use important ideas for your own success.

Save 20% on registration with the code WEBSF11RAD


Social media completely changes the existing knowledge management paradigm

It may be time to put down your tools in trying to make the old model of knowledge management work; social media is a completely new beast that changes many of the rules.

In the old world order, knowledge was usually created and stored as a point in time. In the future, organizational policy or insight may not be formed by an individual creating a document that goes through an approval process and is ultimately published. It will likely begin with an online conversation and it will be forever evolving as more people contribute and circumstances change.

Social media takes knowledge and makes it highly iterative. It creates content as a social object. That is, content is no longer a point in time, but something that is part of a social interaction, such as a discussion. It easily disassembles the pillars of structure as it evolves. As examples: content in a micro-blogging service can shift meaning as a discussion unfolds; conversations in enterprise social networks that link people and customer data can defy categorization; and internal blogs and their comments don't lend themselves to obvious taxonomy.

The days of the single, authoritative voice are coming to an end. The community has prevailed.

The shift to the adoption of enterprise social computing, greatly influenced by consumerization, points to one emergent observation: the future is about managing unstructured content.Let's consider the magnitude of this for a moment. Years of effort, best practices, and technologies for supporting organizational insight in the form of curated, structured insight has to be rethought. It's an enormous challenge, but it may in fact be the best thing that ever happened to knowledge management.

There is an important silver lining to this story

In the long run, social media in the enterprise will likely be a boon for knowledge management. It should mean that many of the benefits we experience in the consumer web space — effective searching, grouping of associated unstructured data sources, and ranking of relevance — will become basic features of enterprise solutions. It's likely we'll see the increasing overlap between public and private data to enhance the value of the private data.

For example: want to know more about a staff member? Internal corporate information will include role, start date, department etc., but we may get additional information pulled in from social networks, such as hobbies, photos or previous employment. Pull up client data and you'll get the information keyed in by other employees, but you might also get the history and values of the company, competitors, and a list of executives, gleaned from the broader repository of the public web. I'll leave the conversation about privacy for another day.

It's likely that social-media-driven knowledge management will require much less of the "management" component. Historically we've spent far too much time cleaning up the data, validating, and categorizing it. In the future, more of our time will be spent analyzing all the new knowledge that is being created through our social interactions. Smart analysis can result in new insight, and that has powerful value for organizations.

No doubt this is an enormously complex space and social media magnifies the challenges. The time is right to evaluate your knowledge management strategy. New value creation starts now.

Photo: Office, by ianmunroe on Flickr



Related:


March 07 2011

Social media in a time of need

The Red Cross and the Los Angeles Fire Department have been at the forefront of adopting social media in crisis response. That's not entirely by choice, given that news of disasters has consistently broken first on Twitter. The challenge is for the men and women entrusted with coordinating response to identify signals in the noise.

Public expectations for those staff are high, as research released by the Red Cross on at the Emergency Social Data Summit last year showed. Nearly half of respondents ask for help on social media and 3 in 4 would expect help to arrive within the hour. "We've set up an expectation that, because we're present in these spaces, we're listening," said Wendy Harman (@wharman), the Red Cross social media director, at a training conference in Tampa, Fla.

At present, those high expectations don't always match up with the capabilities that first responders possess. That's changing. First responders and crisis managers are using a growing suite of tools for gathering information and sharing crucial messages internally and with the public. Structured social data and geospatial mapping suggest one direction where these tools are evolving in the field. Prototype group-messaging platforms like Loqi.me point to ways they may evolve further this year.

Some tools, such as the Red Cross shelter web app and the Shelter View iOS app, allow the public to access information. "If we're going to ask the public for help in a disaster, we need to give them tools," said Trevor Riggen (@triggen), senior director at the Red Cross.

The Red Cross is working on building better filtering tools and mapping geotagged updates to help improve their situational awareness, said Riggen. By aggregating crisis data, visualization, and analysis into operations, they're pressing the opportunity to see where issues are emerging and introducing more relevance into social streams. "One tweet doesn't tell us to shift," said Riggens. "50 tweets will."

The opportunity that more crisis managers are seeing lies in the situational awareness that comes from analyzing massive amounts of social data generated in disasters. In those moments, "the public is a resource, not a liability," said FEMA administrator Craig Fugate (@CraigAtFEMA) last year. "Social media's biggest power, that I see, is to empower the public as a resource."

Where 2.0: 2011, being held April 19-21 in Santa Clara, Calif., will explore the intersection of location technologies and trends in software development, business strategies, and marketing.

Save 25% on registration with the code WHR11RAD



Managing online crisis response at the LAFD


Long before the Red Cross joined Twitter and used it in the Haiti earthquake response, Brian Humphrey (@brianhumphrey) of the Los Angeles Fire Department (LAFD) was already online, listening.

"The biggest gap directly involves response agencies and the Red Cross," said Humphrey, who currently serves as the LAFD's public affairs officer. "Through social media, we're trying to narrow that gap between response and recovery to offer 'real-time relief'."

Humphrey, who added the @LAFD account to Twitter in 2006, may well have been the first emergency responder on the service. In the following video, he talks more about how technology has changed during his career and how the LAFD is using various social platforms.

For instance, Humphrey found a use for Foursquare that goes beyond a simple check-in: real-time sourcing for a given location. If he hears about an issue in the greater Los Angeles area, Humphrey looks on Foursquare to see who's checked in near there and reaches out to them to learn more about the conditions on the ground. Any relevant information is then passed along to operational staff responding to the incident.

In a second interview, below, Humphrey offers some of his best practices for online crisis communication. Currently, his department operates more than 80 social media accounts, many of which are populated using email. The LAFD is also using lightweight audio creation tools, like Cinch, to distribute reports. New social platforms don't replace existing communication infrastructure like the radio or broadcast television — they complement them.

While the tools may change, the goals have not: responders learn what's happening in the community, share relevant information, and protect those in need. What's shifted is that in an age where people are increasingly equipped with camera phones and Internet connections, citizens can become sensors in disasters. As more platforms like the recent geolocation app that connects citizen first responders to heart attack victims come online, keeping those ears on will continue to grow in importance.



Related:




March 01 2011

Empowering digital diplomacy at the edge of the network

On Feb. 15, Secretary of State Hillary Clinton reaffirmed and refined the U.S. Department of State's "Internet freedom" policy. Nestled within her speech was an interesting wrinkle in the State Department's communications strategy: new Twitter accounts that move beyond English. In 2011, digital diplomacy is multilingual.

usa-bil-araby.jpg

On Feb. 17, a Spanish account (@USAenEspanol) and a Hindi account (@USAHindiMein) went online. On Feb. 18, the State Department's French account (@USAenFrancais) appeared. On Feb. 22, a Russian account (@USApoRusski) launched.

"The audiences off those tweets are important audiences," said Alec Ross, shortly before Clinton's speech on Internet freedom at George Washington University. "We need direct-to-people communications with native Farsi and Arabic speakers. There are events unfolding in Iran that we want to be able to communicate directly with the Iranian people about. This is a moment in time when we want to be able to speak directly to people in Arabic as well. This may be a new tactic, but the principles date back to the earliest parts of the Obama administration."

The accounts in Farsi and Arabic are run by people in the Middle East, said Ross. The accounts operate within the policy confines of the State Department, but they have some autonomy. There may be some rough edges yet, however, around customizing the language used in the tweets to connect to the massive "youth bulge" in the Middle East, where about 60% of the region's population is under 30.

Table for the moment relevant questions about whether the United States should support Internet freedom through technology. These new Twitter accounts represent both something old and something new. As new means for communication have become available throughout history, governments have harnessed them to broadcast to their citizens — and to the governments elsewhere in the world. In certain respects, Twitter, Facebook or other social media channels serve as a 21st-century complement to the more traditional broadcasts from Voice of America.

In an online chat that followed Secretary Clinton's speech, Ross elaborated on the use of these connection technologies. "Our use of social media is not for regime change, it's a communication mechanism," he said.

That mechanism has become increasingly relevant in areas where state-controlled media has limited information about what's happening. Twitter feeds in languages like Arabic and Farsi, "allow us to communicate in places where the nation state controls broadcast media; where they might control what you see on TV or hear on the radio," Ross said.

The combination of mobile phones, Al Jazeera, YouTube, Twitter and Facebook in Egypt and Tunisia created a new data nexus for governments and citizens alike. "We've become accustomed to a society where the vast number of people are connected, especially with cellphones," said Ben Scott, policy adviser for innovation at the State Department. "In a lot of countries, they're just starting to get to a critical mass. Many of the challenges we've been facing for years are now occurring in developing countries."

Whether the operators of these accounts are empowered to catalyze reciprocal feedback loops is another matter. Tapping social networking platforms to provide better intelligence for diplomats in Foggy Bottom or the White House will require more than broadcast behavior transplanted from the last century. Applying software to digital diplomacy is about much more than social media. It's about integrating communications across channels.

"We're not checking off what people tweet," said Ross. "We're empowering the edge of the network."

Fundamentally, moving to a distributed strategy for engaged communications would, in fact, be a better example of 21st century statecraft than the creation of any number of new Twitter accounts.

"A lot of the time, the intelligence in networks lives in the edge of the network," said Ross. "People who are in a foreign post may very well have better real-time optics into facts on the ground than people in an office in DC."

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl