Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

February 28 2012

The privacy arc

A while ago, I wrote a short post on the meaninglessness of frictionless sharing. Since then, I've had a few additional thoughts on what frictionless sharing is trying to accomplish (aside from pure and simple marketing), and what we should be trying to build.

The article about Target targeting pregnant women with advertisements caught my attention, not particularly because of Target's practice, but because it gives us a useful way of looking at the history of privacy. What Target did isn't at all surprising. Target's data systems noticed that some women were suddenly buying extra large handbags (for holding diapers), over-the-counter medicines that could be used to fight morning sickness, and skin creams to hide stretch marks. The store concluded that these women were probably pregnant and targeted them with ads featuring products for pregnant women. (If you believe the rather self-serving story about how one girl's father called the store furious about what these ads were implying, then called back the next day to apologize, you're less skeptical than I am.)

It's not surprising that this makes the news, but I asked myself what's really new here. And my answer is, "not much." Think back to the first half of the 20th century. A girl walks into the local pharmacy and buys bicarb for an upset stomach. The pharmacist notes that this girl has never bought anything like this before and also notes that she's looking a bit thicker. He has also seen the girl at the lunch counter and knows she has an iron stomach. He puts two and two together, makes a mental note, and knows what to recommend the next time she's in. And soon after the pharmacist knew it, you can bet that everyone knew it; people never needed the Internet to form networks. I would gladly bet that this story played itself out thousands of times.

What's interesting is what happened in the years that intervened between the '50s and the present. The small town culture (which may never have really existed) in which everyone knew everything about everyone disappeared as we moved into suburbs, where nobody knew anything about anyone. And that's really where our notions of "privacy" arose. The local pharmacies started disappearing, to be replaced by big chains like CVS and Walgreens. As Douden's and Jolly's disappeared from local culture, so did the local pharmacist who knew and remembered who you were and what you bought, and who was able to put two and two together without the help of a Hadoop cluster. Around 60-70 years ago, we didn't really have any privacy; Scott McNealy's infamous statement that "you have zero privacy anyway ... get over it" would have been meaningless. We grew attached to our privacy in the intervening half-century, as the demands of industry created population concentrations that broke the bonds (wanted or not) attaching us to our local neighbors. In the past, we "heard it through the grapevine," but by the time the Internet was invented, that grapevine had been uprooted.

I am the last person to claim that the '50s were some sort of paradise when all was right in America and the world. In many ways, the '50s were a sick and deformed conformist culture. But the '80s were no party either. I was in grad school at the time, and all the non-students I knew (mostly engineers in Silicon Valley) were bemoaning the lack of "community." They lived in anonymous apartment complexes in insipid suburbs; they were tired of the people they worked with; there was no good way to make friends, no good way to be social. The big social story of the '80s and '90s was the decline of "social" and the continued rise of suburban cocooning in detached houses. In this environment, the rise of Facebook and Foursquare (and MySpace, and Friendster, and Orkut and others) was inevitable. Given the boredom of mid-'80s apartment complex existence, software developers did what came naturally and invented a software solution.

We have to look at automated sharing of the music we listen to, the books we read, and the restaurants we visit in light of that arc. As anyone who is interested in books or records knows, the first thing you used to do when you visited someone's house was look at their bookshelves or their stack of records (or CDs). You might lend me a book or a record that I was interested in, moving a step up the ladder from acquaintance to intimacy. That still works, but at O'Reilly's recent TOC conference, it was clear that even publishers understand that the age of print is coming to the end. SOPA and PIPA have more to do with the entertainment industry realizing that CDs and DVDs have come to an end than they have to do with so-called piracy. Print books will survive as fetishized items, as will vinyl LPs: expensive coffee-table books for display, a few high-priced show editions, but nothing as interesting as what you'd find on my bookcase. That inevitable shift signals a profound change for the social nature of reading and listening. While looking through someone's bookshelves is fine, it's not socially acceptable to look through their iPods and Kindles.

In this context, it's surely correct to put a kinder interpretation on automated "frictionless sharing" of your songs and book purchases on Facebook. Yes, if someone is giving you a service for free, you're not the customer — you're the product. It's reasonable to be unhappy that your likes and dislikes are being bought and sold like pork bellies on the Chicago Merc. But there is an oddly pathetic humanity behind automated sharing: It's a clumsy and intrusive attempt to solve a very real human problem with technology. After all, that's what technologists do. Asking a software developer not to write software when faced with an obvious problem is like asking a fish not to swim. As I said, that's how we got Facebook in the first place.

Automated, frictionless sharing is certainly not a solution. As I've often observed, human problems are almost always solved by human solutions, very rarely by technical solutions. We have to ask ourselves what the real solution is, given that we've negotiated an arc from immersion in a social community (with all that entails) to helplessly private insularity to immersion in a virtual world that lacks privacy, but that also lacks human contact. It may be that dating sites are so consistently popular because they are the only online services that require human contact to work.

So how do we think about a solution? Privacy, data, and our social nature are inevitably entangled — always have been and always will be. How do we build satisfying human connections back into our lives without the superficiality and invasiveness of automated sharing? We've given up privacy without gaining the benefits of increased openness, which are tied up with social interaction. Back in the '80s, I couldn't look at your bookshelves unless you invited me to your party. That's real friction. Now, I can see your data, but even if you send me a personal email with your playlist, there's no party. And that's the challenge: bring real human connection back to our sanitized technology. The world isn't just about Facebook and Twitter, or even Google+. It's about making connections and having real parties with real food and real people. Gregory Brown, founder of Mendicant University, and one of the authors I've worked with, is having a party this Spring for "people with interesting ideas." I sure hope I'm invited because that's the only way out.

Photo: Soda fountain by LandVike, on Flickr


November 09 2011

Social network analysis isn't just for social networks

Social networking has become a pervasive part of our everyday online experience, and by extension, that means the analysis and application of social data is an essential component of business.

In the following interview, "Social Network Analysis for Startups" co-author Maksim Tsvetovat (@maksim2042) offers a primer on social network analysis (SNA) and how it has relevance beyond social-networking services.

What is social network analysis (SNA)?

Maksim Tsvetovat: Social network analysis is an offshoot of the social sciences — sociology, political science, psychology, anthropology and others — that studies human interactions by using graph-theoretic approaches rather then traditional statistics. It's a scientific methodology for data analysis and also a collection of theories about how and why people interact — and how these interaction patterns change and affect our lives as individuals or societies. The theories come from a variety of social sciences, but they are always backed up with mathematical ways of measuring if a specific theory is applicable to a specific set of data.

In the science world, the field is considered interdisciplinary, so gatherings draw mathematicians, physicists, computer scientists, sociologists, political scientists and even an occasional rock musician.

As far as the technology aspect goes, the analysis methods are embodied in a set of software tools, such as the Python-based NetworkX library, which the book uses extensively. These tools can be used for analyzing and visualizing network data in a variety of contexts, from visualizing the spread of disease to business intelligence applications.

In terms of marketing applications, there's plenty of science behind "why things go viral" — and the book goes briefly into it — but I find that it's best to leave marketing to marketing professionals.

Does SNA refer specifically to the major social-networking services, or does it also apply beyond them?

Maksim Tsvetovat: SNA refers to the study of relationships between people, companies, organizations, websites, etc. If we have a set of relationships that may be forming a meaningful pattern, we can use SNA methods to make sense of it.

Major social-networking services are a great source of data for SNA, and they present some very interesting questions — most recently, how can a social network act as an early warning system for natural disasters? I'm also intrigued by the emergent role of Twitter as a "common carrier" and aggregation technology for data from other media. However, the analysis methodology is applicable to many other data sources. In fact, I purposefully avoided using Twitter as a data source in the book — it's the obvious place to start and also a good place to get tunnel vision about the technology.

Instead, I concentrated on getting and analyzing data from other sources, including campaign finance, startup company funding rounds, international treaties, etc., to demonstrate the potential breadth of applications of this technology.

Social Network Analysis for Startups — Social network analysis (SNA) is a discipline that predates Facebook and Twitter by 30 years. Through expert SNA researchers, you'll learn concepts and techniques for recognizing patterns in social media, political groups, companies, cultural trends, and interpersonal networks.

Today Only Get "Social Network Analysis for Startups" for $9.99 (save 50%).

How does SNA relate to startups?

Maksim Tsvetovat: A lot of startups these days talk about social-this and social-that — and all of their activity can be measured and understood using SNA metrics. Being able to integrate SNA into their internal business intelligence toolkits should make businesses more attuned to their audiences.

I have personally worked with three startups that used SNA to fine-tune their social media targeting strategies by locating individuals and communities, and addressing them directly. Also, my methodologies have been used by a few large firms: the digital marketing agency DIGITAS is using SNA daily for a variety of high-profile clients. (Disclosure: my startup firm, DeepMile Networks, is involved in supplying SNA tools and services to DIGITAS and a number of others.)

What SNA shifts should developers watch for in the near future?

Maksim Tsvetovat: Multi-mode network analysis, which is analyzing networks with many types of "actors" (people, organizations, resources, governments, etc.). I approach the topic briefly in the book — but much remains to be done.

Also, watch for more real-time analysis. Most SNA is done on snapshot-style data that is, at best, a few hours out-of-date — some is years out-of-date. The release of Twitter's Storm tool should spur developers to make more SNA tools work on real-time and flowing data.

This interview was edited and condensed.

Associated photo on home and category pages: bulletin board [before there was twitter] by woodleywonderworks, on Flickr.


Reposted byRK RK
Sponsored post

November 08 2011

Radar is now on Google+ (officially this time)

O'Reilly Radar on Google Plus
Screenshot of Radar's Google+ page.

If you're a Google+ user and a Radar reader — I'm guessing there's a lot of overlap in that Venn diagram — you might be interested in following Radar's new Google+ page.

We're working out a gameplan for our Google+ coverage that extends and enhances what we do here on the Radar website. In addition to +-specific content, we'll also be surfacing relevant material from our YouTube archive and passing along intriguing content we encounter in our web travels.

The thing I find most interesting about Google+ is its friction-free engagement. The sharing and commenting functions are dead simple, so I anticipate mining those tools to make Radar's Google+ presence a strong two-way channel.

As you can gather, we'll be experimenting with our Google+ offerings quite a bit. If you have things you'd like to see, please chime in through the comments on this post or visit Radar's Google+ page and chat with us there.

(FYI — we've also set up Google+ pages for O'Reilly Media, O'Reilly Tools of Change for Publishing, and Ignite. More to come ...)


August 02 2011

Scaling Google+

Last week at OSCON, Google social web engineer Joseph Smarr sat down for an interview with me about Google+, the long-awaited social network that the search engine giant launched earlier this summer.

We covered a lot of ground during the interview. Smarr connected what he and others learned from Plaxo Pulse, where he was the CTO, to how the Circles tool in Google+ builds granular control into public and private sharing. He also said scalability comes in different flavors — it's not just about infrastructure, but rapidly scaling the user interface with feedback. Finally, we talked about the future of the Google+ platform and the possibilities of an API (more on that below).

When asked about what surprised him the most, Smarr pointed to the high rate of public sharing on Google+, versus how the social network had been used internally at Google before launch. "People are getting these incredibly high engagement discussions," he said.

The Google+ API

Smarr said the Google+ team is spending a lot of time thinking about an API, drawing from what they learned from Google Buzz and the experiences of other Internet companies building platforms.

"On the one hand, clearly the goal is to not just have another social network, to really help not only Google's products but to make the web in general more social, more open, more connected, and APIs are a crucial piece of that," said Smarr. "We actually got a far way along the road with the Buzz APIs, and not only having a lot of access to the activities and the graph and so forth, but with a lot of these modern standards, like pubsubhubbub and Webfinger and so forth. That's the style of thing we'd like to bring. "

The challenge for the Google+ team working on the API, as Smarr explained, is that the devil is in the details.

"One of things people seem to really like about Google+ right now is it's 100% authentic," Smarr said. "Every piece of content was created by a real person sitting in front of Google+ and deciding who to share with. Balancing the obvious need to get more content in and out from more sources while maintaining that authenticity is something that we're spending a lot of time playing and iterating and coming up with. I think you'll see things trickle out over time as we get bits and pieces we're happy with."

Web 2.0 Summit, being held October 17-19 in San Francisco, will examine "The Data Frame" — focusing on the impact of data in today's networked economy.

Save $300 on registration with the code RADAR

Google+ and the identity issue

Smarr also talked about the nature of identity on Google+. Those following the launch of the service know that Google+ and pseudonymity is a hot-button issue. As the Electronic Fronter Foundation's Jillian C. York noted in an essay making a case for pseudonyms, Google+ has changed some of its processes, moving from immediate account deactivation to warning users about the issue and giving them an opportunity to align their Google+ username with its "real name" policy. This week, Kaliya "@IdentityWoman" Hamlin became the most recent person to have her Google+ account suspended. (Given her work and role in the digital identity space, Hamlin's use case is likely to be an interesting one.)

"There's cases where that authenticity and knowing that this is a real person with whatever name they tend to be called in the real world is really a feature," Smarr said during our discussion. "It changes the tone of discussions, it helps you find people you know in the real world. And so, wanting to make sure that there's a space that is preserved and promoted is really important. On any of these social networks, it's not enough to write the code, you have to make the right community. Lots of networks choose different approaches to how they do that, and they all have different consequences. It's not that one is inherently better or or more valid than the others, it's just that if you don't do anything about it, it will kind of take its own course."

There are clearly some gray areas here, particularly given Google's global reach into parts of the world were using your real identity to share content could literally be life-threatening. "Obviously there are a lot of cases where being able to share things not under your real identity is valuable and necessary, and Google has a lot of products like this today, like YouTube," said Smarr. "If you're posting videos of authoritarian governments during a revolution, you may not want to use your real name, and that seems pretty valid. Whether or not that type of use case will be supported in the Google+ as you know it today is something that we're all thinking through and figuring out, but it's not meant to stop you from doing that in other products."

Smarr had one other comment on identity that goes to the difficultly of creating social networks in domains that may be hostile to free expression: "It's not just enough to offer the ability to post under a pseudonymous identifier. If you're going to make the commitment that we're not going to out your real identity, that actually takes a lot of work, especially if you're using your real account to log in and then posting under a pseudonym. We feel a real responsibility that if we're going to make the claim to people 'it's safe, you're not going to get outed,' then we really need to think through the architecture and make sure there aren't any loopholes where all of a sudden you get outed. That's actually a hard thing to do in software … we don't want to do it wrong, and so we'd rather wait until we get it right."

As I said at the end of the interview, if anyone is going to solve the challenge of enabling its users to securely and anonymously connect to its social network, Google would have to be near the top of the list. One potential direction might be further integrating the Tor Project and the Android operating system in the context of a Google+ API. What's clear now, however, is that if Google+ looks like a social backbone for the Internet, there's still a lot of growth ahead.


July 22 2011

Top stories: July 18-22, 2011

Here's a look at the top stories published across O'Reilly sites this week.

Google+ is the social backbone
Google+ is the rapidly growing seed of a web-wide social backbone, and the catalyst for the ultimate uniting of the social graph.
Intellectual property gone mad
Patent trolling could undermine app ecosystems, but who can mount a legitimate challenge? Here's four potential solutions.
Software engineering is a team sport: How programmers can deal with colleagues and non-programmers
Ben Collins-Sussman, tech lead and manager at Google, and Brian Fitzpatrick, engineering manager at Google, explain the "art of mass organizational manipulation."
FOSS isn't always the answer
James Turner says the notion that proprietary software is somehow dirty or a corruption of principles ignores the realities of competition, economics, and context.

Emerging languages show off programming's experimental side
Alex Payne, organizer of OSCON's Emerging Languages track, discusses language experimentation and whether these efforts are evolutionary or revolutionary.

Rugby photo: Scrum by MontyPython, on Flickr; Open sign photo: open by tinou bao, on Flickr

OSCON Java 2011, being held July 25-27 in Portland, Ore., is focused on open source technologies that make up the Java ecosystem. Save 20% on registration with the code OS11RAD

July 02 2011

Brief thoughts on Google Plus

Google Plus The blogosphere is already full with comments and reviews on Google Plus. I won't try to duplicate what other people have said.

I have one very quick reaction, though: I was an early Plus user, just as I was an early Wave user. And there's one big difference. Within hours, the early Plus invitees were actually using Google Plus for conversations. Not just nattering back and forth, not just comments on the service itself, but real conversations.

That's fundamentally different from Google Wave, where I rarely saw any discussion that wasn't about Wave itself. It's possible that's just the company I keep, but I think that's important. For a social network to succeed, people have to be social. And, while I saw loads of cool stuff on Wave, I never saw anything that was social. It was all about plugins, APIs, robots, outages, browser crashes, and the like. I never got into Buzz, so I don't have any basis for comparison. If you have thoughts, don't hesitate to add them to the comments.

I don't think Facebook is quaking in its boots; I do think (as @fredericl pointed out ) that Twitter may have more to fear than Facebook. I've participated in exactly one huddle on Google Plus: a 3-way video conversation with participants in the US and Europe. It was smooth, cool, easy. No echoes, no long delays, better audio than a cell phone. Skype may have more to fear than Facebook or Twitter. (I confess, I'm a long time Skype-disliker).

Google Plus is not without its warts, nor should we expect it to be; it's only been up a few hours. There is an undocumented way to invite arbitrary people. I won't tell you what it is, but it's not hard to discover, and that's important: a social service won't work until you have people to be social with. I wonder how well the "circle" metaphor scales; I already have more people in my circles than are easily visible, and I have more circles than fit on one screen. And I'd like an easier way to re-organize my circles, since I'm not sure my current organization works.

But in short, I think Google has a winner. Bugs and warts are easy to fix when the foundation is solid. Congratulations.

(Google's Joseph Smarr, a member of the Google+ team, will discuss the future of the social web at OSCON. Save 20% on registration with the code OS11RAD.)


April 06 2011

Amygdala FarmVille

Our Brain by perpetualplum, on FlickrYou may have received a recent email that started something like this:

We have been informed by one of our email service providers, Epsilon, that your email address was exposed by an unauthorized entry into that provider's computer system.

Did you rush immediately to Facebook or Twitter and ask "Who the heck is Epsilon?" I hope so because that would be super meta-ironic.

What you just discovered was what I hope will someday be known as Jim's First Law of Personal Data Privacy, which states: "The people that know the most about you are the people you know the least about."

The thing is, people have been trying to figure you out for a long time — long before you got your first email address or bought your first Michael Graves toaster at Target online. Many of the companies doing marketing services have roots that go back to before most of you reading this were born.

The difference is that in the old days they had to painstakingly amass their dossier on you one psychographically informative tidbit at a time. It might have taken a decade or more, but eventually by combining data from a bunch of the places where you did business they could more or less figure you out. Having accomplished that feat of data hoarding and analysis, they might make you 5% more likely to open a piece of direct mail, or discover that people that vote for third-party candidates might also respond to Hummer ads — and these companies would get paid handsomely for that information.

These databases were the crown jewels. If FASB accounting rules were rational, these information stores would appear on the books the way GM lists plant equipment. And they were big data before people were using the term "big data." These folks were prefixing with peta- while the rest of us were still getting our heads around tera.

So I can only imagine the reaction in the boardrooms of those traditional firms when Facebook and Google built their Psychographic Marketing Honeypots and disguised them as a social network and a search engine. "All that data we've worked so hard to source! Merde! People just sit there all day giving it to them!"

And the best part? Need a new field to feed your new and better algorithm? Don't spend months trying to source it from the U.S. Census or a credit card company or, even worse, merge with a frequent flyer program or a phone book to get it. It's way easier than that. Just add a field to the user profile page and they'll fill it out for you!

I wonder when it occurred to Mark Zuckerberg that he was like a casino operator and was building two companies. The part full of bad carpet and distractions that you think you're doing business with and the part you can't see behind the scenes that does, well, other stuff. Do you think it was part of his plan from the beginning, or was it sort of a mid-stage epiphany? Never mind, don't tell me, I'm waiting for "The Social Network" to be free on Netflix. And I don't want any spoilers in the comments, please.

The world has changed though, hasn't it? We have entered the Matrix, but it's not our body heat they want. They want the preference model encoded in our amygdala and a list of all the people that might influence that model tomorrow.

At some level, the relationship between Facebook and one of their advertisers isn't all that different from any other marketing services firm and a company like General Motors. But the way we participate in generating our own profile while we think we're doing something else is fascinating.

I think most people think their bargain with Facebook is like the one they had with broadcast television. I sit here a few hours a day sidestepping drudgery and you feed me ads. But as you know (or know now), that's not it at all. And it's definitely not just for those ads being served on the site.

The bargain we make collectively with "the web" might be Faustian if it were in fact a bargain between two parties. After all, what we are actually doing is trading our most intimate selves into the ether and in return we get creepily prescient ads for erectile dysfunction medication and whatnot. But our bargain is too often an implicit one with the back room we don't know exists. Faust at least knew the terms of his agreement.

Here's what you need to know: Your mind is advanced enough to experience a self, a self that you think has intrinsic value. But that's just a construction in your head. Your actual extrinsic value, I'm sorry to say, is just the sum of your known behaviors and the predictive model they make possible. The stuff you think of as "your data" and the web thinks of as "our data about you — read the ToS," is the grist for that mill. And Facebook's shiny front room is just a place for you to behave promiscuously and observably. While you're farming, well, fake carrots or something, they are farming your amygdala.

Illustration: Our Brain by perpetualplum, on Flickr


February 17 2011

Four short links: 17 February 2011

  1. The True Cost of Publishing on the Kindle -- an article, apparently by a horrified negotiator with Amazon, revealing that magazine and newspaper publishers pay the WhisperNet delivery costs of their editions. That's not Amazon overhead, it comes out of the publisher's royalty slice. (via Hacker News)
  2. Fonts in Use -- examples of sweet typography and the fonts that were used.
  3. Ffffound -- social network for graphic designers (invite only) with a "people who liked also liked" type of recommendation system. Very clever. So as you research "I want to build a cheesy 70s logo", you thumbs up the images you like and soon the system is suggesting designs with elements of cheesy 70s logos to you. I love that it is invitation-only: you're trusting the judgement of the other people, so you had better only let in people whose judgement you trust.
  4. China's Second Wives and Gift Culture -- second wives, status, and brand. But any city that has a middle class is going to have Second Wives. [...] Even Jiang Zemin, the former President, had a very high profile mistress - a singer called Song Zuying who appears on the Chinese New Year programme every year. And it's not a scandal. A reminder that if you think you can export your crappy business built on American status symbols, you're leaping into the Sea of Fail. (via Sciblogs)

December 28 2010

Four short links: 28 December 2010

  1. Amazon Sold 158 Items/Second on Cyber Monday (TechCrunch) -- I remember when 20 hits/s on a Sun web server was considered pretty friggin' amazing. Just pause a moment and ponder the infrastructure Amazon has marshaled to be able to do this: data centers, replication, load balancers, payment processing, fulfillment, elastic cloud computing, storage servers, cheap power, bandwidth beyond comprehension.
  2. Quick Thoughts on Pinboard (Matt Haughey) -- thoughtful comments, and an immediate and just as thoughtful response. (I am a happy pinboard user who is also looking forward to the social networking features to come)
  3. Female Founders -- impressively long list of female startup founders. (via Hacker News)
  4. Less Framework -- cross-device css grid system based on using inline media queries. (via Pinboard)

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
No Soup for you

Don't be the product, buy the product!

YES, I want to SOUP ●UP for ...