Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

January 02 2014

The Snapchat Leak

The number of Snapchat users by area code.The number of Snapchat users by area code.

The number of Snapchat users by geographic location. Users are predominately located in New York, San Francisco and the surrounding greater New York and Bay Areas.

While the site crumbled quickly under the weight of so many people trying to get to the leaked data—and has now been suspended—there isn’t really such a thing as putting the genie back in the bottle on the Internet.

Just before Christmas the Australian based Gibson Security published a report highlighting two exploits in the Snapchat API claiming that hackers could easily gain access to users’ personal data. Snapchat dismissed the report, responding that,

Theoretically, if someone were able to upload a huge set of phone numbers, like every number in an area code, or every possible number in the U.S., they could create a database of the results and match usernames to phone numbers that way.

Adding that they had various “safeguards” in place to make it difficult to do that. However it seems likely that—despite being explicitly mentioned in the initial report four months previously—none of these safeguards included rate limiting requests to their server, because someone seems to have taken them up on their offer.

Data Release

Earlier today the creators of the now defunct SnapchatDB site released 4.6 million records—both as an SQL dump and as a CSV file. With an estimated 8 million users (May, 2013) of the app this represents around half the Snapchat user base.

Each record consists of a Snapchat user name, a geographical location for the user, and partially anonymised phone number—the last two digits of the phone number having been obscured.

While Gibson Security’s find_friends exploit has been patched by Snapchat, minor variations on the exploit are reported to still function, and if this data did come from the exploit—or a minor variation on it—uncovered by Gibson, then the dataset published by SnapchatDB is only part of the data the hackers now hold.

In addition to the data already released they would have the full phone number of each user, and as well as the user name they should also have the—perhaps more revealing—screen name.

Data Analysis

Taking an initial look at the data, there are no international numbers in the leaked database. All entries are US numbers, with the bulk of the users from—as you might expect—the greater New York, San Francisco and Bay areas.

However I’d assume that the absence of international numbers is  an indication of laziness rather than due to any technical limitation. For US based hackers it would be easy to iterate rapidly through the fairly predictable US number space, while foreign” numbers formats might present more of a challenge when writing a script to exploit the hole in Snapchat’s security.

Only 76 of the 322 area codes in the United States appear in the leaked database, alongside another two Canadian area codes, mapping to 67 discrete geographic locations—although not all the area codes and locations match suggesting that perhaps the locations aren’t derived directly from the area code data.

Despite some initial scepticism about the provenance of the data I’ve confirmed that this is a real data set. A quick trawl through the data has got multiple hits amongst my own friend group, including some I didn’t know were on Snapchat—sorry guys.

Since the last two digits were obscured in the leaked dataset are obscured the partial phone number string might—and frequently does—generate multiple matches amongst the 4.6 million records against a comparison number.

I compared the several hundred US phone numbers amongst my own contacts against the database—you might want to do that yourself—and generated several spurious hits where the returned user names didn’t really seem to map in any way to my contact. That said, as I already mentioned, I found several of my own friends amongst the leaked records, although I only knew it was them for sure because I knew both their phone number and typical choices of user names.


As it stands therefore this data release is not—yet—critical, although it is certainly concerning, and for some individuals it might well be unfortunate. However if the SnapchatDB creators choose to release their full dataset things might well get a lot more interesting.

If the full data set was released to the public, or obtained by a malicious third party, then the username, geographic location, phone number, and screen name—which might, for a lot of people, be their actual full name—would be available.

This eventuality would be bad enough. However taking this data and cross-correlating it with another large corpus of data, say from Twitter or Gravatar, by trying to find matching user or real names on those services—people tend to reuse usernames on multiple services after all—you might end up with a much larger aggregated data set including email addresses, photographs, and personal information.

While there would be enough false positives—if matching solely against user names—that you’d have a interesting data cleaning task afterwards, it wouldn’t be impossible. Possibly not even that difficult.

I’m not interested in doing that correlation myself. But others will.


November 28 2013

Sponsored post

October 18 2013

Four short links: 18 October 2013

  1. Science Not as Self-Correcting As It Thinks (Economist) — REALLY good discussion of the shortcomings in statistical practice by scientists, peer-review failures, and the complexities of experimental procedure and fuzziness of what reproducibility might actually mean.
  2. Reproducibility Initiative Receives Grant to Validate Landmark Cancer StudiesThe key experimental findings from each cancer study will be replicated by experts from the Science Exchange network according to best practices for replication established by the Center for Open Science through the Center’s Open Science Framework, and the impact of the replications will be tracked on Mendeley’s research analytics platform. All of the ultimate publications and data will be freely available online, providing the first publicly available complete dataset of replicated biomedical research and representing a major advancement in the study of reproducibility of research.
  3. $20 SDR Police Scanner — using software-defined radio to listen to the police band.
  4. Reimagine the Chemistry Set — $50k prize in contest to design a “chemistry set” type kit that will engage kids as young as 8 and inspire people who are 88. We’re looking for ideas that encourage kids to explore, create, build and question. We’re looking for ideas that honor kids’ curiosity about how things work. Backed by the Moore Foundation and Society for Science and the Public.

August 21 2013

"But what agencies like the FBI are now worried about is that individuals are « going dark » by using…

"But what agencies like the FBI are now worried about is that individuals are “going dark” by using freely available encryption software to prevent their email and phone conversations to be captured by law enforcement agencies. [...]

Bimen Associates, which has its headquarters in McLean, Virginia, near the headquarters of the Central Intelligence Agency, provided custom designed software tools developed exclusively for the FBI to crack encrypted conversations, says Soghoian. Agency staff and contractors access computers of suspects remotely to install this software to allow them to watch everything that the target types or says.

#nsa #surveillance #hacking #FBI #blackhat

August 12 2013

DEF CON : The Documentary - Jason Scott

DEF CON : The Documentary - Jason Scott

DEF CON is the world’s largest hacking conference, held in Las Vegas, Nevada. In 2012 it was held for the 20th time. The conference has strict no-filming policies, but for DEF CON 20, a documentary crew was allowed full access to the event. The film follows the four days of the conference, the events and people (attendees and staff), and covers history and philosophy behind DEF CON’s success and unique experience. Written by Jason Scott

où l’on voit le général Alexander

voir aussi cet autre #film #documentaire de Jason Scott :

#hacking #convention #silicon_army

May 08 2013

Four short links: 8 May 2013

  1. How to Build a Working Digital Computer Out of Paperclips (Evil Mad Scientist) — from a 1967 popular science book showing how to build everything from parts that you might find at a hardware store: items like paper clips, little light bulbs, thread spools, wire, screws, and switches (that can optionally be made from paper clips).
  2. Moloch (Github) — an open source, large scale IPv4 packet capturing (PCAP), indexing and database system with a simple web GUI.
  3. Offline Wikipedia Reader (Amazon) — genius, because what Wikipedia needed to be successful was to be read-only. (via BoingBoing)
  4. Storing and Publishing Sensor Data — rundown of apps and sites for sensor data. (via Pete Warden)

April 26 2013

Glowing Plants

I just invested in BioCurious’ Glowing Plants project on Kickstarter. I don’t watch Kickstarter closely, but this is about as fast as I’ve ever seen a project get funded. It went live on Wednesday; in the afternoon, I was backer #170 (more or less), but could see the number of backers ticking upwards constantly as I watched. It was fully funded for $65,000 Thursday; and now sits at 1340 backers (more by the time you read this), with about $84,000 in funding. And there’s a new “stretch” goal: if they make $400,000, they will work on bigger plants, and attempt to create a glowing rose.

Glowing plants are a curiosity; I don’t take seriously the idea that trees will be an alternative to streetlights any time in the near future. But that’s not the point. What’s exciting is that an important and serious biology project can take place in a biohacking lab, rather than in a university or an industrial facility. It’s exciting that this project could potentially become a business; I’m sure there’s a boutique market for glowing roses and living nightlights, if not for biological street lighting. And it’s exciting that we can make new things out of biological parts.

In a conversation last year, Drew Endy said that he wanted synthetic biology to “stay weird,” and that if in ten years, all we had accomplished was create bacteria that made oil from cellulose, we will have failed. Glowing plants are weird. And beautiful. Take a look at their project, fund it, and be the first on your block to have a self-illuminating garden.

October 03 2012

Biohacking: The next great wave of innovation

Genspace and Biocurious logosGenspace and Biocurious logosI’ve been following synthetic biology for the past year or so, and we’re about to see some big changes. Synthetic bio seems to be now where the computer industry was in the late 1970s: still nascent, but about to explode. The hacker culture that drove the development of the personal computer, and that continues to drive technical progress, is forming anew among biohackers.

Computers certainly existed in the ’60s and ’70s, but they were rare, and operated by “professionals” rather than enthusiasts. But an important change took place in the mid-’70s: computing became the domain of amateurs and hobbyists. I read recently that the personal computer revolution started when Steve Wozniak built his own computer in 1975. That’s not quite true, though. Woz was certainly a key player, but he was also part of a club. More important, Silicon Valley’s Homebrew Computer Club wasn’t the only one. At roughly the same time, a friend of mine was building his own computer in a dorm room. And hundreds of people, scattered throughout the U.S. and the rest of the world, were doing the same thing. The revolution wasn’t the result of one person: it was the result of many, all moving in the same direction.

Biohacking has the same kind of momentum. It is breaking out of the confines of academia and research laboratories. There are two significant biohacking hackerspaces in the U.S., GenSpace in New York and BioCurious in California, and more are getting started. Making glowing bacteria (the biological equivalent of “Hello, World!”) is on the curriculum in high school AP bio classes. iGem is an annual competition to build “biological robots.” A grassroots biohacking community is developing, much as it did in computing. That community is transforming biology from a purely professional activity, requiring lab coats, expensive equipment, and other accoutrements, to something that hobbyists and artists can do.

As part of this transformation, the community is navigating the transition from extremely low-level tools to higher-level constructs that are easier to work with. When I first leaned to program on a PDP-8, you had to start the computer by loading a sequence of 13 binary numbers through switches on the front panel. Early microcomputers weren’t much better, but by the time of the first Apples, things had changed. DNA is similar to machine language (except it’s in base four, rather than binary), and in principle hacking DNA isn’t much different from hacking machine code. But synthetic biologists are currently working on the notion of “standard biological parts,” or genetic sequences that enable a cell to perform certain standardized tasks. Standardized parts will give practitioners the ability to work in a “higher level language.” In short, synthetic biology is going through the same transition in usability that computing saw in the ’70s and ’80s.

Alongside this increase in usability, we’re seeing a drop in price, just as in the computer market. Computers cost serious money in the early ’70s, but the price plummeted, in part because of hobbyists: seminal machines like the Apple II, the TRS-80, and the early Macintosh would never have existed if not to serve the needs of hobbyists. Right now, setting up a biology lab is expensive; but we’re seeing the price drop quickly, as biohackers figure out clever ways to make inexpensive tools, such as the DremelFuge, and learn how to scrounge for used equipment.

And we’re also seeing an explosion in entrepreneurial activity. Just as the Homebrew Computer Club and other garage hackers led to Apple and Microsoft, the biohacker culture is full of similarly ambitious startups, working out of hackerspaces. It’s entirely possible that the next great wave of entrepreneurs will be biologists, not programmers.

What are the goals of synthetic biology? There are plenty of problems, from the industrial to the medical, that need to be solved. Drew Endy told me how one of the first results from synthetic biology, the creation of soap that would be effective in cold water, reduced the energy requirements of the U.S. by 10%. The holy grail in biofuels is bacteria that can digest cellulose (essentially, the leaves and stems of any plant) and produce biodiesel. That seems achievable. Can we create bacteria that would live in a diabetic’s intestines and produce insulin? Certainly.

But industrial applications aren’t the most interesting problems waiting to be solved. Endy is concerned that, if synthetic bio is dominated by a corporate agenda, it will cease to be “weird,” and won’t ask the more interesting questions. One Synthetic Aesthetics project made cheeses from microbes that were cultured from the bodies of people in the synthetic biology community. Christian Bok has inserted poetry into a microbe’s DNA. These are the projects we’ll miss if the agenda of synthetic biology is defined by business interests. And these are, in many ways, the most important projects, the ones that will teach us more about how biology works, and the ones that will teach us more about our own creativity.

The last 40 years of computing have proven what a hacker culture can accomplish. We’re about to see that again, this time in biology. And, while we have no idea what the results will be, it’s safe to predict that the coming revolution in biology will radically change the way we live — at least as radically as the computer revolution. It’s going to be an interesting and exciting ride.


April 18 2012


April 06 2012

Announcing Make's Hardware Innovation Workshop

Hardware Innovation Workshop

The maker movement is a remarkable new source of innovation. We are starting to see what results from a powerful combination of open hardware + personal fabrication tools + connected makers. Sometimes this innovation is hard to identify in the excitement that surrounds Maker Faire. Yet at Maker Faire, you can find new products and new startups at various stages of development that you will see almost nowhere else. Business people tell me they come to Maker Faire expecting to have a good time with their family but unexpectedly walk away impressed by the creativity and innovation they find there. As the song says, "there's something happening here." Even now, the pace of development is quickening and the number of hardware startups is rapidly growing.

Tim O'Reilly has been urging that the opportunity is now to showcase makers as professionals who are starting new businesses and developing new products. So, I'm happy to announce a new business conference during the week of Maker Faire, taking advantage of the makers who are already coming to Maker Faire. Presented by Make, the Hardware Innovation Workshop will be held Tuesday and Wednesday, May 15-16, at Xerox PARC in Palo Alto, California. (I'm excited to have PARC host us and this event because of its long history as a source of technology innovation.)

The Hardware Innovation Conference will present a number of hardware-related startups and review the major platforms and the new toolset for prototyping and personal fabrication. It's an intimate setting to meet the leaders of the maker movement and understand how makers are changing the technology landscape, in much the same way that enthusiasts once helped to create the personal computer industry.

Our presenters will include:

  • Massimo Banzi of Arduino, an Italian interaction designer and engineer who created this open source micro controller. The Arduino platform has become the Linux of open source hardware and it is found at the heart of many maker projects.

  • Carl Bass of Autodesk, a maker himself whose new consumer division, which acquired Instructables, is exploring the software and services needed by this emerging maker market.

  • Jay Rogers of Local Motors is creating an open source car through collaborative design and he's built a micro factory for assembly of these cars by the owners themselves.
  • Ayah Bdeir of Little Bits is one of those non-traditional product designers who has developed a new educational product.
  • Allan Chochinov of Core 77 is starting a new program called Products of Design at the School of Visual Arts in NYC, which is reshaping product design around what makers are able to do.
  • Nathan Seidle of Sparkfun Electronics runs one of the major suppliers for maker projects. He's also a partner for makers who have the idea but not the factory to build a new product.
  • Bre Pettis of MakerBot will explore the 3D printing opportunity in consumer markets. MakerBot is the Apple II of the personal fabrication revolution. Brad Feld of Foundry Group will tell us why he's invested in Makerbot.
  • Mark Hatch of TechShop, whose membership model for a community workshop has become a hub for hardware innovators. i>
  • Bunnie Huang of Chumby and author of "Hacking the Xbox," who understands how Asia's manufacturing capacity might be tapped by makers.

  • Check our event website for full program details.

    The lesson for us from makers is that hardware isn't as hard as it used to be. It's benefiting from the same forces that allowed open source to reshape the software industry and create the web economy. Makers are part of a prototyping revolution that is inviting a new audience to design and develop products. Open technologies and new collaborative processes just might change the face of manufacturing by making it much more personal and more automated. Unlike traditional manufacturers, makers are able to pivot easily to serve niche markets. In addition, larger companies are hiring makers and maker advocates to infuse their own teams with creative ideas and keep track of these new market opportunities.

    The conventional wisdom is that Silicon Valley investors don't like hardware startups, but that's not stopping makers. We even see hardware startups raising capital from non-traditional sources such as Kickstarter. (Twine raised over $850,000.) This is causing some investors to pay attention. As an angel investor said to me recently: "Everybody's just looking at mobile/social. I want to look at things outside that well-developed space and that's why I'm looking at makers."

    Please join me along with Tim O'Reilly and the creative team of Make Magazine and Maker Faire for a program focused on maker-led innovation at a historic location in the Silicon Valley. Due to the venue, we are limited to 300 participants. If you're coming from outside the Bay Area, you can stay for the weekend of fun at Maker Faire, May 19-20th.

    Event: Hardware Innovation Workshop
    Dates: May 15-16
    Location: Xerox PARC, Palo Alto, CA

    November 03 2011

    Developer Week in Review: The hijacking of an insulin pump

    A future batch of kindlingIt was a great week at the Turner household! Although we love our house, we've frequently said to each other, "You know what we could really use? A 25-foot-long tree limb wrapped in power lines blocking our driveway." Well, this weekend mother nature decided to help us fill this void in our landscaping, and threw in some ornamental cherry firewood as well (chainsawing not included). Thankfully, I spent the extra bucks on Saturday to get our LPG tank topped off, so I've got generator power for 10-14 days. Given we're on day four with no power in sight, that was a good decision.

    It could have been worse, of course. For example ...

    A scene from an upcoming technothriller

    Plucky researcher Ann McManna walked across the room toward the podium, ready to reveal the details of the fiendish plot she had uncovered to the waiting reporters. Now the world would know about the conspiracy to corner the world supply of macadamia nuts. Her heart pounded with excitement, her mouth was dry and she perspired, in spite of the air conditioning that was making the room practically an ice box. As she approached the stage, she bumped against a table, stumbling and suddenly having trouble seeing her path through blurry eyes. Something was wrong, but she couldn't focus, couldn't identify what was happening to her, even as she collapsed to the ground. Minutes later, the paramedics would close the eyelids of her corpse.

    Some fanciful invention of Tom Clancy or Robin Cook? Not anymore, thanks to research by McAfee's Barnaby Jack, presented at this year's Hacker Halted conference. Using some custom software and a special antenna, Jack was able to control Medtronic insulin pumps as far as 300 feet from the controller. He was able to disable the tones that warn a user that insulin is being pumped, and trigger a 25-unit bolus of insulin. In some circumstances, this could kill a victim.

    As networked computers appear in more life-critical items, this is a good reminder that security should be job No. 1, not something to think about if you have time. Too many proprietary device manufacturers seem to depend on security through obscurity, rather than security in depth.

    Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

    Save 20% on registration with the code RADAR20

    The first taste is free, but you'll be back

    One of the perils of depending on public APIs from for-profit companies is that they may get turned into a profit center down the road. Users of the Google Maps API learned that lesson recently, as Google announced that high-volume users will no longer have free access to the APIs starting next year. Before you start panicking, the definition of high-volume will be more than 25,000 calls a day (2,500 if you use the custom styling features), and the rate over 25,000 is $4/1,000 calls. Google claims that less than 1% of all users will run up against this limit.

    The problem with using beta or "free" services in your products is that, unless the terms of use specifically say that it will be free forever, you have no contractual agreement to lean on, and the provider is able at any point to change how (or even if) the service is provided.

    Linus Torvalds vs. C++

    Linux progenitor Linus Torvalds has a reputation for diplomacy and fence building — that's practically the only way to herd the stampede of cats that is the Linux developer community. But when he gets upset, the results can peel the paint off the walls.

    We got a good example this week, as Torvalds responded to a complaint about the fact that the git source control system was written in pure C, rather than C++. In a nutshell, Torvalds called C++ a lousy language that attracts substandard programmers and leads to sloppy, unmaintainable code. In general, I tend to take any blanket condemnation of a programming language as hyperbole, but Torvalds seems to genuinely loathe C++. We'll have to see if his anger against the language alienates any of the kernel developer base, or if people will just shrug it off as Linus being Linus.

    Got news?

    Please send tips and leads here.


    September 29 2011

    Developer Week in Review: Android proves fruitful for Microsoft

    The ball has finally dropped at Apple, and we know that October 4 is the big day that iOS 5 and some undisclosed subset of iPhone devices will be unveiled. Oddly, developers still haven't received the Gold Master of iOS 5, which means that Apple is cutting things close if it wants to give people time to update apps in the store, not to mention those of us who have to revise books once the NDA lifts on iOS 5.

    So, while we wait for Godot Tim Cook, let's see what other mischief is afoot.

    Royalties for Redmond

    As we've reported previously, one of the big winners in the growth of Android has been Microsoft, as phone manufactures have been lining up to pay royalties to Redmond to avoid patent lawsuits. Samsung joined the fray this week, agreeing to pony up a reported $5 per phone to stay out of court.

    In light of this, Google's purchase of Motorola Mobility is looking less and less wise. The widely held view was that the sale was intended to shield Android-based phones behind Motorola's rich patent portfolio, but every major player is caving into Microsoft anyway.

    Between the squeeze play on Android and the long-standing siphoning of Linux revenues from companies such as Novell, Microsoft seems to be following a business plan reminiscent of a certain Monty Python sketch.

    Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders.

    Save 20% on registration with the code AN11RAD

    SPARC? Oh yeah, I remember that ...

    SPARC T4Back when dinosaurs ruled the earth, the absolutely hottest thing you could have on your desk was a Sun-4. The SPARC-based systems were leaps and bounds ahead of anything else in their price range, except perhaps for some esoteric hardware from Silicon Graphics (remember them?)

    Time has not been kind to the SPARC, alas. Sun's hardware market share shrank as people discovered that Linux on cheap hardware could give a better bang for the buck, and the entire venture was eventually swallowed by Oracle. The conventional wisdom was that Oracle bought Sun largely for its hardware line, and there was some confirmation of that this week. While much of the rest of Sun's holdings have been left to languish or spun off entirely, Larry's gang has evidently been busy with hardware. The SPARC T4 is the result.

    The problem is, while the T4 brings some modern features like out-of-order execution to the SPARC line, these are things that other processor families have had for a decade or more. While it may staunch the flow of former SPARC customers defecting to x86 systems, it's unlikely to gain many new converts. And as any Harvard MBA can tell you, a business model based on not losing existing customers is not a formula for success in the long term.

    Might want to rethink those voting machines (and the people who use them)

    We've been hearing for years that direct recording electronic voting machines are potentially hackable. With a powder-keg election forthcoming, it was therefore not reassuring news this week that researchers at Argonne National Laboratory were able to totally subvert the voting counts on Diebold voting machines, simply by installing a $10 circuit between a ribbon cable and the connector. Since Diebold machines are not tamper resistant, this means that pretty much anyone with the technical savvy to create the device could hijack the polls.

    I see this as part of a larger problem in the computer industry — an almost blind belief that technology can solve social problems in isolation. People seem to think that making government data transparent or turning to social networking can solve society's ills. In reality, the things that need to be re-engineered are the people. The best software in the world won't make people give up irrational belief systems, or stop hating others (be they red state or blue) because they're different. And as long as hate, intolerance and ignorance run wild, technology will be as likely to be used as a weapon as a solution.

    Got news?

    Please send tips and leads here.


    September 28 2011

    Fighting the next mobile war

    It's arguable that with the arrival of touch displays, the current form factor for the smartphone is going to be with us for some time to come. You can't get much simpler than a solid block of glass and aluminum with a button. Unless you remove the button. Thinking about it, that's probably a solid suggestion — I'd look for that next.

    If things aren't going to change very much on the surface, underneath the glass things might not be much different either. Oh, the devices will be faster, and they'll have more cores, better displays, faster network connections, and the batteries will last longer. But fundamentally, they'll still be the same. The device won't provide you with any new levers on the world. With the exception of NFC, which admittedly is a big exception, there are no new sensory modalities on the horizon that are likely to be integrated into handsets. You'll interact with your smartphone tomorrow in much the same way you interact with it today, at least in the near term.

    That said, it's quite possible that your smartphone will interact with the world in a very different way. That's because the next mobile war has already begun, and you've seen nothing yet.

    The phoney war

    It began quietly, with little noise or fanfare, just over two years ago with Apple's announcement of iOS 3, the External Accessory Framework, and the opportunity for partners in the MFi program to build external hardware that connected directly to the iPhone.

    For the first time, it was easy, at least for certain values of easy, to build sensor hardware that connected to a mass-market mobile device. And for the first time, the mobile device had enough computing power and screen real estate to do something interesting with the sensor data.

    Except of course, it wasn't easy. While initially the External Accessory Framework was seen as having the potential to open up Apple's platform to a host of external hardware and sensors, little of the innovation people were expecting actually occurred. Much of the blame was laid squarely at the feet of Apple's own MFi program.

    There was some headway made using the devices as sensor gateways, mainly in the medical community, which Apple had initially pushed heavily during the launch. But in the end, the framework was used to support a fairly predictable range of audio and video accessories from big-name manufacturers — although more recently there have been a few notable exceptions.

    Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders.

    Save 20% on registration with the code AN11RAD

    Opening a second front

    Things stayed quiet until earlier this year when Google announced the Android Accessory Development Kit (ADK) at Google I/O in May.

    While there was a lot of criticism of Google's approach, it was justifiably hailed as a disruptive move by Google in what had become a fairly stagnant accessories market. Philip Torrone hit the right note when he speculated that this might mean the end of Apple's restrictive MFi program.

    I've talked about the Arduino here before. It allows rapid, cheap prototyping for embedded systems. Making Android the default platform for development of novel hardware was a brilliant move by Google. Maybe just a little too brilliant.

    The counterattack by Apple

    Around the middle of the year, right in the middle of Apple's WWDC conference, I was approached by Redpark and sworn to secrecy. Apple was on the brink of approving a serial cable for iOS that they would let Redpark sell into the hobbyist market.

    I'd known about the existence of the cable since the preceding November with the release of the SkyWire telescope control kit. I'd begged Redpark for developer access to their cable, and after signing a thick stack of NDAs, I got my hands on one around mid-December. At the time there seemed little chance of Apple ever approving the cable except for specific use cases where the cable and an accompanying iOS application were approved together as part of the MFi program — exactly as Apple had for Skywire for telescopes and Cisco had for networking gear.

    The news that the cable might soon be generally available to hobbyists was surprising. Despite Apple's beginnings — and the large community of indie developers surrounding its products — the hobbyist market isn't something Apple is known for caring about these days. Quite the opposite: Apple is notorious for keeping its products as closed as possible.

    Controlling an Arduino with an iPhone.

    Close on the heels of Google's ADK announcement, Apple's sudden enthusiasm was suspiciously timed. Someone high up at Apple had obviously realized the disruptive nature of the ADK and this was their response, their counter-attack. Despite the Android ADK actually being an Arduino, it was now easier to talk to an Arduino from iOS using Redpark's cable than it was to talk to an Arduino from Android.

    The long war

    The Android ADK board is only now appearing in large numbers as the open hardware community gears up to produce compatible boards cheaper than Google's ruinously expensive initial batch of "official" developer boards. The Redpark cable also faced supply issues, with the initial production run selling out on the Maker Shed within a few days. We're only now seeing it in larger volumes. So, despite appearances, it's still the early days.

    Discussing the Redpark cable at OSCON 2011.

    I think the availability of both these products is going to prove to be amazingly disruptive in the longer term. After spending two days at the recent World Maker Faire in New York, I know there's a lot of enthusiasm inside the Maker community for that disruption — and Apple may have the edge.

    Because of Apple's policy restrictions, you can only develop applications that work with Redpark's cable for your own personal use or for distribution inside an enterprise environment without going through the MFi program. The ease of use and popularity of the iOS platform with developers means there will still be a big uptake, and after a few people struggle through the process, I think that, with time, the cable will spell the end of the MFi program.

    Over the next couple of years, we'll be seeing some real innovation in the external accessory product space. Rapid prototyping combined with ease of access to increasingly powerful mobile platforms means that the next mobile war, and the next big thing of a real ubiquitous computing environment, is just around the corner.


    September 09 2011

    Top Stories: September 5-9, 2011

    Here's a look at the top stories published across O'Reilly sites this week.

    The new guy wants to hack the city's data
    Instead of quietly settling in like most new residents, Tyler, Texas, transplant Christopher Groskopf is on a mission to find and unlock his new city's datasets.

    RIP Michael S. Hart
    Michael Hart was the founder of Project Gutenberg, an incredible visionary for online books, and someone who played an important role in Nat Torkington's life.

    Look at Cook sets a high bar for open government data visualizations
    One of the best recent efforts at visualizing open government data can be found at, which tracks government budgets and expenditures from 1993-2011 in Cook County, Illinois.

    Master a new skill? Here's your badge
    The Mozilla Foundation's Erin Knight talks about how the badges and open framework of the Open Badge Project could change what "counts" as learning.

    The boffins and the luvvies
    Whether we're discussing ancients versus moderns, scientists versus poets, or the latest variant — computer science versus humanities, the debate between science and art is persistent and quite old.

    Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively. Save 30% on registration with the code ORM30.

    August 27 2011

    July 07 2011

    3 Android predictions: In your home, in your clothes, in your car

    In advance of his upcoming webcast and the Android Open conference, I asked "Learning Android" author Marko Gargenta to weigh in on Android's future. Below he offers three predictions that focus on Android's expansion beyond mobile devices.

    Prediction 1: Android controls the home

    Marko GargentaMarko Gargenta: Google painted their vision of Android @ Home at the last Google I/O. I think this has huge potential to make Android the de-facto controller for many other devices, from lights to music players to robots and factory machinery. We are seeing the first stage with numerous home security systems being developed using Android, as well as set-top boxes powered by Android. At the moment, many of these devices simply use Android as a replacement for embedded Linux and they're still just self-contained devices.

    In the second stage, manufacturers will start exposing libraries so developers can build custom applications for their devices, effectively turning them into platforms. I predict this will happen later this year as manufacturers realize the power of letting users hack their systems. The latest case study with Microsoft Kinetic should help pave the way.

    In the third stage, various devices will be able to interact with one another — my phone can detect my TV and my TV can communicate with my stereo. This will take a bit longer to get to as we still don't have common protocols for this type of communication. We also run the risk of companies developing their own proprietary protocols, such as a Samsung TV only talking to a Samsung phone, etc. Compatibility may require Google stepping in and using the Compatibility Test Suite (CTS) as a tool to enforce common protocols.

    Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders.

    Save 20% on registration with the code AN11RAD

    Prediction 2: Wearable Android

    Marko Gargenta: The form factor for Android boards is getting to be very small and the price of the actual chipset is approaching the $100 point for a full-featured device. This allows for development of wearable Android-powered devices. Some of them will be for fashion purposes, such as watches. Others will be for medical and safety applications. I predict that toward the end of this year we're going to start seeing high-end fashion accessories based on Android. We may not be aware they are Android-powered, and we may not be able to develop for them. At the same time, early medical devices will emerge, initially for non-critical applications. These will likely be closed, purpose-built systems with little opportunity for development or extension.

    Prediction 3: Android and networked cars

    Android logoMarko Gargenta: This is the next big frontier for Android to seize. The car industry is now at the point where the mobile phone industry was 5-10 years ago. People are going to want more from their car systems as they realize that things like Google Maps beat any stock navigation system. Consumers will want car-based connectivity to the Internet as well as apps.

    The first stage of networked car development will involve using Android to build proprietary systems. This is already underway with commercial systems being built for cars without users even knowing the systems are based on Android. The second stage will involve connecting the cars to the Internet. This can be done in a couple of ways: cars can have radios with their own connections to the Internet or a driver's mobile phone can be tapped for online access.

    Whatever approach we take, 4G and LTE network developments will help the process quite a bit. Once the cars are connected, manufactures will have the opportunity to open up kits for developers to build purpose-built applications for those systems. It is likely that manufacturers may tightly control what apps are allowed into what vehicles by running their own proprietary app stores with strict policies and quality control. This is simply the nature of the auto industry to self-police itself and focus heavily on testing the software. It is not very likely that we'll be able to simply download car apps from a major app market right away.


    November 23 2010

    Four short links: 23 November 2010

    1. Goodbye App Engine -- clear explanation of the reasons why Google AppEngine isn't the right thing to build your startup on. Don't read the comments unless you want to lose faith in humanity. (via Michael Koziarski on Twitter)
    2. Neato Robotics XV-11 Tear-down -- the start of hackable LIDAR, which would enable cheap and easy 3D mapping, via a Roomba-like robovacuum with a LIDAR module in it. (via Chris Anderson on Twitter)
    3. Boilerpipe -- code to remove boilerplate wrappers from a webpage, returning just the text you care about. (via Andy Baio)
    4. Visual Eyes -- web-based authoring tool developed at the University of Virginia to weave images, maps, charts, video and data into highly interactive and compelling dynamic visualizations. (via Courtney Johnston's Instapaper feed)

    June 13 2010

    The Guardian's Science Weekly podcast: The man behind the Large Hadron Collider

    We were honoured to have theoretical physicist Professor Sir Chris Llewellyn Smith in the studio with us.

    Chris is a former director general of Cern and was instrumental in creating the Large Hadron Collider. He's now chair of the council of SESAME (Synchrotron-light for Experimental Science and Applications in the Middle East).

    Chris updates us on progress in the field of nuclear fusion, and reveals some fascinating details about his time at Cern.

    Our own science correspondent Ian Sample has written a book about the LHC's quest for the Higgs boson, Massive: The Hunt for the God Particle. He was happy to tell us all about it.

    Producer Andy visited the new Skin exhibition at London's Wellcome Collection, where he got to wear a "social condom".

    We also highlight an experiment in science journalism we are carrying out on our website. Story Tracker. It might just revolutionise the way we cover major science stories.

    After reading nearly all of the documents at the centre of the University of East Anglia hacked emails furore, Guardian environment writer Fred Pearce wrote a book that is billed as the definitive account of the scandal. The Climate Files is out this week. Listen to James Randerson's interview with Fred in full in the latest Science Weekly Extra podcast.

    Finally, next weekend is Science Hack Weekend: Get Excited and Make Things with Science! at the Guardian's offices in King's Cross, London. Bring your own bunsen burner. (Actually, don't.)

    What is Science Hack Weekend? According to the organisers:

    "A bunch of geeks get together in the same physical space to collaborate and create awesome things, usually by mashing up APIs. A Hack Day is usually 48 hours long and involves a sleepover ...although not much sleeping happens when everyone is either hacking or playing Werewolf."

    Follow the podcast on our Science Weekly Twitter feed and receive updates on all breaking science news stories from Guardian Science.


    Join our Facebook group.

    Listen back through our archive.

    Subscribe free via iTunes to ensure every episode gets delivered. (Here is the non-iTunes URL feed).

    June 03 2010

    "Hackers" at 25

    Steven Levy wrote a book in the mid-1980s that introduced the term "hacker" -- the positive connotation -- to a wide audience. In the ensuing 25 years, that word and its accompanying community have gone through tremendous change. The book itself became a mainstay in tech libraries.

    O'Reilly recently released an updated 25th anniversary edition of "Hackers," so I checked in with Levy to discuss the book's development, its influence, and the role hackers continue to play.

    Writing "Hackers"

    Do you remember the original pitch for "Hackers"?

    Steven LevySteven Levy: I don't remember it, though I can tell you that it didn't wind up being what the book was. I thought I was going to embark on a series of magazine articles.

    Soon after I started researching, it seemed like it was going to be a two-part book starting with the Homebrew Computer Club and then the game hackers and that emerging industry. But then I realized that the whole hacker culture started at MIT. That was where I had to go, and it turned out to be a key section of the book.

    Of all the stories and profiles in the book, which resonated most with you?

    SL: The MIT story was just amazing. I stumbled upon this important history and no one else had chronicled it. It's difficult to overestimate how important that community was to hatching the culture of hacking, and really the culture of computing. It had ripples far beyond the hacker community that went out to the way we all use computers.

    I would learn about these people like Richard Greenblatt and Bill Gosper, that no one ever heard of. The way they expressed themselves and the reverberations they created were very influential. They were legends within the walls of MIT.

    It's a little bittersweet that the book itself wound up defining them to a larger audience. From a journalistic and psychological perspective, it was interesting to talk to Greenblatt and Gosper. They had mixed feelings about sharing with me. Gosper had the biggest vacillation. He was okay with the book, but he never went to a Hacker Conference. And there's things in the book that were a little embarrassing for Greenblatt, but he was okay. He felt: "Well, that's just journalism. The more important thing is hacking, and if the book is good for hacking, that's all right."

    Some of the people who grudgingly talked to me ended up appreciating the book. Marvin Minsky, for example, was a mercurial character. He'd get up and walk away and you didn't know whether the interview was over. I would follow him and keep asking questions. I ran into him at a Science Foo Camp last summer, and he said such nice things about the book. I had never heard these things from him. That was just super for me.

    Is there anyone you wish you had included?

    SL: I didn't set out to create an exhaustive chronicle of all hackers. Some people have told me I should have included them in the book. But I was following a narrative. Just because someone isn't in the book, doesn't mean they're not important.

    The "Hackers" impact and the 25th anniversary edition

    How has the definition of "hacker" changed over the last 25 years?

    Hackers: 25th Anniversary EditionSL: The original term was upbeat. The MIT people adopted it for computer wizards who took systems to the limit. It was a badge of pride, but it became associated with vandals and thieves. There was a lot of consternation in the mid-'70s when the word became synonymous with a destructive element. I never thought that was something to spend a lot of time worrying about, though. The original definition was still lurking there. The movement always went forward, and the people who were inclined to be hackers -- the ones who had the "hacker gene" -- they just kept on.

    More recently, the positive version of "hacker" has made a big comeback. I'm happy to see that.

    You mentioned a "hacker gene." Is there a connection across hacker generations? Does the book tap into that?

    SL: There definitely is. It's been so gratifying to hear people say the book changed their lives. Other times, I'll be reading a biography or an article about someone prominent in the computer world, and they'll mention "Hackers." Some have told me "I recognize myself in this book, and I didn't know those other people were out there."

    If you were writing the book today, how would you approach it?

    SL: It's hard to say because so much now is taken for granted that wasn't even thought of back then. One of the precepts of the hacker ethic -- which I tried to codify based on what I saw as the implicit assumptions of hackers for various generations -- was that you can create art and beauty on a computer. That was a crazy idea back then, but now it's pretty obvious. The super edgy ideas at that time are now so mainstream you don’t even think about them. So, now, it would have to be a really different kind of book.

    The 25th anniversary edition contains updated material. What did you add?

    SL: I went back to some of the people originally in the book. I also talked to people who, if I were writing the book now, would've been in it.

    Working behind the update was the idea that we had all gotten older. That just happens after 25 years. Everyone was still as enthusiastic as before, though. Take Bill Gates. Even though he has a different job, he's still as hardcore as he was. He's hacking vaccinations, now. And don't think for a minute he's lost track of what happens with Microsoft.

    The new people were great. Mark Zuckerberg, to me, is the person who embodies the kind of hacker who doesn't see the conflict so much between commerce and computer creativity. It's all at peace to him.

    Hacking in the present day

    Are we in a position now where low-cost storage and pay-as-you-go resources will unlock a level of innovation similar to what you encountered when you were writing "Hackers"?

    SL: It's a continuation. At the time, creating a program that lived on a paper tape and that was shared among a dozen people was as revolutionary and as dramatic a change as these cloud-based application centers are now. That same sort of flexibility and launching pad for creativity snowballed into something that the masses now participate in.

    Is Apple's success with closed products (iPhone, iPad, etc.) an ominous sign for hacking?

    SL: Even though Apple has an iron hand in executing the limits that they've set, there's a lot of creativity that happens on the iPhone and the iPad. Setting those limits isn't a very hackerish thing to do. But certainly if you're a hacker and you're so inclined, you could work within those limits and you could write something that Apple would approve. And people have. Just because Apple has top-down regulation, doesn't mean there's no room for creativity.

    What is the next frontier for hacking? Physical goods? Biological? Something else?

    SL: I don't think we're done with computer hacking. It's more important than ever to have creative ways to make use of this unbelievable infrastructure that we've created.

    Any time you think it's all been done, someone comes up with something that changes everything. Look at Napster. There was this period where people thought all the big companies would make all the big decisions, and then Napster came out of a dorm room and knocked everything upside down. We're going to see more of those. Facebook came from a dorm room, right? A great idea that's well implemented can go farther than it's ever been able to go before.

    This interview was condensed and edited.


    May 24 2010

    Four short links: 24 May 2010

    1. Google Documents API -- permissions, revisions, search, export, upload, and file. Somehow I had missed that this existed.
    2. Profile of Wikileaks Founder Julian Assange (Sydney Morning Herald) -- he draws no salary, is constantly on the move, lived for a while in a compound in Nairobi with other NGOs, and cowrote the rubberhose filesystem which offers deniable encryption.
    3. OpenPCR -- producing an open design for a PCR machine. PCR is how you take a single piece of DNA and make lots of copies of it. It's the first step in a lot of interesting bits of molecular biology. They're using Ponoko to print the cases. (via davetenhave on Twitter)
    4. Metric Mania (NY Times) -- The problem isn’t with statistical tests themselves but with what we do before and after we run them. First, we count if we can, but counting depends a great deal on previous assumptions about categorization. Consider, for example, the number of homeless people in Philadelphia, or the number of battered women in Atlanta, or the number of suicides in Denver. Is someone homeless if he’s unemployed and living with his brother’s family temporarily? Do we require that a women self-identify as battered to count her as such? If a person starts drinking day in and day out after a cancer diagnosis and dies from acute cirrhosis, did he kill himself? The answers to such questions significantly affect the count. We can never be reminded enough that the context for data must be made as open as the data. To do otherwise is to play Russian Roulette with the truth.

    Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
    Could not load more posts
    Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
    Just a second, loading more posts...
    You've reached the end.
    No Soup for you

    Don't be the product, buy the product!

    YES, I want to SOUP ●UP for ...