Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

January 22 2014

Four short links: 22 January 2014

  1. How a Math Genius Hacked OkCupid to Find True Love (Wired) — if he doesn’t end up working for OK Cupid, productising this as a new service, something is wrong with the world.
  2. Humin: The App That Uses Context to Enable Better Human Connections (WaPo) — Humin is part of a growing trend of apps and services attempting to use context and anticipation to better serve users. The precogs are coming. I knew it.
  3. Spoiled Onions — analysis identifying bad actors in the Tor network, Since September 2013, we discovered several malicious or misconfigured exit relays[...]. These exit relays engaged in various attacks such as SSH and HTTPS MitM, HTML injection, and SSL stripping. We also found exit relays which were unintentionally interfering with network traffic because they were subject to DNS censorship.
  4. My Mind (Github) — a web application for creating and managing Mind maps. It is free to use and you can fork its source code. It is distributed under the terms of the MIT license.

December 05 2013

Four short links: 5 December 2013

  1. DeducerAn R Graphical User Interface (GUI) for Everyone.
  2. Integration of Civil Unmanned Aircraft Systems (UAS) in the National Airspace System (NAS) Roadmap (PDF, FAA) — first pass at regulatory framework for drones. (via Anil Dash)
  3. Bitcoin Stats — $21MM traded, $15MM of electricity spent mining. Goodness. (via Steve Klabnik)
  4. iOS vs Android Numbers (Luke Wroblewski) — roundup comparing Android to iOS in recent commerce writeups. More Android handsets, but less revenue per download/impression/etc.
Sponsored post

April 16 2013

Four short links: 16 April 2013

  1. Triage — iPhone app to quickly triage your email in your downtime. See also the backstory. Awesome UI.
  2. Webcam Pulse Detector — I was wondering how long it would take someone to do the Eulerian video magnification in real code. Now I’m wondering how long it will take the patent-inspired takedown…
  3. How Microsoft Quietly Built the City of the FutureThe team now collects 500 million data transactions every 24 hours, and the smart buildings software presents engineers with prioritized lists of misbehaving equipment. Algorithms can balance out the cost of a fix in terms of money and energy being wasted with other factors such as how much impact fixing it will have on employees who work in that building. Because of that kind of analysis, a lower-cost problem in a research lab with critical operations may rank higher priority-wise than a higher-cost fix that directly affects few. Almost half of the issues the system identifies can be corrected in under a minute, Smith says.
  4. UDOO (Kickstarter) — mini PC that could run either Android or Linux, with an Arduino-compatible board embedded. Like faster Raspberry Pi but with Arduino Due-compatible I/O.

December 24 2012

Four short links: 25 December 2012

  1. RebelMouse — aggregates FB, Twitter, Instagram, G+ content w/Pinboard-like aesthetics. It’s like aggregators we’ve had since 2004, but in this Brave New World we have to authenticate to a blogging service to get our own public posts out in a machine-readable form. 2012: it’s like 2000 but now we have FOUR AOLs! We’ve traded paywalls for graywalls, but the walls are still there. (via Poynter)
  2. Data Visualization Course Wiki — wiki for Stanford course cs448b, covering visualization with examples and critiques.
  3. Peristaltic Pump — for your Arduino medical projects, a pump that doesn’t touch the liquid it moves so the liquid can stay sterile.
  4. Breeze — MIT-licensed Javascript framework for building rich web apps.

November 28 2012

October 16 2012

Four short links: 16 October 2012

  1. — news app for iPhone, which lets you track updates and further news on a given story. (via Andy Baio)
  2. DataWrangler (Stanford) — an interactive tool for data cleaning and transformation. Spend less time formatting and more time analyzing your data. From the Stanford Visualization Group.
  3. Responsivator — see how websites look at different screen sizes.
  4. Accountable Algorithms (Ed Felten) — When we talk about making an algorithmic public process open, we mean two separate things. First, we want transparency: the public knows what the algorithm is. Second, we want the execution of the algorithm to be accountable: the public can check to make sure that the algorithm was executed correctly in a particular case. Transparency is addressed by traditional open government principles; but accountability is different.

September 17 2012

Mobile developers, integration, and discovery are what count now

The iPhone 5 may or may not be the most beautiful handheld device, but it barely matters anymore. Competitors have rendered its beauty and craftsmanship irrelevant. Amazon has received the message and responded with its latest set of tablets, and Google has responded with the Motorola Droids and the Nexus 7. These devices now have sufficient quality in their materials, specs, and base operating systems so that they can make any consumer happy. So if hardware is a toss up, where will the next battles be fought?

The answer: developers, integration, and discovery.

First, the very best developers will build apps that tap key trends: improved camera quality is making real-world text and face recognition more possible, geofencing data stores are making proximity–based apps more possible, and despite Steve Jobs’ assertion that God gave us 10 styli, there’s clearly a host of applications that are benefiting from pressure-sensitivity and pens. The level to which Apple and Google embrace these new technologies and extend the current state of the art in voice and gesture recognition will factor heavily into the quality and emergence of new applications. In addition, the extent to which Apple and Google can expose these new technologies — like NFC or always-on Glass cameras in Google’s case — will provide an advantage to developers.

Second, since many new applications will undoubtedly emerge for both Android and iOS at the same time, the way that the device and its applications fit into the users’ life will matter most. And it’s in this arena that Google is starting to respond with some of Apple’s own medicine. According to stats shared during Apple’s iPhone 5 announcement, the company has 435 million iTunes accounts and those users have downloaded 20 billion songs. Tim Cook acknowledged how powerful this integration is, saying “what sets us so far ahead of the competition … is how [apps, iCloud, and devices] work together.”

The alternatives Google has for managing a rich, high-quality media collection are lousy. But on the flip side, the number of people I know who are leaving their iPads at home in favor of their Nexus 7 tablets is remarkable. They’re switching because they use Gmail, Google Calendar, Maps and other Google services. The integration of these applications is deep and seamless in Android 4.1 (Jelly Bean) — just like media is seamlessly integrated on Apple. Google’s offering isn’t directly better than Apple’s. Rather, it’s a step beyond because the Google services started in the cloud, not on the desktop, and the services are critical to your daily life.

Developers, therefore, can profit from this integration by deeply integrating with native services from Google and Apple. And while the XCode/iOS development environment is easier and more accessible to developers than the current Android environment, the services available to developers are far more limited. Google has an edge here, and developers are smart enough that they’ll push through the limitations of the Android SDK.

Third, it’s downright impossible to discover great new apps. Amazon realized long ago that if they were going to have a massive bookstore, they needed to make discovery work, so they built the personalization and community teams, delivering innovative recommendations and a great reviews system. Neither Google nor Apple has these tools yet. In fact, one VC commented to me that until you are featured on the App Store, your downloads will be very few. And even Amazon hasn’t solved the casual browsing problem particularly well. While “Listmania!” lists were an attempt to create a curated list of things you might like, they can’t begin to approach the experience of going to an independent, well-curated bookstore.

The third battle, therefore, is for discovery. It’s not about the devices, the OS, or even the apps themselves. And I would argue that it’s not about search, either, though a great search experience is part of the solution. A great discovery experience will require great curators, high-quality inventory, and painless trial. Apple leads in this space today, if only because of its higher quality inventory that is at least partially a result of a more homogenous platform. But there’s tremendous room for growth, as startups like Xyologic and others take on the challenge.

It’s a great time to be mobile. We have access to beautiful devices with near “classic Leica” quality, and we have increasingly integrated experiences across our maps, email, calendar, and contacts. But the next major changes won’t come in our devices: they’ll come from developers building apps that make your device even more useful, and you’ll discover these apps in new ways.

August 24 2012

Four short links: 24 August 2012

  1. Speak Like a Pro (iTunes) — practice public speaking, and your phone will rate your performance and give you tips to improve. (via Idealog)
  2. If Hemingway Wrote Javascript — glorious. I swear I marked Andre Breton’s assignments at university. (via BoingBoing)
  3. R Open Sciopen source R packages that provide programmatic access to a variety of scientific data, full-text of journal articles, and repositories that provide real-time metrics of scholarly impact.
  4. Keeping Your Site Alive (EFF) — guide to surviving DDOS attacks. (via BoingBoing)

July 24 2012

Apps Rush: The Unilever Series, Bing Get MeThere, SoFit, Goldstar Savings Bank, Jurassic Park Builder and more

What's new on the app stores on Tuesday 24 July 2012

A selection of 13 new and notable apps for you today:

The Unilever Series at Tate Modern

London's Tate Modern has launched an official app for its 13-year "Unilever Series' of installations, "from Olafur Eliasson's sun to Ai Weiwei's carpet of sunflower seeds". That means more than 250 photos and 12 videos, as well as articles by curators and artists, and some of the early sketches for each exhibit.

Bing Get MeThere

Microsoft has launched a London travel app for iPhone using its Bing brand, promising "true door-to-door directions using Bing maps and live tube updates". Favourite journeys can also be set up for quick access.


SoFit is the latest social fitness app (hence the name, presumably), which awards points every time you exercise. It promises real-life rewards for this: "exclusive products from your favorite brands; downloads like music, videos and games; as well as fundraise for the causes you care about".

Goldstar Savings Bank

This iPad app wants to teach children about financial basics, without making it dry and boring. A tall order, but Goldstar Savings Bank may just have nailed it: the idea being it's an app for children to record their savings and earn money for household chores, in order to buy rewards.

Jurassic Park Builder

The latest family-friendly brand to spawn its own freemium game is Jurassic Park, with this new iOS game from Ludia. It follows the Smurfs' Village / FarmVille template with players building their own parks, buying virtual bucks through in-app purchases to fund it. $99.99 IAP in a game that's likely to appeal to children? Hmm. The game is US-only for now.
iPhone / iPad


Assistant is the latest Siri-like voice recognition app for a non-iOS platform. In this case: Windows Phone. It's a "virtual buddy for your smartphone that uses natural language technology" to answer questions, search for information and launch apps, hooking into Google, Facebook, Twitter, Foursquare, Evernote and other services.
Windows Phone

The Icky Mr Fox

UK studio Ickypen has launched a children's book app that sees Icky Mr Fox trying to ruin the afternoon tea of Mr Rabbit and Mr Mole, with "tippy-tappy objects" that speak their name when touched. Unusually, it's available on Android and BlackBerry PlayBook as well as iPad.
Android / iPad / BlackBerry PlayBook

Around The Clock

Swedish developer Wombi Apps has a characterful new iOS app for children all about clocks. It includes a mini-game for each hour of the day, from teeth-brushing and biking home from pre-school to hammering nails and slicing butter. The idea being to familiarise children with the clock, rather than overtly teach them how to read it.
iPhone / iPad

X-Ray for Android

Android owners concerned about nasty malward have a number of apps to choose from, as security companies pile onto the platform to capitalise on reports of Android viruses. X-Ray for Android is the latest, promising to scan for vulnerabilities and "keep your carrier honest".

5K To Marathon Runmeter GPS

Completed the programme set by a "couch to 5k" app? Time to step up, perhaps: this app focuses on going beyond 5k races to "give you feedback and motivation to go farther, be healthier, and live longer".

Party Wave

Cartoon-surfing game Party Wave looks fun on iOS, getting you to position a bunch of surfers to ride a big wave in top-down view, before switching to a side-on perspective to guide them through it. The game is also notable, though, for being the first from Japanese developer Mistwalker – founded by Final Fantasy creator Hironobu Sakaguchi.
iPhone / iPad

Cardagram Postcard

French digital-to-physical postcards app Cardagram has launched in the UK. Like established rival Touchnote, it turns iPhone photos into real postcards to be sent worldwide – usually charging £1.99, although it's £0.99 in a launch offer. One nice touch: it can pull in photos from Instagram and Facebook.

Historables: Marie Ant-toinette

Yes, Marie Antoinette re-imagined as a cartoon "ant queen" in a story-app for children. No, I have no idea how they handle the guillotine part. But yes, the app sees Marie baking and decorating a cake, setting up a castle room and wander through underground ant tunnels. More Historables apps are following from developer Base Camp Films: stand by for Teddy Bear Roosevelt and Lionardo Da Vinci...
iPad © 2012 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

June 01 2012

Four short links: 1 June 2012

  1. BeWell App (Google Play) -- continuously tracks user behaviors along three key health dimensions without requiring any user input — the user simply downloads the app and uses the phone as usual. Finally, someone tracking my behaviour for my own good.
  2. Met 3D -- the Metropolitan Museum of Art hosts its first 3d printing and scanning hackathon. [O]n June 1 and 2, approximately twenty-five digital artists and programmers will gather at the Met to experiment with the latest 3-D scanning and replicating technologies. Their aim will be to use the Museum's vast encyclopedic collections as a departure point for the creation of new work. THIS. IS. AWESOME. (via Alison Marigold)
  3. The Perfected Self (The Atlantic) -- everything you knew about B. F. Skinner was wrong, and you should know about him because you're using his techniques to lose weight, stop smoking, and do your homework. (via Erica Lloyd)
  4. Google Blockly -- (Google Code) A web-based, graphical programming language. Users can drag blocks together to build an application. No typing required. Open sourced.

May 08 2012

Four short links: 8 May 2012

  1. Gmail Vault -- app to backup and restore the contents of your gmail account. (via Hacker News)
  2. Leaving Apps for HTML5 (Technology Review) -- We sold 353 subscriptions through the iPad. We never discovered how to avoid the necessity of designing both landscape and portrait versions of the magazine for the app. We wasted $124,000 on outsourced software development. We fought amongst ourselves, and people left the company. There was untold expense of spirit. I hated every moment of our experiment with apps, because it tried to impose something closed, old, and printlike on something open, new, and digital. (via Alex Howard)
  3. Your Two Weeks of Fame, and Your Grandmother's (PDF) -- researchers mined 20C news articles to see whether shrinking news cycles caused briefer fame. Instead they found duration of celebrity is largely steady across the entire century, though depending on how they measured celebrity they could sometimes see changes in the duration with the most famous. (via Google Research)
  4. Dan Pink's Travel Tips -- the author travels a lot and has passed on his tips in these videos.

April 27 2012

What if ebook DRM goes away tomorrow?

This post originally appeared on Joe Wikert's Publishing 2020 Blog ("What if DRM Goes Away?"). This version has been lightly edited.

TOC Latin America was held last Friday in the beautiful city of Buenos Aires. Kat Meyer, my O'Reilly colleague, and Holger Volland did a terrific job producing the event. As is so often the case with great conferences, part of the value is spending time with speakers and other attendees in between sessions and at dinner gatherings.

Last Thursday night, I was fortunate enough to have dinner with Kat, Holger and a number of other TOC Latin America speakers. We discussed a number of interesting topics, but my favorite one was asking each person this question: What happens if DRM goes away tomorrow?

The DOJ suit against Apple and five of the Big Six has led to a lot of speculation. One of the most interesting scenarios raised is that if the government is intent on limiting the capabilities of the agency model, publishers need to figure out what other tools they can use to combat the growing dominance of Amazon.

Charlie Stross is right: DRM is a club publishers gave to Amazon and then insisted that Amazon beat them over their heads with it. So, what if we woke up tomorrow and DRM for books disappeared, just like it has (for the most part) with music?

I was unable to reach a consensus at that dinner, but here's what I think would happen: Initially, not much. After all, Amazon has a lot of momentum. If current U.S. estimates are accurate, Amazon controls about 60-65% of the ebook market and B&N is second with about 25-28%. That only leaves 7-13% for everyone else. And if you've been buying ebooks from Amazon up to now, you're not likely to immediately switch to buying from B&N just because they both offer books without DRM. On the surface, Amazon's and B&N's ebooks use incompatible formats — mobi for the former and EPUB for the latter. But that's where it gets interesting.

Converting from mobi to EPUB (or vice versa) is pretty simple with a free tool like Calibre. I've played around with it a bit, converting some of the DRM-free ebooks we sell on I didn't do those conversions to get our books in other formats. After all, when you buy a book from you're buying access to all the popular formats (mobi, EPUB and PDF, as well as others), not just the one format a device-maker wants to lock you into. I did the conversions because I wanted to see what's involved in the process.

If you've ever used Microsoft Word to save or convert a DOC file to PDF you'll find it's just as easy to go from mobi to EPUB in Calibre, for example. But just because the tool is available, does that mean if DRM goes away we'd suddenly see a lot of Kindle owners buying EPUBs from B&N and converting them to mobi with Calibre? I doubt it. Those Kindle owners are used to a seamless buying experience from Amazon, so unless there's a compelling reason to do so, they're not likely to switch ebook retailers. And that leads me to the most important point ...

Creating the best buying and reading experience is one way any ebook retailer can steal market share from the competition. Amazon has a pretty darned good one, that's for sure, but there's plenty of room for improvement. I'm not convinced any ebook retailer has pushed the envelope on innovation and exciting new features in their devices or reader apps. In fact, these enhancements seem to move at a glacial pace. So, what if B&N (or anyone else, for that matter) suddenly invested heavily in reader app functionality that puts them well ahead of the competition? And what if some of those features were so unique and innovative that they couldn't be copied by others? I'd much rather see a competitive marketplace based on this approach than the one we currently have, where the retailer with the deepest pockets wins.

Innovation is better than predatory pricing. What a concept. The iPod revolutionized music, an industry that was highly fragmented and looking for a way forward in the pre-iPod days. The iPhone turned the cellular market on its head. Think about how significantly different the original iPod and iPhone were when compared to the clumsy MP3 players and flip phones that preceded them. I believe today's crop of ebook readers and apps are, in many ways, as clumsy and simplistic as those MP3 players and flip phones. In other words, we haven't experienced a radical transformative moment in the ebook devices and app world yet.

Of course, all of this innovation I'm dreaming of could happen today. We don't need to wait for a DRM-free world. Or do we? Amazon has no incentive to innovate like this. They already have a majority market share, and it's only going to get larger when the DOJ dust settles.

This is more of a rallying cry for B&N, Kobo and every other device and ebook retailer. If DRM goes away tomorrow, nothing much changes unless these other players force it to. But why wait till DRM disappears? It might not happen for a long time. Meanwhile, the opportunity to innovate and create a path to market share gain exists today. I hope one or more of the minority market share players wakes up and takes action.

The future of publishing has a busy schedule.
Stay up to date with Tools of Change for Publishing events, publications, research and resources. Visit us at


February 13 2012

Apple's iTV and the implications of what Steve said

If I accept conventional wisdom, Apple is getting into the TV-making business because:

  1. The living room is the last consumer segment that Apple has yet to completely remake in its image.
  2. Apple creates new markets where none exist, and it isn't satisfied with merely improving upon existing ones.
  3. Steve Jobs allegedly said that he'd cracked the code for creating an integrated TV set.
  4. If the iPad is really "just" a big iPod Touch, and has already sold 55 million units, then a TV that is "just" a big iPad could do gonzo business.
  5. The business of making TVs is broken, and Apple has to fix it.
  6. Cable and satellite providers are evil, and Apple has to liberate consumers.
  7. Tim Cook "needs" a hit.

As I stated in my last post following Apple's gaudy earnings numbers, I don't accept conventional wisdom because conventional wisdom is dead! Apple killed it.

Most fundamentally, all assumptions about Apple seem to stem from a misunderstanding of how differently Apple thinks and operates from everyone else.

For starters, Apple doesn't chase markets just because they're there. Nor do they get sucked into market share battles just so they can say they sold the most units (see: iOS vs. Android).

Further, neither the aggrandizement of the CEO's ego nor the altruistic care-taking of the consumer drive Apple's product strategy.

Rather, Apple pursues markets purely and vigorously based upon a simple logic. Do they believe that their integrated hardware + software + service approach can be applied in a leveraged fashion to create a differentiated offering that delights consumers, appeals to the masses, and can be sold at high margins at a predictable run rate?

If the answer is "yes," then game on. If the answer is "no," then leave it as a hobby (such as the current Apple TV), or avoid the market altogether.

This is the backdrop for understanding the rumors about Apple building a new-fangled television set. Rumors and whispers notwithstanding, in the words of Dr. Hannibal Lecter, the obvious question is:

"Of each particular thing, ask: What is it in itself? What is its nature?"

Apple TV matrix
Top layer = iOS devices; Middle layer = Core device functions; Bottom layer = Noteworthy hardware subsystems.

In the case of a serious living room play, if you check out the above graphic, what stands out most about the Apple TV in its current incarnation is its lack of apps, web, and communications support. These elements are the three biggest game changers that propelled the iPhone, iPod Touch and iPad beyond the impressive media foundation that marked the pre-iOS iPod.

What is also lacking is the mainstream television programming (HBO, ESPN, ABC) that the typical consumer demands. A 'purdy' new TV doesn't remedy that problem, now does it?

But, remember, Apple is long removed from their anti-establishment days, whereby for the company to succeed the incumbent had to fail. Hence, the rebirth of the Mac was predicated on getting into bed with Microsoft; the rise of the iPod was predicated on getting into bed with the music industry; and the rise of the iPhone was predicated on getting into bed with mobile carriers.

When framed that way, who hasn't Apple gotten into bed with yet that they need to get in bed with to succeed in a mainstream way?

You guessed it; the cable and satellite providers. Why? Because as noted venture capitalist Bill Gurley sagely pointed out, "When it Comes to Television Content, Affiliate Fees Make the World Go 'Round."

In other words, for an Apple TV to be free-flowing with first-tier TV content in the same way that an iPod flows with first-tier music, Apple will need DIRECTV and/or Comcast to bless it.

ESPN, after all, earns $4.69 per subscriber household in affiliate fees on each and every cable subscriber. Apple's good friend, Disney, owns ESPN, ABC, Disney Channel and a slew of other channels. Disney simply isn't going to throw billions of dollars away in affiliate fees just so they can help Apple. All of the major TV content players view the world similarly.

So where does that get you when you connect the dots? I'll tell you where it doesn't get you ... to a television-like device that:

  1. Is priced 2-4X the cost of an iPad.
  2. Has sales cycles of one device every 5-10 years.
  3. Has bad margins.
  4. Has a serviceable form factor that for many people is good enough. (Apple challenges industries where the baseline experience is terrible. Television hardware wouldn't seem to qualify.)

Conversely, what if you could buy a set-top box that plugged into your modern, big-screen TV, and:

  1. It just worked.
  2. Had every channel you currently get on cable.
  3. You could run those same channels as apps on your other iOS devices.
  4. Your TV could be controlled by any of those same iOS devices.
  5. You could upgrade to the newest version of the set-top box every 2-3 years (on a carrier-subsidized basis).
    1. Who wouldn't buy this device? And why wouldn't the cable and satellite providers be all over this? After all, does anyone seriously like their set-top box?

      As a sanity check, a carrier subsidy on a sub-$500 device is meaningful, whereas a carrier subsidy on a $1,500+ device like a TV set is nothing.

      Wait! But, didn't Steve Jobs say that he'd like to make an integrated TV set?

      Even if he did say that, do you really think that in his final official act as Apple spokesman, Jobs would telegraph to the world his company's grand intentions in the living room?


January 24 2012

January 19 2012

The art of the app

David Hockney is not the only artist using an iPad to create new work. Here is a selection of art sent in by our readers that was created using various apps on the iPhone and iPad

December 14 2011

You can't get away with a bad mobile experience anymore

In the past, we accepted certain limitations in mobile content and speed because it was the actual, real web on a mobile device! But as Strangeloop president Joshua Bixby (@joshuabixby) notes in the following interview, that mode of thinking is on its way out. Now, most customers expect a full and fast web experience whether they're on a desktop, tablet or phone. The companies that offer that are poised to succeed. And for those that don't, ridicule will be the least of their concerns.

Should companies be thinking "mobile first" now?

Joshua BixbyJoshua Bixby: We talk about "the web" and "the mobile web" as if the two are different, but they aren't. I'm the first to admit that I'm as guilty of doing this as the next person. Using these terms is helpful for discussing differences in how people browse via different devices, but at the end of the day, it's all one web. Users want the same breadth and depth of content, no matter what device they're using. They want a consistent, reliable user experience. They don't want to interact with your site one way at their desks, then learn a whole new way when they're tablet-surfing on the couch, and then learn a third way when they're roaming around with their phones. Site owners who can deliver an experience that feels the same, regardless of the platform, are the ones who are going to own the web of the not-too-distant future.

Will all phones be smartphones at some point? Or will non-smartphones remain an important segment for years to come?

Joshua Bixby: It really depends on which market you look at. Last summer, comScore published a survey that showed that, despite the rapid rate of smartphone adoption, 155 million American mobile phone users still don't have smartphones. That's obviously a pretty significant number. But if you take a global view, things flip around and we see that, especially in developing countries, new mobile users are jumping right on the smartphone wagon. This allows users to bypass both dumb phones and the desktop web. It's a whole different way of interacting with the Internet.

I think an even better question might be: How great a disruptor will tablets be to the mobile market? Site owners have been caught with their mobile pants down, so to speak, in their inability to recognize the importance of this market. Forrester did a survey of retailers and found that, on average, about 20% of holiday mobile traffic came via tablets, with some retailers reporting that more than half their mobile traffic came via tablets. But most "mobile-optimized" sites look terrible on tablets. In 2012, site owners are going to be scrambling to catch up with this paradigm shift.

Should mobile app development be prioritized over mobile site development?

Joshua Bixby: Mobile apps will continue to have a role for repeat, loyal users of an online store or service, but it's incredibly narrow-sighted of companies to focus on app development over site development. People are always going to want to access the full public Internet. It's circa-1995, AOL-style thinking to assume otherwise.

I've seen scenario after scenario where apps present a huge usability problem because they don't anticipate the various ways that the web and apps intersect. Media sites are some of the worst culprits. Here's a common scenario: You're checking your Twitter feed and click on a link, but instead of taking you to the article, you're funneled to an intermediary "download our app" page. Often, this is a demand, not a request. There's no way to bypass this app block. And then say you do download the app — after it's installed, you don't even get served the article you originally wanted to see. You just get served the home page of the site, so you have to go back to Twitter and find the original link again.

This is just one scenario, but there are plenty more, and they all highlight the fact that many site owners don't recognize the complexity of mobile user paths through the web.

What are the most important mobile key performance indicators (KPIs)? How do these differ from the KPIs of traditional websites?

Joshua Bixby: This really depends on the type of site. Obviously, for ecommerce sites, you're always going to care about revenue and conversion, but these aren't necessarily as important for mobile as for desktop traffic. For all you know, your mobile user is standing in your bricks-and-mortar store doing some price-checking or reading product reviews. They're going to convert in your store. For these users, what you care about is engagement, not conversions, which is where KPIs like bounce rate and page views come in.

Site owners should look closely at bounce rate, page views, and time on page for their mobile traffic. These numbers should be commensurate with their desktop traffic. If they're way off — say you're averaging 2.3 page views per mobile visitor and 5.9 page views per desktop visitor — that could be an indicator that the mobile site is doing something to drive users away. In that case, you should analyze how fast your pages are loading or how usable they are.

Does a one-second delay on the mobile side have more significant repercussions than a one-second delay on the desktop side?

Joshua Bixby: I'm in the process of researching this very question. I recently did some fairly exhaustive analysis of mobile KPIs for a couple of Strangeloop's customers, which involved looking at more than 500,000 unique visits and analyzing how page load time affected metrics like conversion, abandonment rate, cart size, and page views. I presented my findings at Velocity Europe and Velocity Berlin. When I got back home, I asked myself, "Why didn't I compare these numbers against some kind of desktop baseline?" So I'm doing that now. I'll be happy to share my findings when I'm done.

Have you found that users of one type of mobile OS or device are more accepting of delays than others?

Joshua Bixby: This is a really neat area of research, and one that we've only begun to explore. In the mobile research I just mentioned, we did a little experiment where we used network quality as a proxy for performance. The reason for doing this was to try to get a sense of what major changes in page load — say the difference between pages that load in six seconds versus pages that load in 20-plus seconds — do to metrics. We found a few interesting things.

To start, when it comes to bounce rate, we found that iPad users have about the same bounce rate as Android and iPhone users — about 22-24% — when pages are served slowly to all three groups (iPad, iPhone and Android). But speeding pages up had a much greater impact on the iPad group than it did on the other two groups: The iPad users' bounce rates dropped to around 5%, compared to 8% for iPhone users and 11% for Android users.

What this means is anyone's guess, but I'd conjecture that the iPad experience is probably more conducive to extended browsing, so iPad users are more likely to stick around once they're confident that they'll get a reasonable user experience. Or maybe it's because iPad users are more likely to be sitting at home on their couch, while smartphone users are more likely to be out and about.

What are the most common mobile optimization mistakes? How should they be addressed?

Joshua Bixby: The most common mistake I see is when site owners aggressively optimize their sites for very specific platforms. There's a blog that I'm kind of addicted to called WTF Mobile Web, that is filled with examples of this from companies that should probably know better. For example, if you go to Canon's site on your iPad, you're served a fixed-width site that takes up about two-fifths of the screen. If I'm using a tablet, I've got a big shiny screen. I want to see pictures and rich content.

Canon's website on an iPad
Something seems to be missing ... fails to take full advantage of the iPad's screen size.

Speaking of stripped-down sites, there's such a thing as being too stripped down. The overly minimalist look of so many "mobile-optimized" pages was an understandable reaction to the wave of complaints about pages being too busy on mobile devices. But the bare-bones menu concept is not a solution. People expect more from websites in the aesthetic sense, and they want more than a plain column of buttons.

Here's one last complaint: "Mobile-optimized" sites that contain a "See full site" link, but then after you click on it, it's impossible to return to the mobile version. This has happened to me a couple of times, and it's kind of maddening. When you're used to the relative ease and fluidity of the web, these rabbit holes are hard to accept.

What's your take on the Kindle Fire and the Silk browser?

Joshua Bixby: The Kindle Fire and the Silk browser are really fascinating developments because they represent the truly out-of-the-box thinking that's going to usher in the next phase of aggressive performance optimization. That said, I don't think they're going to radically change the landscape in and of themselves.

We all got excited when Amazon described Silk's so-called "split browser" design — the fact that Silk can offload complex browser tasks to Amazon's cloud, resulting in dramatically faster load times. The logical next questions became: Will there be any imitators among the other browser vendors? Does the split-browser concept represent a new direction for browsers? The short answer to both questions is "no," at least for 2012.

For starters, there are privacy concerns surrounding the fact that this kind of split architecture has the potential to capture private user data. And while Silk offers a performance boost for some tablet content, even its own product manager, Brett Taylor, says of tablet browsing, "It's not meant to process and crunch a lot of heavy data."

Basic optimization techniques — such as those embedded in Silk — that transform code on a per-page basis to render pages faster in the browser, can actually slow down, or even break, pages. Web pages are becoming even more complex, data-intensive, and dynamic. For now, advanced content optimization — which takes a big-picture approach to accelerating an entire site — is still the only reliable way to radically optimize sites without causing harm.

Since we're in the midst of the holiday shopping season, what role do you see mobile playing this year?

Joshua Bixby: The 2011 holiday shopping season has proven that the mobile web is no longer a curiosity: 15.2% of online traffic came from mobile devices on Thanksgiving Day, more than double the previous year. Not only were these mobile shoppers spending money, they spent more on average — particularly shoppers using iPads — than desktop shoppers. This will be a wake-up call to retailers. Rather than keeping mobile on the sideline, companies will grow their mobile teams to match the size and scope of their regular development teams.

Joshua Bixby discussed mobile speed and KPIs at Velocity Europe. His full presentation is available in the following video.


December 07 2011

A sensible look at HTML5 and publishing

EPUB 3 and Kindle Format 8 both boast support for HTML5, but what exactly is HTML5 and what is its role in publishing? For insight on these questions — and practical ways HTML5 can be used by publishers — I reached out to Sanders Kleinfeld (@sandersk), author of "HTML5 for Publishers." (Kleinfeld also will present an HTML5 for Publishers webcast on Wednesday, December 14, 2011 — you can register to attend here.)

Our interview follows.

Why should publishers care about HTML5?

Sanders Kleinfeld: HTML5 is the future of digital publishing. If you're a publisher who's interested in staying competitive in the ebook landscape, it's quite crucial that you understand what HTML5 is all about.

So what is HTML5, exactly? The term is thrown around a lot, but it seems undefined.

SandersKleinfeld.jpgSanders Kleinfeld: The term "HTML5" is indeed used very fluidly in tech discourse, and it has really become a signifier for a constellation of different technologies, some only loosely related to actual HTML markup. When people refer to HTML5, they're usually talking about some combination of the following next-generation web technologies: Canvas, geolocation, native audio/video, local storage, and CSS3.

In your book, you instruct readers on using the <canvas> element. What is that and why is it helpful?

Sanders Kleinfeld: The <canvas> element allows you to embed an interactive sketchpad into your web or ebook content. You can control it with JavaScript. Because the canvas is scriptable, it opens the door to everything from computer-generated drawings to animations and full-fledged games. If you're interested in "app-ifying" your ebook (i.e., adding the kinds of interactive features that are the hallmark of iPhone or Android Apps), the <canvas> element and its associated API are the tools that are going to allow you to accomplish that.

How can publishers make use of HTML5's geolocation abilities?

Sanders Kleinfeld: Much as websites like Google already customize search results and advertisements based on users' locations, geolocation enables publishers to tailor their ebook content based on where their readers are currently located. This seems particularly beneficial to publishers of travel or restaurant guides, as they can sort and customize hotel/dining reviews based on proximity to the reader's location, suggest points of interest nearby, and perhaps even offer directions from one locale to another.

In "HTML5 for Publishers," I explore the possibility of geolocated fiction, where the reader's current location actually figures into the text of the story. [Click here to see an example of this in action.]

More avant-garde uses of geolocation in ebooks might extend to interactive activities and games like geocaching.

The International Digital Publishing Forum (IDPF) has signed off on EPUB 3. What effect will EPUB 3 have on HTML5?

Sanders Kleinfeld: Prior to the finalization of EPUB 3, the EPUB format already had a huge amount of momentum behind it, as an open standard supported by nearly every major ereading platform: iBooks for iPhone/iPad, Nook, Sony Reader, Adobe Digital Editions, etc. — Amazon's Kindle is really the only notable exception. The release of the EPUB 3 standard, which designates HTML5 as the language to be used for ebook content documents, firmly aligns the format with next-generation web technology. I think it's going to serve as one of the primary catalysts for publishers to get into the HTML5 game and for the major ereading platforms to adopt robust HTML5 support. Publishers are clamoring to enhance their ebooks with interactive and multimedia features, ereader manufacturers want to support these features, and EPUB 3 provides a clearly defined path forward.

We're already beginning to see support for HTML5 features emerge on some of the most popular ereaders. Both iBooks and the Nook Color already support HTML5 audio and video, as do cloud platforms like Ibis Reader. IBooks also supports many <canvas> features. I think it's just a matter of time before other ereaders follow suit.

HTML5 for Publishers — This free ebook provides an overview of some of the most exciting features HTML5 provides to ebook content creators — audio/video, geolocation, and the canvas — and shows how to put them in action.

What's your take on Kindle Format 8?

Sanders Kleinfeld: Kindle Format 8 (KF8) is Amazon's answer to EPUB 3. It's a proprietary standard for Amazon's ereader platforms that adds support for HTML5 and CSS3. Amazon recently published a list of KF8's new capabilities.

Prior to KF8, Kindle's CSS support in Mobi 7 was rather rudimentary, which posed many challenges to ebook publishers with highly graphical content that demanded sophisticated, precise layout. KF8 provides the necessary tools for producing these types of books. It will facilitate the creation of children's books, comic books, and other graphically rich content for Kindle.

More generally, KF8 is also going to make it easier for publishers to make "prettier" ebooks for Kindle, and I think it's important not to dismiss the value of aesthetics to the ereading experience. With the release of the Kindle Fire, Amazon is clearly looking to establish itself as a player in the tablet market, and I think KF8 is going to help Kindle keep pace with iBooks.

That said, while I'm encouraged to see Kindle adopt greater HTML5 support, as a staunch open source advocate and sometimes-beleaguered ebook developer who would love all ereaders to unite behind one file format, very little would make me happier than seeing Amazon adopt the EPUB 3 standard.

What's the best way for publishers to approach your book? Is it more of an introduction, or do they need some basic knowledge first?

Sanders Kleinfeld: In "HTML5 for Publishers," I provide an overview of the HTML5 technologies I believe will be most important to the next wave of ebook innovation, along with sample code and demos showing these HTML5 features in action. No formal knowledge of HTML or programming is necessary to appreciate "HTML5 for Publishers," but if you're interested in diving in and developing your own HTML5 content, some background in HTML, CSS, and JavaScript will most certainly be helpful. I provide links to additional HTML5 resources throughout the book for those looking to learn more.

What should publishers keep in mind as they explore HTML5 for their own needs?

Sanders Kleinfeld: As with every new technology, I think it's important for publishers to take a step back and not allow the hype to distract from practicality.

Consider what aspects of HTML5 might benefit and enhance your ebook program, and employ them judiciously. For example, if you're publishing a series of foreign language guides, embedding HTML5 audio/video content throughout your ebooks will likely be received as a welcome enhancement to readers. But if you're publishing serious literature, adding lots of audio and video may be a distraction. Don't be afraid to be innovative, but always put your readership's needs first.

This interview was edited and condensed.


December 06 2011

Four short links: 6 December 2011

  1. How to Dispel Your Illusions (NY Review of Books) -- Freeman Dyson writing about Daniel Kahneman's latest book. Only by understanding our cognitive illusions can we hope to transcend them.
  2. Appify-UI (github) -- Create the simplest possible Mac OS X apps. Uses HTML5 for the UI. Supports scripting with anything and everything. (via Hacker News)
  3. Translation Memory (Etsy) -- using Lucene/SOLR to help automate the translation of their UI. (via Twitter)
  4. Automatically Tagging Entities with Descriptive Phrases (PDF) -- Microsoft Research paper on automated tagging. Under the hood it uses Map/Reduce and the Microsoft Dryad framework. (via Ben Lorica)

October 04 2011

PhoneGap basics: What it is and what it can do for mobile developers

PhoneGapPorting mobile apps across systems is, to put it kindly, an inelegant process. There's considerable work involved — so much so that developers are sometimes forced to limit their efforts to one platform.

PhoneGap, an open-source mobile framework, offers an alternative: It helps developers build a common codebase for their apps so the apps work across devices and systems.

I recently spoke with Joe Bowser (@infil00p), creator of PhoneGap's Android implementation, to get his take on the strengths and limitations of PhoneGap and what developers need to know before putting it to use. Bowser will dive into a number of related topics during his session at next week's Android Open conference.

Our interview follows. (Note: this interview was conducted before Adobe announced its acquisition of PhoneGap's parent company, Nitobi.)

What is PhoneGap and why should mobile developers consider using it?

Joe Bowser: PhoneGap is an application framework that allows developers to use HTML, JavaScript and CSS to create apps that are present as first-class applications on the phone. That means the apps have their own icons and operate similarly to native applications without a browser frame around them. They are distributed via the application stores, such as the Android Market and the Apple App Store, and they have access to a set of native functions to further make them work like native apps.

Developers use PhoneGap because it allows them to have a common codebase for all their application code. It doesn't force developers to reinvent the wheel every time they move from platform to platform.

Are there downsides to using PhoneGap?

Joe Bowser: You are subject to the limitations of the browser and the JavaScript engine that comes with your device. On Android 2.3, this isn't too bad. Earlier versions of Android don't support certain features, and many of them use older JavaScript interpreters, which can impact an application. Also, there are certain things that are better implemented in native code, like cryptography or 3-D graphics. Most apps don't use features like this — they simply display information, which the web does well.

Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders.

Save 20% on registration with the code AN11RAD

What challenges did you face when creating the Android PhoneGap implementation?

Joe Bowser: The Android PhoneGap implementation was our first implementation after the iPhone, so there were questions about whether this was possible at all. At that time, the Android 1.0 SDK was just being released, and the only devices that ran Android were the HTC Dream and T-Mobile G1. This has obviously changed, but the most challenging thing is still testing on all the real devices that are out there. Every device has its own implementation of the Android OS and its own implementation of the WebKit rendering engine.

What's the best way for PhoneGap developers to handle device-specific needs?

Joe Bowser: It depends on the feature set. Most applications don't need many device-specific features beyond the user interface, but there are numerous plugins that can help with this approach. The best approach is to decide what features you need and to use only those features. There are many applications that have permissions turned on that they don't need. For example, a simple ebook doesn't need access to your phone state, GPS or contacts.

What is a hybrid app?

Joe Bowser: A hybrid application is one that has features of both a web application and a native application. Certain features, such as Image Capture, NFC or Android OpenAccessory, may be implemented natively since there is currently no way to do this in JavaScript. But the application logic and the UI are implemented using web technologies to allow for a consistent and unique user experience across devices.

This interview was edited and condensed.


August 30 2011

How to create sustainable open data projects with purpose

mySocietyThere has been much hand-wringing of late about whether the explosion of government-run app contests over the last couple of years has generated any real value for the public. With only one of the Apps for Democracy projects still running, it's easy to see the entire movement being written off as an overly optimistic fad.

The organisation that I'm lucky enough to lead — mySociety — didn't come from the world of app contests, but it does build the kind of open-source, open-data-grounded civic apps that such contests are suppose to produce. I believe that mySociety's story shows that it's possible to build meaningful, impactful civic and democratic web apps, to grow them to a scale where they're unambiguously a good use of time and money, then sustain them for years at a time. Right now we're launching a new site, FixMyTransport, that is trying to try to raise the bar for the ambition and scale of civic apps, so this seems a good moment to share some thoughts about what it takes to build good services and get them to last more than a few months.

You have to be just as focused on user needs as any company (and perhaps more so)

People have needs. Sometimes they need to eat, sometimes they need to sleep. And sometimes they need to send an urgent message to a local politician, or get a dangerous hanging branch cleared off of a road.

What people never, ever do is wake up thinking, "Today I need to do something civic," or, "Today I will explore some interesting data via an attractive visualisation." MySociety has always been unashamed about packaging civic services in a way that appeals directly to real people with real, everyday needs. I gleefully delete the two or three emails a year that land in our inbox suggesting that FixMyStreet should be renamed to FixOurStreet. No, dude, when I'm pissed it's definitely my street, which is why people have borrowed the name around the world.

We learned this lesson most vividly from Pledgebank, a sputtering site with occasional amazing successes and lots and lots of "meh." The reason it never took off was because, unlike the later (and brilliant) Kickstarter, we didn't make it specific enough. We didn't say "use this site to raise money for your first album," or "use this site to organise a march." We said it was a platform for "getting things done," and the users walked away in confusion. That's why our new site is called FixMyTransport, even though it's actually the first instance of a general civic-problem-fixing platform that could handle nearly any kind of local campaigning.

Being focused on user needs means not starting things you think you probably can't finish

In mySociety's history we have run four calls for proposals, asking the whole world what we should build next. Like most idea gathering processes, there's about 100 bad ideas for every good one, but the bad ideas have value in that they reveal a habitual digital era trait — being insanely optimistic about the effort required to build things to a high standard.

Now, clearly, I'm not saying it is impossible to hack brilliant things without piles of VC gold. But if you are going to hack something really, genuinely valuable in just a couple of weeks, and you want it to thrive and survive in the real Internet, you need to have an idea that is as simple as it is brilliant. Matthew Somerville's accessible Traintimes fits into this category, as does, and But ideas like this are super rare — they're so simple and powerful that really polished sites can be built and sustained on volunteer-level time contributions. I salute the geniuses who gave us the four sites I just mentioned. They make me feel small and stupid.

If your civic hack idea is more complicated than this, then you should really go hunting for funding before you set about coding. Because the Internet is a savagely competitive place, and if your site isn't pretty spanking, nobody is going to come except the robots and spammers.

To be clear — FixMyTransport is not an example of a super-simple genius idea. I wish it were. Rather it's our response to the questions "What's missing in the civic web?" and "What's still too hard to get done online?" But we didn't start building it until we knew we had the money, and we didn't try to fit it into evenings and weekends. It was painful to wait and not rush with it, but it was the right thing to do to build something up to the expectations of an Internet-using public habituated to websites with billion-dollar budgets. And we are emotionally and financially prepared for the six months of rapid iteration that will follow once the public arrives.

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code ORM30

Data is your servant, not your master

I love open data. I love structured data. I love data, full stop. But my love of data is not the same as respecting our users' needs. There are more than 300,000 bus stops, train stations, ferry routes and so on in the FixMyTransport back end, munged together over months of hard work from dirty, dirty public data sources. Can you see any sign of this on the homepage? No sir, because users want to fix transport problems, not revel in our mastery of databases.

Demand fewer, larger grants from government and funders

MySociety got lucky. It was born into a period of high public spending, 2003/4, and its second ever grant was for 0.02% of a government funding pot worth more than a billion dollars — about a quarter of a million dollars. It was amazing luck for a small organisation with no track record, possible only because so much money was being thrown around. Those days are gone on both sides of the Pond, but governments everywhere should note that that funding of this scale got us right through our first couple of years, until sites like WriteToThem were mature and had proved their public value (and picked up an award or two).

In the subsequent few years, we saw the "thousand flowers bloom" mentality really take over the world of public-good digital funding, and we saw it go way beyond what was sensible. Time and again, we'd see two good ideas get funding and eight bad ones at the same time because of the sense that it was necessary to spread the money around. It would be great if someone could make the case to public grant funders that good tech ideas — and the teams that can implement them — are vanishingly rare. There is nothing to be ashamed about dividing the pot up two or three ways if there are only a few ideas or proposals or hacks that justify the money. The larger amounts this would produce wouldn't mean champagne parties for grantees, it would mean the best ideas surviving long enough to grow meaningful traffic and learn how to make money other ways.

After a long road supported by public grant funding, mySociety is now 50% commercially funded and 50% private-grant funded, but we'd never have arrived there without being 100% public-grant funded for the first couple of years. Now our key donors are philanthropic, with Indigo Trust in particular covering most of the core development cost for FixMyTransport.

Respect the geeks

All great technology projects have one or more über geeks at the heart of them. If you find the right über geeks, they'll understand politics, society and users just as much as they understand their code. If you find someone as ferociously multi-talented as, say, Louise Crow, who built FixMyTransport almost single-handedly, listen to them and change your plans when they say "no." Luckily, she said "yes" to building this project, and I hope those of you who care about civic tech give her the props appropriate to building something on this scale. Respect her, and respect the geeks like her, and you'll be one step closer to civic app success.


Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
No Soup for you

Don't be the product, buy the product!

YES, I want to SOUP ●UP for ...