Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

November 29 2011

Sometimes one screen isn't enough

This is part of an ongoing series related to Peter Meyers' project "Breaking the Page: Transforming Books and the Reading Experience." We'll be featuring additional material in the weeks ahead. (Note: This post originally appeared on A New Kind of Book. It's republished with permission.)

I've been fiddling with the idea of using multiple displays to give a presentation — putting different slides on different screens. One design sketch — working title: "Documan" — has gotten some chuckles around my office (yes, I work alone):

Illustration of five iPads attached to a man, standing next to a computer display
Man-mounted iPads, plus a nearby monitor. A few possibilities not shown: each iPad could contain images, not just text; objects could move between iPads or from iPad to monitor; and presenter could rotate one or more iPads.

Why on earth does the world need to see a man strap on a half dozen iPads? And, more importantly, what kind of message would benefit from a rig like this?

Beats me. But I do think that content experiments, designed expressly for the screens we all use — rather than our ancestors' print pages or single PowerPoint slides — are the best way to figure out how stories and teaching change when they move onto the touchscreen.

I'll spare you, for now, the words and images I'm testing out to fill those screens. (One teaser, though: think about how easy Keynote for iPad makes it to build an action that exits screen right and enters screen left. Now, if you could just get the timing right when using two iPads ...).

Clearly, I'm not the only guy playing around here. Ahead, I round up a few content confections that span multiple screens. Some involve separate physical displays, others use different virtual windows. Not all of this stuff is new. But I find it thought provoking how creative types are using the small, medium, and large screens that increasingly coexist near each other.

iPad + projector

Joe Sabia calls himself an "iPad storyteller" — love it! He showed off his stuff at a recent TED talk where he uses his tablet and a variety of different apps (iBooks, a drawing app, Google Earth, Photos, and so on) to entertain an audience that is variously fixed on him, the big projector screen which his iPad is attached to, and the iPad's display itself.

iPad + magician

Sleight-of-hand artist and iPad maestro Simon Pierro pulls off some awfully clever tricks with his iPad and a real tennis ball, a glass of milk, and a weather forecaster's hair (she's on a video inside the iPad). I have no idea what's magic, what's video editing trickery, or what he and the iPad are actually doing. And, you know what? It doesn't matter. What he demonstrates here is how man and machine can team up to entertain in really innovative ways. Don't miss his part two, where he — sorta/kinda — sheds light on what he's done.

iPad-powered window displays

Gin Lane Media filled up three of Saks 5th Avenue's storefront windows with 64 iPads and nine 27-inch displays.

iPad/iPhone partnerships

A few apps use the big and small screen of a tablet and a smartphone in tandem. The iOS app Scrabble, for example, lets you conduct group games in which the iPad serves as publicly viewable board and the iPhone is each player's private letter stash. Remote Palette is a painting app where the iPad is the canvas and the iPhone is the paint palette.

Multiple browser windows

The band Arcade Fire worked with director Chris Milk to compose this mind-blowing HTML5-powered interactive video for its song "We Used to Wait." You give this web app the address of the house or building where you grew up in. It then whips together a custom-built video (woven around some stock footage) that incorporates Google Maps footage of your old neighborhood and other graphical magic mashups … all in multiple browser windows of various sizes. (It only works in the Chrome browser.) If you like this one, you'll love sour-mirror.jp, which uses snapshots of you from your laptop's webcam, and your Facebook and Twitter feed, to compose a multi-window extravaganza. It all culminates in a mosaic of your face built out of pix pulled from your social media feeds.

Multi-screen patterns

Here's a pattern-style analysis of different content and interaction designs for multiple displays, from the basic (how Amazon uses Whispersync to keep book location and notes coordinated across a user's different reading devices) to some innovative software that helps end users take an image, chop it up, and display it on their own collection of displays. That's what the next item is about.

Junkyard Jumbotron

Free to use (beta) software from some MITers that automatically splits up an image and displays it on whatever collection of screens (smartphones, tablets, PCs) you assemble. This demo shows it in action.

The multi-screen experience

Here's a five-minute video, with a bunch of TV and consumer electronics execs and analysts. Nothing hugely revelatory, but a nice little brain-tickler about how we are entering an age wherein audience and content producers alike are thinking about how to create and consume stories that play across displays of many different sizes.

Splitscreen: A Love Story

Heartwarming. Winner of a Nokia smartphone video-making contest, this video shows how split-screen stories can add up to more than the sum of their parts.

Google Wave cinema: "Pulp Fiction"

Not really — okay, not at all — safe for work, but a really nifty example of how innovative, multi-pane software (in this case, the soon-to-be late Google Wave), allowed one artist to take a scene from "Pulp Fiction" and render it within this program, weaving in videos, image, text, and maps.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb. 13-15, 2012, in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Related:

November 17 2011

What we could do with really big touchscreens

This is part of an ongoing series related to Peter Meyers' project "Breaking the Page: Transforming Books and the Reading Experience." We'll be featuring additional material in the weeks ahead. (Note: This post originally appeared on A New Kind of Book. It's republished with permission.)

I don't hear much talk about Microsoft's Surface computers, those industrial-strength touchscreens-on-a-tabletop. But maybe the idea was about $10,000 too expensive and a few years ahead of its time. Hear me out while I play connect-the-anecdata-points and argue that 10-inch tablets are just the start of the touchscreen publishing revolution. I'll bet that large, touchscreen canvases are coming, and I think they're going to change the kinds of documents we create.

But first a quick bit on why on earth we need larger compositional spaces. After all, any decent novelist, blogger, or journalist can get by with a 11-inch laptop, right? Sure, but what about creative types who like scattering notes, sketches, and outlines across their physical desktops? And what if they want to mix and match different kinds of media and incorporate touchscreen gestures? Some tools (Objective-C, HTML5) exist, but how many creative minds have the skills necessary to use that stuff?

Last week in my digital publishing tools webcast I previewed a handful of apps and online software that let people create "media mashups": compositions that break free from the rigidly sequenced vertical layouts that many writing tools impose. Take for example Microsoft Word or pretty much any blogging tool — only with some serious effort can you break free from producing a stacked sequence of editorial elements:

<some text>

<an image>

<a header>

<some text>

etc.

Rigid layout structures like that are, of course, great for mainly-prose narrative. But they make rich page layout — think: the interior design you see in a magazine, infographics, and their touchscreen successors — tough.

I hope you all take some time to play around with the software I mentioned — Webdoc, Blurb Mobile, Polyvore's editor, Storify, Hype, and Mixel. Only by practicing with these rich media canvases will we begin to see the kinds of stories and messages that might emerge if we move away from the constraints of tools that segregate word from image.

But what I didn't mention in my webcast, and the heart of this post, is a hardware development that feels increasingly likely: the arrival of large touchscreens that will make composition even easier than it currently is on devices like the iPad. Consider how the spread of really big touchscreens could improve the kinds of personal publishing projects we all work on ... from family photo books to website design, and from slideshow presentations to scrapbooking. If we could combine the touchscreen's signature talent (allowing us to signal our layout wishes directly: put this picture over there) with the large displays and workspaces that many of us enjoy at our work desks, wouldn't that change the kinds of documents we create? And wouldn't that require authoring tools that make it easy for us to mix and match different media types?

So, here's my list of recently spotted data points and observations:

The slow but steady convergence of Mac OS X and iOS

Anyone who follows Apple closely knows the deal here. Some headline developments for those who aren't Mac geeks: Lion's elevation of iOS-style, touch-friendly app icons; the increasingly high profile of touch gestures on all Mac laptops and, for the desktops, the availability of the Magic Trackpad. Steve Jobs rightfully dismissed the notion that we'd ever reach out and touch vertical displays. But it only takes a quick stroll down memory lane, and a glimpse at the sunflower-inspired iMac, to imagine a screen design that could easily shift between vertical (for long-form writing and reading) and horizontal-ish for touchscreen activities like page layout.

The heart of Windows 8: the touchscreen-friendly Metro

Microsoft's next big operating system update is built around the premise that people will want to switch between keyboard/mouse-controlled computers and those operated via touchscreen. They're counting on manufacturers to build tablets that do both. In one of their Metro demos, presenter Jensen Harris (a senior executive on the Windows user experience team) makes the case that in a few years it'll be rare to find any display — tablet, laptop, or desktop — that isn't touchscreen capable.

Touchscreen software for the big display

The New York Times' Nick Bilton wrote recently about a sneak peak Adobe gave him of a 50-inch "drafting table running Photoshop Touch where you can essentially draw and create on a screen." As Adobe CTO Kevin Lynch told him: "The creative process has been tied to a keyboard and mouse until now, and we want people to be able to touch the screen to create, just like we all used to use pencils and X-acto knives in the past."

Decreasing prices

We all know how this works: new technology gets cheaper as it matures. Those first generation Kindles sold for $399; now they start at $79. It's not hard to imagine a time when not only 20-plus-inch desktop monitors (the swivel variety, as I described above) are affordable, but also imagine portable touchscreen displays everywhere from your office walls to your refrigerator.

Growing familiarity with touchscreen gestures

Beyond early adopters, you see it everywhere: toddlers, deliverymen, senior citizens, checkout clerks — all of 'em understand how to tap, pinch, swipe. As a culture, we're becoming touchscreen literate.

The way I work

This one's personal, but I wonder how unique I am. My writing method often involves a bunch of writing surfaces: draft notes that I crank out on my desktop display; a sheet of physical notebook paper where I take notes on what I've written; another piece of paper on which I construct an annotated outline. I don't quite know what it is, but I just need to see it all spread out. And, man, do I love — do I need — to be able to draw lines, curves, circles, and arrows, connecting this idea over here, to that idea over there.

Writing, for me, on a laptop display feels claustrophobic. (I'm talking about the idea-generating and the drafting phase here; when it's time to revise, I'm plenty happy blocking out all distractions and focusing on a single, limited-size writing viewport.) LiquidText is one company I'm following closely; they're developing touchscreen-friendly reading tools that let so-called active readers tap, touch, highlight, and move text in ways that resemble my compositional tactics. They call it "multitouch document manipulation," and it's just one reason I'm incredibly excited about what may turn out to be the next desktop publishing revolution.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb 13-15, 2012, in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Photo on home and category pages: 40+242 Work by bark, on Flickr

Related:

September 20 2011

Five digital design ideas from Windows 8

This is part of an ongoing series related to Peter Meyers' project "Breaking the Page: Transforming Books and the Reading Experience." We'll be featuring additional material in the weeks ahead. (Note: This post originally appeared on A New Kind of Book. It's republished with permission.)


Microsoft deserves most of the design criticism it gets, but let's give them credit when they move in the right direction. What they've previewed in Windows 8 — especially the Metro touchscreen interface — is really lovely. It's humane, efficient, and innovative. In fact, I think there's plenty in it for digital book designers to think about emulating. I whipped out my notepad while watching one of their Build presentations — "8 traits of great Metro style apps" — and jotted down some key takeaways. (Also included are approximate timestamps so you don't have to sit through the whole 90 minutes.) The best part? Whether or not Microsoft actually ships something that matches their demo, you can benefit from the great thinking they've done.


Tablet users' postures and hand positions (16:31)

Microsoft did loads of research, hoping to identify how tablet users sat and where they placed their hands when holding these devices. The results are probably intuitive for anyone who's spent time with a tablet, but the conclusions are nevertheless helpful. Most people use both hands to hold a tablet, and the most frequent touch zones are on the edges. The lesson? "To design for comfort, you need to position [key controls] near the edge" (19:23). And: "It takes a posture change to reach comfortably into the center of the screen (in any orientation)." In other words, it's not that users can't reach things in the middle of the screen, but it does require they change how they're sitting. So, "put frequently used interaction surfaces near the edge," and "locate key controls to be comfortable to use while holding on to the edges of a device."

The difference between "fast & fluid" and "slow & jerky" (25:45)

The first phrase is Microsoft's (it's how they claim Windows 8 will perform; by the looks of the demo, they're pretty far along). The second phrase is mine, but in the demo it's clear that's what they want developers to stop doing. How? By using Microsoft-supplied transitional effects — for example, animating the way picture icons arrive on screen as users add them to a list. This might sound like frivolous eye candy, but the demo makes the point convincingly: these little points of polish make users feel a closer connection to the content and less like there's an engineer standing between them and what they want to do.

Specifically, what Microsoft is encouraging developers to do is use Windows 8's "Animation Library" to implement these effects and take advantage of things like hardware acceleration. This, they argue, saves programmers from having to master animation flourishes or learn After Effects; the ready-to-use animations take care of the design work. I mention all this because a sluggish reading experience — even one that's half a second too slow — can cause readers to bail.

This reminds me of a conversation I had last winter with Theo Gray, author of "The Elements for iPad" and one of the principals behind Touch Press. He was previewing an in-progress app for me and stopped the demo mid-way through. One of the gems onscreen that was supposed to spin was lagging a tiny bit. If you're even off by a little, he said, users will notice. Sweating the details like this may be one reason the Touch Press apps are so successful.

"A language for touch" (27:30)

The point Microsoft makes in this part of the presentation is, if you're making a touchscreen app, don't have fingers and touch gestures replicate what a mouse does. Multitouch screens can and should be controlled differently than our regular computers. And Microsoft makes this case by poking fun at the cumbersome steps an iOS user has to go through to drag an app icon from one home screen to another that's far away: "it's like driving a car from one side of the ocean to another." Anyone who's got more than a few screen's worth of apps knows what they're talking about. What Apple has currently designed is really the equivalent of how you'd scroll horizontally with a mouse (except in iOS there are no quick scrollbar shortcuts).

The solution that Microsoft demos is neat (28:48): you hold the app icon you want to move in place with one finger and then, with your other hand, you pan under it, swiping the screens quickly to get to the new placement spot where you want to drop the icon. It's very slick, and it's a reminder of the benefits of designing explicitly for a touchscreen.

"Semantic" zoom (33:25)

By now we're all used to tapping touchscreens to zoom in closer on an image or bump up the font size of an article. Microsoft has introduced a twist: zooming gestures now frequently deliver more and different kinds of info as users view content at different magnification levels. For example, when viewed up close, a group of neighboring app icons on the home screen might look like this:

But when the user zooms out to a bird's-eye view, that same group acquires a label, delivering an extra helping of information to help browsers decide where to go next or to rearrange groups into a different order.

The same kinds of semantic additions at different zoom levels could be helpful for digital book designers looking to provide different views (book-wide, chapter-level, and so on) for readers browsing through different levels of detail. A few months ago I wrote about Glo Bible and something similar they've done with their outline zooming tool.

True multitasking (46:42)

In Metro, two apps can co-exist side by side on the main screen. One sits center stage, and the other gets tucked in this so-called "snap" state: a compressed rectangular view that apps occupy when they cede the main part of the window to another app.

"A great snapped state," presenter Jensen Harris says, "invites users to keep an app on screen longer." These truncated views are fully functional. One fun example that gets a mention: imagine a piano app in snap state, a drum app on the main screen, and the user playing both of them at the same time. In other words, true multitasking and a world in which users are encouraged to make their apps interact with each other. It's a compelling reminder of something many serious readers (and writers) do all the time in the real world: keep multiple documents open simultaneously.

Webcast: Digital Bookmaking Tools Roundup #2 — Back by popular demand, in a second look at Digital Bookmaking Tools, author and book futurist Pete Meyers explores the existing options for creating digital books.

Join us on Thursday, November 10, 2011, at 10 am PT
Register for this free webcast

Related:

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl