Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

November 16 2010

New directions in web architecture. Again.

In 2005, Jesse James Garrett at Adaptive Path published the seminal blog "Ajax: A New Approach to Web Applications" and ushered in new age of web architecture. Ajax meant using the possibilities latent in JavaScript (specifically, the XMLHttpRequest object) so that a web page could contact the server asynchronously and request new data.

This was revolutionary; within months, we were seeing pages that were more dynamic and interactive. Ajax short-circuited the submit/response loop that dominated web applications up to that time. Instead of making an HTTP request, receiving an entire web page, and rendering that page as a replacement for the current page, the browser requested a chunk of data. It used that chunk of data to interact with the DOM and rewrite the page it was displaying on the fly.

Around the same time, the RESTful paradigm started taking hold. REST represented a much simpler, web-oriented way for servers to interact with their clients. As Roy Fielding pointed out in his dissertation, the basic operations of the HTTP protocol were capable of providing general access to data; stateful applications could be built upon stateless protocols; hypermedia could be used to maintain application state. Although Fielding's dissertation dates back to 2000, it took a few years of bad experience with SOAP and its heirs to realize its importance. With REST, it becomes easier for a website to see itself as a source of data for machines to process, rather than as a source of content for humans to read. Websites become data servers.

New Twitter
The HTML page you get from the new Twitter is largely a bunch of empty divs, with a big wad of JavaScript. The JavaScript is the entire application.

Important as Ajax and REST have been to the history of the web, each only represents half of a larger revolution. And in the past few months, we've seen some new sites that have taken the revolution to its logical conclusion. Specifically: take a look at the new Twitter. It's a nice web application, sure -- but look at the HTML. There's not much there. The HTML page you get from Twitter is largely a bunch of empty divs, with a big wad of JavaScript. What's happening? The JavaScript is the entire application; the divs exist only to provide tags so the JavaScript can rewrite the DOM at will. In turn, the JavaScript is constantly (and asynchronously) making requests from the Twitter site, which is just returning data from its API. In fact, the Twitter site is returning the same data for its web page that it would return for its mobile app, for TweetDeck, or for any of the apps in the Twitter ecosystem.

This design isn't particularly new; we've seen it ever since developers started reverse-engineering GMail and Google Maps to get ideas for their own projects. Those big Google apps may have been the first examples of this architectural trend. They were certainly among the first to use JavaScript as a full-fledge client programming language. But we're seeing many more sites built along these lines. Why now, what does this shift mean, and why is it important?

"Why now" is perhaps the easiest question to answer. A few short years ago, web developers only had one platform to support, and that was the "browser." Granted, there were a dozen or so browsers of significance, and the browser world was riddled with incompatibilities. We're in a different world now. Browser compatibilities have been ironed out, to some extent (though conscientious developers still support "legacy" browsers, all the way back to IE6 or even IE5). But it's no news that the most important new apps these days run on devices ranging from phones (iPhone, Android, BlackBerry, Windows Phone), tablets (iPad, Android/ChromeOS devices), and potentially ebook readers and other new devices. With these new devices on the table, browser incompatibilities pale in significance. It's another sign of the times that I can't conceive of an interesting application that doesn't access data across the network. A static application that never accesses remote data -- that's so 1990s.

So, while it's tempting to say that the new age is characterized by the browser as platform, and that applications running in the browser can do anything that native code can do, that's looking in the wrong place. HTML5 certainly ups the ante, as far as browser capabilities -- and is supported to some extent by all of the other devices we're concerned with. But the real meaning and importance of this architectural shift is on the server side, driven both by the need to support many heterogenous device and application types, and by the primacy of live data in modern applications.

In the browser-dominated world, static content and data were inevitably mixed. Yes, we had templating systems that let developers separate static content and design elements from data. But once the application server did its magic, what was delivered to the browser was HTML pages mixing data with other content. Browsers were similar enough that, with some browser detection hacks on both the server and client side, it was relatively easy (though a pain) to generate pages that would run anywhere. That doesn't work any more. It's naive to think that you can wrap some HTML around data and be done with the job; the chances are that you're leaving a huge chunk of your human audience behind, and making things more difficult for another audience -- machines that just want to consume your data. To build a modern application, developers must focus on the data: they must see themselves as data providers, they must develop documented and stable public APIs for accessing their data. Over the past few years, we've realized the importance of data. What's the value of Google without the data behind it? Or Facebook? Or, going back 15 or so years, GNN? It took a long time for us to understand the importance of data, as opposed to "content." But when you've gotten that lesson, your design goals change: designing and publishing a stable API to a data service becomes the highest priority.

That's the driving force behind this architectural shift. Front ends, user interfaces, clients, apps, whatever you decide to call them, don't disappear. But we have learned how important it is to keep the data interface separate from the user interface. Your next project will probably have multiple front ends, some delivered through HTML5, and some delivered through native code. Building them on a common data API is going to be much cleaner and simpler. In addition, third parties can build their own apps on top of your API. An important component of Twitter's success has been the ecosystem of applications that have built on their data: TweetDeck, TweetGrid, Tweetie, Twitdom, etc. The Twitter applications directory lists more than 1,800 apps. Until the "new Twitter" went live, the Twitter website was significantly less capable than most of the third-party apps.

Although it has been a long time coming, we're finally finishing the revolution that started with Ajax. Get data that users care about, make it available via an API, provide a data presence that's distinct from your HTML-based web presence, and build multiple front ends to serve your customers, on whatever platforms they care about. Your value is in the data.


September 24 2010

June 16 2010

Four short links: 16 June 2010

  1. So You Want to Be A Consultant -- absolutely spot-on tips for understanding the true business of a consultant. (via Hacker News)
  2. BBYIDX -- a free and open source idea-gathering application written in Ruby, [...] the basis of the Best Buy IdeaX website.
  3. The Git Parable -- The following parable will take you on a journey through the creation of a Git-like system from the ground up. Understanding the concepts presented here will be the most valuable thing you can do to prepare yourself to harness the full power of Git. The concepts themselves are quite simple, but allow for an amazing wealth of functionality to spring into existence. (via Pete Warden)
  4. Ext JS + jQTouch + Raphael = Sencha -- merging some touch and rich graphics libraries and developers. We’re setting up a foundation called Sencha Labs that will hold the copyright and trademarks for all the non-commercial projects affiliated with Sencha. Our license of choice for these projects is, and will continue to be, the MIT license. We will fund maintainers for our non-commercial projects with contributions from Sencha and the communities of these projects. (via bjepson on Twitter)

December 17 2009

The Best and the Worst Tech of the Decade

With only a few weeks left until we close out the 'naughts and move into the teens, it's almost obligatory to take a look back at the best and not-so-best of the last decade. With that in mind, I polled the O'Reilly editors, authors, Friends, and a number of industry movers and shakers to gather nominations. I then tossed them in the trash and made up my own compiled them together and looked for trends and common threads. So here then, in no particular order, are the best and the worst that the decade had to offer.

The Best

AJAX - It's hard to remember what life was like before Asynchronous JavaScript and XML came along, so I'll prod your memory. It was boring. Web 1.0 consisted of a lot of static web pages, where every mouse click was a round trip to the web server. If you wanted rich content, you had to embed a Java applet in the page, and pray that the client browser supported it.

Without the advent of AJAX, we wouldn't have Web 2.0, GMail, or most of the other cloud-based web applications. Flash is still popular, but especially with HTML 5 on the way, even functionality that formerly required a RIA like Flash or Silverlight can now be accomplished with AJAX.

Twitter - When they first started, blogs were just what they said, web logs. In other words, a journal of interesting web sites that the author had encountered. These days, blogs are more like platforms for rants, opinions, essays, and anything else on the writer's mind. Then along came Twitter. Sure, people like to find out what J-Lo had for dinner, but the real power of the 140 character dynamo is that it has brought about a resurgence of real web logging. The most useful tweets consist of a Tiny URL and a little bit of context. Combine that with the use of Twitter to send out real time notices about everything from breaking news to the current specials at the corner restaurant, and it's easy to see why Twitter has become a dominant player.

Ubiquitous WiFi: I want you to imagine you're on the road in the mid-90s. You get to your hotel room, and plop your laptop on the table. Then you get out your handy RJ-11 cord, and check to see if the hotel phone has a data jack (most didn't), or if you'll have to unplug the phone entirely. Then you'd look up the local number for your ISP, and have your laptop dial it, so you could suck down your e-mail at an anemic 56K.

Now, of course, WiFi is everywhere. You may end up having to pay for it, but fast Internet connectivity is available everywhere from your local McDonalds to your hotel room to an airport terminal. Of course, this is not without its downsides, since unsecured WiFi access points have led to all sorts of security headaches, and using an open access point is a risky proposition unless your antivirus software is up to date, but on the whole, ubiquitous WiFi has made the world a much more connected place.

Phones Get Smarter: In the late 90s, we started to see the first personal digital assistants emerge, but this has been the decade when the PDA and the cell phone got married and had a baby called the smartphone. Palm got the ball rolling with the Treos about the same time that Windows Mobile started appearing on phones, and RIM's Blackberry put functional phones in the hands of business, but it was Apple that took the ball and ran for the touchdown with the iPhone. You can argue if the droid is better than the 3GS or the Pre, but the original iPhone was the game-changer that showed what a smartphone really could do, including the business model of the App Store,

The next convergence is likely to be with Netbooks, as more and more of the mini-laptops come with 3G service integrated in them, and VoIP services such as Skype continue to eat into both landline and cellular business.

The Maker Culture: There's always been a DIY underground, covering everything from Ham radio to photography to model railroading. But the level of cool has taken a noticeable uptick this decade, as cheap digital technology has given DIY a kick in the pants. The Arduino lets anyone embed control capabilities into just about anything you can imagine, amateur PCB board fabrication has gone from a messy kitchen sink operation to a click-and-upload-your-design purchase, and the 3D printer is turning the Star Trek replicator into a reality.

Manufacturers cringe in fear as enterprising geeks dig out their screwdrivers. The conventional wisdom was that as electronics got more complex, the "no user serviceable parts" mentality would spell the end of consumer experimentation. But instead, the fact that everything is turning into a computer meant that you could take a device meant for one thing, and reprogram it to do something else. Don't like your digital camera's software? Install your own! Turn your DVR into a Linux server.

Meanwhile, shows like Mythbusters and events like Maker Faire have shown that hacking hardware can grab the public's interest, especially if there are explosions involved.

Open Source Goes Mainstream: Quick! Name 5 open source pieces of software you might have had on your computer in 1999. Don't worry I'll wait...

How about today? Firefox is an easy candidate, as are Open Office, Chrome, Audacity, Eclipse (if you're a developer), Blender, VLC, and many others. Many netbooks now ship with Linux as the underlying OS. Open Source has gone from a rebel movement to part of the establishment, and when you combine increasing end user adoption with the massive amounts of FLOSS you find on the server side, it can be argued that it is the 800 pound Gorilla now.

As Gandhi said, "First they ignore you, then they laugh at you, then they fight you, then you win." When even Microsoft is releasing Open Source code, you know that you're somewhere between the fight and win stages.

Bountiful Resources: 56K modems, 20MB hard drives, 640K of RAM, 2 MHz processors. You don't have to go far back in time for all of these to represent the state of the art. Now, of course, you would have more than that in a good toaster...

Moore's Law continues to drive technology innovation at a breakneck pace, and it seems that related technologies like storage capacity and bandwidth are trying to follow the same curve. Consider that AT&T users gripe about the iPhone's 5GB/month bandwidth cap, a limit that would have taken 10 solid days of transferring to achieve with a dialup connection.

My iPhone has 3,200 times the storage of the first hard drive I ever owned, and the graphics card on my Mac Pro has 16,000 times the memory of my first computer. We can now do amazing things in the palm of our hands, things that would have seemed like science fiction in 1999.

The Worst

SOAP: The software industry has been trying to solve the problem of making different pieces of software talk to each other since the first time there were two programs on a network, and they still haven't gotten it right. RPC, CORBA, EJB, and now SOAP now litter the graveyard of failed protocol stacks.

SOAP was a particularly egregious failure, because it was sold so heavily as the final solution to the interoperatibility problem. The catch, of course, was that no two vendors implemented the stack quite the same way, with the result that getting a .NET SOAP client to talk to a Java server could be a nightmare. Add in poorly spec'd out components such as web service security, and SOAP became useless in many cases. And the WSDL files that define SOAP endpoints are unreadable and impossible to generate by hand (well, not impossible, but unpleasant in the extreme.)

Is it any wonder that SOAP drove many developers into the waiting arms of more useable data exchange formats such as JSON?

Intellectual Property Wars: How much wasted energy has been spent this decade by one group of people trying to keep another group from doing something with their intellectual property, or property they claim was theirs? DMCA takedowns, Sony's Rootkit debacle, the RIAA suing grandmothers, SCO, patent trolls, 09F911029D74E35BD84156C5635688C0, Kindles erasing books, deep packet inspection, Three Strikes laws, the list goes on and on and on...

At the end of the day, the movie industry just had their best year ever, Lady Gaga seems to be doing just fine and Miley Cyrus isn't going hungry, and even the big players in the industry are getting fed up sufficiently with the Trolls to want patent reform. The iTunes store is selling a boatload of music, in spite of abandoning DRM, so clearly people will continue to pay for music, even if they can copy it from a friend.

Unfortunately, neither the RIAA nor the MPAA is going gently into that good night. If anything, the pressure to create onerous legislation has increased in the past year. Whether this is a last gasp or a retrenchment will only be answered in time.

The Cult of Scrum: If Agile is the teachings of Jesus, Scrum is every abuse ever perpetrated in his name. In many ways, Scrum as practiced in most companies today is the antithesis of Agile, a heavy, dogmatic methodology that blindly follows a checklist of "best practices" that some consultant convinced the management to follow.

Endless retrospectives and sprint planning sessions don't mean squat if the stakeholders never attend them, and too many allegedly Agile projects end up looking a lot like Waterfall projects in the end. If companies won't really buy into the idea that you can't control all three variables at once, calling your process Agile won't do anything but drive your engineers nuts.

The Workplace Becomes Ubiquitous: What's the first thing you do when you get home at night? Check your work email? Or maybe you got a call before you even got home. The dark side of all that bandwidth and mobile technology we enjoy today is that you can never truly escape being available, at least until the last bar drops off your phone (or you shut the darn thing off!)

The line between the workplace and the rest of your life is rapidly disappearing. When you add in overseas outsourcing, you may find yourself responding to an email at 11 at night from your team in Bangalore. Work and leisure is blurring together into a gray mélange of existence. "Do you live to work, or work to live," is becoming a meaningless question, because there's no difference.

So what do you think? Anything we missed? Hate our choices? With us 100 percent? Let us know in the comments section below.

December 09 2009

GWT Now With SpeedTracer

speedtracer google

Google is releasing v2 of GWT (pronounced "Gwit") tonight at a Campfire One in Mountain View. The open-source Google Web Toolkit enables developers to code Ajax web apps in Java. This latest release is focused on speed (just like the latest iPhone) and improved dev-designer collaboration.

I was on a call with Bruce Johnson and Andy Bowers to learn more about the release. There are three new major features being released tonight.

Of the three SpeedTracer (screenshot above) seems to have the greatest implications. It is a separate application that allows developers to watch "where time goes" as a page is being loaded. The example Johnson gave was a recent re-write of the AdWords Campaign Manager. There's a 100-row table that is a part of the application. In the first iteration the table took several seconds to load. Using SpeedTracer they realized that the order of the UI statements was delaying DOM translation. They changed a few lines and the table loadtime dropped to half a second.

SpeedTracer only works with Chrome. Google is making a play for the developer. And by focusing on developer tools they are likely to succeed. The speed improvements should translate to other browsers and in reality they have to. Focusing on speed improvements for a currently-minority browser would not win over many devs.

Code Splitting, the second feature, is a way of segmenting out non-essential portions of an Ajax app for future, as-needed download. The example given was that for a mail app you don't always need to be able to write or the settings tab. You could download the code for those functions on demand. Wave apparently uses this functionality to keep their loadtimes down.

Finally, UIBinder is aimed at improving developer-designer workflow. It allows designers to hand over HTML elements in an XML Template. They can also use GWT Widget library for certain elements. Developers can easily plug their application into this XML presentation layer. This is a far cry from being able to use Photoshop mockups to generate code ala Flash Catalyst, but if the design department knows HTML it should be very handy.

GWT is increasingly being used on Google's own applications. It was quite famously used to make Wave. It is also used on the revamped Orkut (I hadn't been there in over a year, it has had quite a makeover), AdWords, Google Squared (labs), Google Profiles, and Moderator (20% project). DoubleClick is being converted to GWT and a lot of Google's internal infrastructure is built using it. Quite notably some of Google's mobile app have been built with it: Mobile Maps and Latitude for the iPhone. It's great to see Google putting it's money where it's mouth is (quite literally with AdWords), but I really wonder if we'll ever see GMail powered by GWT.

In the future I'd be surprised if there wasn't increased support for HTML5 elements such as Canvas. I also expect that they'll add explicit support for mobile app. Of course, using GWT doesn't block a developer from using non-GWT functionality, but having explicit support always helps.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!