Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

Why clouds and web services will continue to take over computing

Series

What are the chances for a free software cloud?

  • Resolving the contradictions between web services, clouds, and open source (12/13)
  • Defining clouds, web services, and other remote computing (12/15)
  • Why clouds and web services will continue to take over computing (12/17)
  • Why web services should be released as free software (12/20)
  • Reaching the pinnacle: truly open web services and clouds (12/22)

Additional posts in this 5-part series are available here.

Previous section:

Definitions: Clouds, web services, and other remote computing

The tech press is intensely occupied and pre-occupied with analyzing the cloud from a business point of view. Should you host your operations in a cloud provider? Should you use web services for office work? The stream of articles and blogs on these subjects show how indisputably the cloud is poised to take over.

But the actual conclusions these analysts reach are intensely
conservative: watch out, count up your costs carefully, look closely
at

regulations and liability issues
that hold you back, etc.
The analysts are obsessed with the cloud, but they're not
encouraging companies to actually use it--or at least
they're saying we'd better put lots of thought into it
first.

My long-term view convinces me we all WILL be in the cloud.
No hope in bucking the trend. The advantages are just too compelling.

I won't try to replicate here the hundreds and hundreds of
arguments and statistics produced by the analysts. I'll just run
quickly over the pros and cons of using cloud computing and web
services, and why they add up to a ringing endorsement. That will help
me get to the question that really concerns this article: what can we
do to preserve freedom in the cloud?

The promise of the cloud shines bright in many projections. The
federal government has committed to a "Cloud First" policy in its
recent

Information Technology reform plan
.
The companies offering IaaS, and Paas, and SaaS promulgate
mouth-watering visions of their benefits. But some of the advantages I
see aren't even in the marketing literature--and some of them, I bet,
could make even a free software advocate come around to appreciating
the cloud.

Advantages of cloud services

The standard litany of reasons for moving to IaaS or PaaS can be
summarized under a few categories:

Low maintenance

No more machine rooms, no more disk failures (that is, disk failures you know about and have to deal with), no more late-night calls to go in and reboot a critical server.

These simplifications, despite the fears of some Information
Technology professionals, don't mean companies can fire their system
administrators. The cloud still calls for plenty of care and
feeding. Virtual systems go down at least as often as physical ones,
and while the right way to deal with system failures is to automate
recovery, that takes sophisticated administrators. So the system
administrators will stay employed and will adapt. The biggest change
will be a shift from physical system management to diddling with
software; for an amusing perspective on the shift see my short story

Hardware Guy
.


Fast ramp-up and elasticity

To start up a new operation, you no longer have to wait for hardware to arrive and then lose yourself in snaking cables for hours. Just ask the cloud center to spin up as many virtual systems as you want.

Innovative programmers can also bypass IT management, developing new
products in the cloud. Developers worry constantly whether their
testing adequately reproduces the real-life environment in which
production systems will run, but if both the test systems and the
final production systems run in the cloud, the test systems can match
the production ones much more closely.

The CIO of O'Reilly Media, citing the goal of directing

60 percent of IT spending into new projects
,
has made internal and external cloud computing into pillars of

O'Reilly's IT strategy
.

Because existing companies have hardware and systems for buying
hardware in place already, current cloud users tend to come from
high-tech start-ups. But any company that wants to launch a new
project can benefit from the cloud. Peaks and troughs in usage can
also be handled by starting and stopping virtual systems--you
just have to watch how many get started up, because a lack of
oversight can incur run-away server launches and high costs.

Cost savings

In theory, clouds provide economies of scale that undercut anything an individual client could do on their own. How can a private site, chugging away on a few computers, be more efficient than thousands of fungible processors in one room under the eye of a highly trained expert, all strategically located in an area with cheap real estate and electricity?

Currently, the cost factor in the equation is not so cut and dried.
Running multiple servers on a single microprocessor certainly brings
savings, although loads have to be balanced carefully to avoid slowing
down performance unacceptably. But running processors constantly
generates heat, and if enough of them are jammed together the costs of
air conditioning could exceed the costs of the computers. Remote
computing also entails networking costs.

It will not take long, however, for the research applied by cloud
vendors to pay off in immense efficiencies that will make it hard for
organizations to justify buying their own computers.


Elasticity and consolidation make IaaS so attractive that large
companies are trying to build "private clouds" and bring all the
organization's server hardware into one department, where the
hardware is allocated as virtual resources to the rest of the company.
These internal virtualization projects don't incur some of the
disadvantages that this paper address, so I won't consider them
further.

Advantages of web services

SaaS offers some benefits similar to IaaS and PaaS, but also
significant differences.

Low maintenance

No more installation, no more upgrades, no more incompatibilities with other system components or with older versions of the software on other people's systems. Companies licensing data, instead of just buying it on disks, can access it directly from the vendor's site and be sure of always getting the most recent information.


Fast ramp-up and elasticity

As with IaaS, SaaS frees staff from running every innovation past the IT group. They can recreate their jobs and workflows in the manner they want.


Feedback

To see what's popular and to prioritize future work, companies love to know how many people are using a feature and how long they spend in various product functions. SaaS makes this easy to track because it can log every mouse click.

Enough of the conventional assessment. What hidden advantages lie in clouds and web services?

What particularly should entice free and open software software advocates is web services' prospects for making money. Although free software doesn't have to be offered cost-free (as frequently assumed by those who don't know the field), there's no way to prevent people from downloading and installing it, so most of the money in free software is made through consulting and additional services. Web services allow subscriptions instead, a much more stable income. Two popular content management systems exemplify this benefit: WordPress offers hosting at wordpress.com and Drupal at drupalgardens.com, all while offering their software as open source.

But I find another advantage to web services. They're making
applications better than they ever have been in the sixty-year history
of application development.

Compare your own experiences with stand-alone software to web sites. The quality of the visitor's experience on a successful web site is much better. It's reminiscent of the old cliché about restaurant service in capitalist versus socialist economies.

According to this old story, restaurants in capitalist countries
depend on repeat business from you and your friends, driving the
concern for delivering a positive customer experience from management
down to the lowest level of the wait staff. In a socialist economy,
supposedly, the waiters know they will get paid no matter whether you
like their service or not, so they just don't try. Furthermore,
taking pains to make you happy would be degrading to them as heroes of
a workers' society.

I don't know whether this phenomenon is actually true of restaurants,
but an analogous dynamic holds in software. Web sites know that
visitors will vanish in half a second if the experience is not
immediately gripping, gratifying, and productive. Every hour of every
day, the staff concentrate on the performance and usability of the
site. Along with the business pressure on web services to keep users
on the page, the programmers there can benefit from detailed feedback
about which pages are visited, in which order, and for how long.

In contrast, the programmers of stand-alone software measure
their personal satisfaction by the implementation of complex and
sophisticated calculations under the product's surface. Creating
the user interface is a chore relegated to less knowledgeable staff.

Whatever the reason, I find the interfaces of proprietary as well as
free software to be execrable, and while I don't have statistics to
bolster my claim. I think most readers can cite similar experiences.
Games are the main exception, as well as a few outstanding consumer
applications, but these unfortunately do not seem a standard for the
vast hoards of other programmers to follow.

Moving one's aching fingers from stand-alone software to a web
service brings a sudden rush of pleasure, affirming what working with
computers can be. A bit of discipline in the web services world would
be a good cold bath for the vendors and coders.


Drawbacks of clouds and web services

So why are the analysts and customers still wary of cloud computing? They have their reasons, but some dangers are exaggerated.

Managers responsible for sensitive data feel a visceral sense of vulnerability when they entrust that data to some other organization. Web services have indeed had breaches, because they are prisoners of the twin invariants that continue to ensure software flaws: programmers are human, and so are administrators. Another risk comes when data is transmitted to a service such as Amazon.com's S3, a process during which it be seen or even in theory altered.

Still, I expect the administrators of web and cloud services to be better trained and more zealous in guarding against security breaches than the average system administrator at a private site. The extra layer added by IaaS also creates new possibilities. An article called "Security in the Cloud" by Gary Anthes, published in the November 2010 Communications of the ACM, points to research projects by Hewlett-Packard and IBM that would let physical machines monitor the virtual machines running on them for viruses and other breaches of security, a bit like a projectionist can interrupt a movie.

A cloud or web service provider creates some risk just because it
provides a tasty target to intruders, who know they can find thousands
of victims in one place. On the other hand, if you put your data in
the cloud, you aren't as likely to lose it to some drive-by
trouble-seeker picking it up off of a wireless network that your
administrator failed to secure adequately, as famously happened to
T.J. Maxx (and they weren't alone).

And considering that security experts suspect most data breaches to be
internal, putting data in the cloud might make it more secure by
reducing its exposure to employees outside of the few programmers or
administrators with access rights. If the Department of Defense had
more systems in the cloud, perhaps it wouldn't have suffered such a
sinister security breach in 2008 through a

flash drive with a virus
.

In general, the solution to securing data and transactions is to
encrypt everything. Encrypting the operating systems loaded in IaaS,
for instance, gives the client some assurance that no one can figure
out what it's doing in the cloud, even if another client or even the
vendor itself tries to snoop. If some technological earthquake
undermines the integrity of encryption technologies--such as the
development of a viable quantum computer--we'll have to rethink the
foundations of the information age entirely anyway.

The main thing to remember is that most data breaches are caused by
lapses totally unrelated to how servers are provisioned: they happen
because staff stored unencrypted data on laptops or mobile devices,
because intruders slipped into applications by exploiting buffer
overflows or SQL injection, and so on. (See, for instance, a
U.S. Health & Human Services study saying that
"Laptop theft
is the most prevalent cause of the breach of health information
affecting more than 500 people.
")

Regulations such as HIPAA can rule out storing some data off-site, and
concerns about violating security regulations come up regularly during
cloud discussions. But these regulations affect only a small amount of
the data and computer operations, and the regulations can be changed
once the computer industry shows that clouds are both valuable and
acceptably secure.

Bandwidth is a concern, particularly in less technologically developed
parts of the world (like much of the United States, come to think of
it), where bandwidth is inadequate. But in many of these areas, people
often don't even possess computers. SaaS is playing a major role
in underdeveloped areas because it leverages the one type of computer
in widespread use (the cell phone) and the one digital network
that's widely available (the cellular grid). So in some ways,
SaaS is even more valuable in underdeveloped areas, just in a
different form from regions with high bandwidth and universal access.

Nevertheless, important risks and disadvantages have been identified
in clouds and web services. IaaS and PaaS are still young enough (and
their target customers sophisticated enough) for the debate to keep up
pretty well with trends; in contrast, SaaS has been crying out quite a
while for remedies to be proposed, such as the
best practices
recently released by the Consumer Federation of America. This article
will try to raise the questions to a higher level, to find more
lasting solutions to problems such as the following.

Availability

Every system has down time, but no company wants to be at the mercy of a provider that turns off service, perhaps for 24 hours or more, because they failed to catch a bug in their latest version or provide adequate battery backup during a power failure.

When Wikileaks was forced off of Amazon.com's cloud service, it sparked outrage whose echo reached as far as a Wall Street Journal blog and highlighted the vulnerability of depending on clouds. Similarly, the terms of service on social networks and other SaaS sites alienate some people who feel they have legitimate content that doesn't pass muster on those sites.

Liability

One of the big debates in the legal arena is how to apportion blame when a breach or failure happens in a cascading service, where one company leases virtual systems in the cloud to provide a higher-level service to other companies.


Reliability

How can you tell whether the calculation that a service ran over your corporate data produced the correct result? This is a lasting problem with proprietary software, which the free software developers argue they've solved, but which most customers of proprietary software have learned to live with and which therefore doesn't turn them against web services.

But upgrades can present a problem. When a new version of stand-alone
software comes out, typical consumers just click "Yes" on the upgrade
screen and live with the consequences. Careful system administrators
test the upgrade first, even though the vendor has tested it, in case
it interacts perniciously with some factor on the local site and
reveals a bug. Web services reduce everyone to the level of a passive
consumer by upgrading their software silently. There's no
recourse for clients left in the lurch.


Control

Leaving the software on the web service's site also removes all end-user choice. Some customers of stand-alone software choose to leave old versions in place because the new version removed a feature the customers found crucial, or perhaps just because they didn't want the features in the new version and found its performance worse. Web services offer one size to fit all.

Because SaaS is a black box, and one that can change behavior without
warning to the visitors, it can provoke concerns among people
sensitive about consistency and reliability. See my article

Results from Wolfram Alpha: All the Questions We Ever Wanted to Ask About Software as a Service
.

Privacy

Web services have been known to mine customer data and track customer behavior for marketing purposes, and have given data to law enforcement authorities. It's much easier to monitor millions of BlackBerry messages traveling through a single server maintained by the provider than the messages bouncing in arbitrary fashion among thousands of Sendmail servers. If a customer keeps the data on its own systems, law enforcement can still subpoena it, but at least the customer knows she's being investigated.

In the United States, furthermore, the legal requirements that investigators must meet to get data is higher for customers' systems than for data stored on a third-party site such as a web service. Recent Congressional hearings (discussed on O'Reilly's Radar site highlighted the need to update US laws to ensure privacy for cloud users).


These are knotty problems, but one practice could tease them apart:
making the software running clouds or web services open source.

A number of proponents for this viewpoint can be found, such as the Total Information Outsourcing group, as well as a few precedents. Besides the WordPress and Drupal services mentioned earlier, StatusNet runs the microblogging site identi.ca and opens up its code so that other people could run sites that interoperate with it. Source code for Google's AppEngine, mentioned earlier as a leading form of IaaS, has been offered for download by Google under a free license. Talend offers data integration and business intelligence as both free software and SaaS.

The Free Software Foundation, a leading free software organization that provides a huge amount of valuable software to Linux and other systems through the GNU project, has created a license called the GNU Affero General Public License that encourages open code for web services. When sites such as StatusNet release code under that license, other people are free to build web services on it but must release all their enhancements and bug fixes to the world as well.

What problems can be ameliorated by freeing the cloud and web service software? Can the companies who produced that software be persuaded to loosen their grip on the source code? And what could a world of free cloud and web services look like? That is where we will turn next.

Next section:
Why web services should be released as free software.

Get rid of the ads (sfw)

Don't be the product, buy the product!

Schweinderl