Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

July 18 2013

BGH schickt Streit um Gebrauchtsoftware zurück nach München

Der Bundesgerichtshof hat ein Urteil des Oberlandesgerichts München aufgehoben, das dem Händler Usedsoft verboten hatte, gebrauchte Software-Lizenzen von Oracle weiterzuverkaufen. Nachdem inzwischen auch der Europäische Gerichtshof für den Weiterverkauf entschieden hatte, muss das Oberlandesgericht jetzt neu darüber befinden.

Seit 2006 streiten Oracle und Usedsoft inzwischen darüber, ob der Handel mit Gebrauchtlizenzen von Oracle-Software Urheberrechte verletzt. Vor dem Landgericht München, vor dem Oberlandesgericht München und vor dem Bundesgerichtshof, der sich schließlich an den Europäischen Gerichtshof gewandt hatte, um die europarechtlichen Fragen prüfen zu lassen. Am Mittwoch hat der BGH das Urteil des Oberlandesgerichts München gegen den Weiterverkauf aufgehoben, das nun erneut darüber befinden muss. Die Entscheidung selbst ist noch nicht veröffentlicht.

Der eigentliche Paukenschlag kam bereits mit dem Urteil des Europäischen Gerichtshofs vor einem Jahr (C–128/11): Auch digital gekaufte Software darf weiterverkauft werden, wenn man mit ihr eine zeitlich unbegrenzte Lizenz erworben hat. Wer eine Software erwirbt, kauft demnach auch im Wortsinn. Softwarehersteller vertreten dagegen die Position, dass der Kunde nur ein Nutzungsrecht erwirbt, das in den AGB oder Endbenutzer-Lizenzverträgen häufig deutlich eingeschränkt wird.

„Erschöpfen” sich digitale Güter?

Rechtlich geht es im Streit vor allem um den sogenannten Erschöpfungsgrundsatz. Er besagt, dass das Verbreitungsrecht des Herstellers endet, wenn er einen Kaufgegenstand rechtmäßig auf den europäischen Markt gebracht hat („Inverkehrbringen”). Ein Rechteinhaber kann dadurch nicht den gesamten Lebenszyklus des Produkts kontrollieren, weil das – so die Begründung – den Käufer als Eigentümer, aber auch den freien Warenverkehr übermäßig beschränken würde. Softwarehersteller wiederum befürchten sinkende Umsätze durch einen unkontrollierbaren Gebrauchtmarkt.

Durch den „Erschöpfungsgrundsatz” können Verbraucher Bücher, CDs und andere körperliche Werkexemplare zum Trödelmarkt tragen, für digitale Downloads aber bleibt der Weiterverkauf insgesamt umstritten. Zumindest für Software hatten die Luxemburger Richter klargestellt, dass es keine Rolle spielt, ob die Käufer sie direkt von der Hersteller-Seite herunterladen oder von einer CD-ROM oder DVD kopieren.

Signalwirkung hatte das EuGH-Urteil zwar auch für Musikdateien, E‑Books oder Filme, direkt anwendbar ist es aber nicht: Für Software gibt es eine eigene EU-Richtlinie, die der EuGH ausgelegt hatte. Für andere Inhalte sind vor allem die EU-Urheberrechtsrichtlinie und weitere Regelungen maßgebend.

Wer darf mit digitalen Gütern handeln?

Wer digitale Inhalte weiterverkaufen kann, darüber streiten mittlerweile Rechteinhaber, Plattformbetreiber, Händler und Nutzer. Für Unternehmen ist der Verkauf von überschüssigen Softwarelizenzen zum Beispiel bei Umstrukturierungen und anderen Veränderungen lohnend; Nutzer würden ebenso profitieren, wenn sie Software oder andere Inhalte weiterverkaufen können. Einen Gesetzentwurf „zur Ermöglichung der privaten Weiterveräußerung unkörperlicher Werkexemplare“ hatte letztes Jahr die Linke vorgestellt, Rechteinhaber-Verbände wie der Börsenverein des deutschen Buchhandels protestierten.

Zugleich setzen Softwarehersteller zunehmend auf Mietmodelle und Cloud-Anbindung für ihre Programme. Auch bei Musik oder Filmen könnte der Zugang per Streaming den Besitz ablösen. Der Weiterverkauf würde dann nur einen kleinen Teil des Marktes betreffen, selbst wenn sich die nutzerfreundliche Rechtsprechung durchsetzt.

Amazon und Apple wiederum haben eigene Weiterverkaufs-Systeme mit entsprechendem Rechtemanagement als Patent angemeldet – ob sie tatsächlich umgesetzt werden, ist offen.

Volumenlizenzen, Updates und Support

Das Oberlandesgericht muss jetzt also erneut entscheiden, ob die Bedingungen für den Weiterverkauf auch im konkreten Fall von Usedsoft erfüllt sind – unter anderem, ob die ursprünglichen Käufer zeitlich unbegrenzte Lizenzen erworben haben. Der Bundesgerichtshof weist außerdem darauf hin, dass der Weiterverkauf nur dann erlaubt ist, wenn der Erstverkäufer seine Kopie unbrauchbar gemacht hat.

Eine weitere offene Frage dürfte sich daraus ergeben, dass Usedsoft auch mit Volumenlizenzen handelt. Der EuGH hatte sich kritisch dazu gezeigt, solche Lizenzbündel aufzuspalten und einzeln zu verkaufen. Anders hatte zuletzt aber das Oberlandesgericht Frankfurt in einem ähnlichen Streit über Lizenzen der Adobe Creative Suite entschieden.

Auch die Frage, wie weit Updates und Support für den Gebrauchtkäufer bereit stehen, könnte die Münchner Richter beschäftigen. Der EuGH hatte nahe gelegt, dass ein Wartungsvertrag nicht zum Gebrauchtkäufer mitwandert, Updates aber zum Kaufgegenstand gehören. Usedsoft steht auf dem Standpunkt, ein Hersteller könne die Wartung nicht verweigern, ohne „schwerwiegende rechtliche Bedenken” hervorzurufen. Für die Münchner Richter bleibt auch nach dem Urteil des EuGH noch Einiges zu prüfen.

July 04 2012

EuGH-Urteil zu Gebrauchtsoftware: Eine revolutionäre Entscheidung für die Informationsgesellschaft

Der Europäische Gerichtshof hat entschieden: Der Weiterverkauf von Software ist auch dann legal, wenn sie im Rahmen einer dauerhaften Nutzungslizenz erworben und aus dem Netz herunterg

Weiterlesen

June 01 2012

May 25 2012

Developer Week in Review: Oracle's big bet fails to pay off

I've been taking the opportunity this week to do some spring office cleaning. Unfortunately, I clean my home office infrequently enough that at a certain point, cleaning it becomes more an exercise in archeology than organization. There's nothing like finding a six-month-old check you never deposited to encourage more frequent cleaning.

The same can be said for code, of course. It's far too easy to let crufty code build up in an application, and then be faced with the mother of all refactoring efforts six months down the road, when your code finally reaches a critical mass of flaky behavior. It's worth the effort to continually refactor and improve your code, assuming you can convince your product management that quality is as important as new features.

Android is almost out of the woods

It wouldn't be a Week in Review without the latest in Godzilla vs. Gamera Oracle vs. Google. Things aren't looking all too sunny for Oracle at the moment, as the jury in the case just threw out all the patent-related claims in the lawsuit. This doesn't leave Oracle with much left on the plate, as the case now boils down to the question of whether the Java APIs are copyrightable. That's a matter the jury is deadlocked on.

Like all things legal, this is going to drag on for years as there are appeals and retrials and the like. But for the moment, it appears that Android is out of the woods, at least as far as the use of Java is concerned. Of course, there's still all those pesky International Trade Commission issues keeping many Android handsets waiting at the border, but that's a battle for another day ...

Scripters of the world, rejoice!

For Perl developers, a point release of the language is a major event, as it only occurs roughly once a year. This year's edition has just been released, and Perl 5.16 packs a ton of improvements (nearly 600,000 lines' worth!).

Since Perl is such a mature language, most of the changes are incremental. Probably the most significant is further enhancements in Unicode support. Nonetheless, there should be something useful for the serious Perl developer.

FreeBSD bids GCC farewell

As the licensing on the GCC compiler has become increasingly restrictive, some of us have been wondering when the fallout would start. Wait no longer: The FreeBSD team has ditched GCC for the more BSD-friendly licensing of Clang.

GCC has spent decades as the compiler of choice for just about everything, but recent changes in the GPL have made it less attractive to use, especially in commercial development. With the Apple-sponsored Clang compiler now seen as a viable (and perhaps even superior) alternative, with a much less restrictive license, the Free Software Foundation may need to decide if it would rather stand on principle, or avoid becoming marginalized.

Got news?

Please send tips and leads here.

OSCON 2012 — Join the world's open source pioneers, builders, and innovators July 16-20 in Portland, Oregon. Learn about open development, challenge your assumptions, and fire up your brain.

Save 20% on registration with the code RADAR20


Related:


May 16 2012

A federal judge learned to code

The last couple of days, there's been a fair amount of blogosphere angst over Coding Horror's "Please Don't Learn to Code." Ironically, the best argument for learning to code appeared this morning, when it turned out that Judge William Alsup in the Google case could program, and learned Java in the course of the trial, and wasn't going for Oracle's claim that a short range-checking function was days of work. Alsup recognized immediately (and says he wrote the function hundreds of times during the course of the trial) that it's just a few minutes work for a competent programmer.

The importance of learning to code isn't so that everyone will write code, and bury the world under billions of lines of badly conceived Python, Java, and Ruby. The importance of code is that it's a part of the world we live in. I've had enough of legislators who think the Internet is about tubes, who haven't the slightest idea about legitimate uses for file transfer utilities, and no concept at all about what privacy (and the invasion of privacy) might mean in an online space. I've had enough of patent inspectors who approve patents for which prior art has existed for decades. And I've had enough of judges making rulings after listening to lawyers arguing about technologies they don't understand. Learning to code won't solve these problems, but coding does force engagement with technology on a level other than pure ignorance.

Coding is a part of cultural competence, even if you never do it professionally. Alsup is a modern hero.

May 04 2012

Developer Week in Review: Are APIs intellectual property?

Returning after a brief hiatus due to my annual spring head cold, welcome back to your weekly dose of all things programming. Last week, I was attending the Genomes, Environments and Traits conference (I'm a participant in the Personal Genome Project), when I got notified that WWDC registration had opened up. I ended up having to type in my credit card information on my iPhone while listening to the project organizers discuss what they were doing with the saliva I had sent them. The conference itself was very interesting (although I was coming down with the aforementioned cold, so I wasn't at the top of my game). The cost to sequence a genome is plummeting — it's approaching $1,000 a pop — and it has the potential to totally revolutionize how we think about health care.

It's also an interesting example of big data, but not how we normally think about it. An individual genome isn't all that big in the scheme of things (it's about 3GB uncompressed per genome), but there are huge computational challenges involved in relating individual variations in the genome to phenotype variations (in other words, figuring out what variations are responsible for traits or diseases).

While all the West Coast developers who slept through the WWDC registration period lick their wounds, here's the rest of the news.

APIs are copyrightable, unless they aren't?

These days, I feel like you need to consider a minor in law to go with your computer science degree. In the latest news from the front, we have conflicting opinions regarding the status of APIs. On the one hand, the judge in the Oracle versus Google lawsuit has instructed the jury they should assume that APIs are copyrightable. As the linked article discusses, this could have ominous implications for any third-party re-implementation of a programming language or other software that is not open source.

Over in Europe, however, a new ruling has stated that programming languages and computer functionality are not copyrightable. So, depending on which side of the ocean you live on, APIs are either open season, or off limits. No word yet as to the legal status of APIs on the Falkland Islands ...

Fluent Conference: JavaScript & Beyond — Explore the changing worlds of JavaScript & HTML5 at the O'Reilly Fluent Conference (May 29 - 31 in San Francisco, Calif.).

Save 20% on registration with the code RADAR20

Code to make your head hurt.

For those of you who like to celebrate the perversities of life, it's hard to beat the International Obfuscated C Competition, which just released its 2011 winners. For your viewing pleasure, we have programs that compute pi, chart histograms, and even judging programs for obfuscation, all written in a manner that will have code reviewers running to the toilet with terminal bouts of nausea.

And speaking of C ...

We tend to focus a lot of attention on emerging languages, partially because many of them have novel features, and partially because the grass is always greener in a different language. It's instructive to step back sometimes and take a look at what people are actually using. The latest TIOBE Programming Community Index, which measures how much code there is out there in each of the various languages, has a new top dog, and it's our old friend C. In fact, when you factor in C#, C++ and Objective-C, C-related languages pretty much own the category. Java has now fallen to the second position, and you have to go all the way down to sixth place to find a scripting language, PHP.

Importantly, all the hot new languages, like Erlang and Scala, don't even make the top 20, and you only need half-a-percentage point to get in that list. As much as we like the new darlings on the block, the old veterans still are where most of the action (and money) is.

Got news?

Please send tips and leads here.

Related:

Reposted bynunatak nunatak

April 14 2012

MySQL in 2012: Report from Percona Live

The big annual MySQL conference, started by MySQL AB in 2003 and run
by my company O'Reilly for several years, lives on under the able
management of Percona. This
fast-growing company started out doing consulting on MySQL,
particularly in the area of performance, and branched out into
development and many other activities. The principals of this company
wrote the most recent two editions of the popular O'Reilly book href="http://shop.oreilly.com/product/0636920022343.do">High
Performance MySQL
.

Percona started offering conferences a couple years ago and decided to
step in when O'Reilly decided not to run the annual MySQL conference
any more. Oracle did not participate in Percona Live, but has
announced href="http://www.oracle.com/us/corporate/press/1577449">its own MySQL
conference for next September.

Percona Live struck me as a success, with about one thousand attendees
and the participation of leading experts from all over the MySQL
world, save for Oracle itself. The big players in the MySQL user
community came out in force: Facebook, HP, Google, Pinterest (the
current darling of the financial crowd), and so on.

The conference followed the pattern laid down by old ones in just
about every way, with the same venue (the Santa Clara Convention
Center, which is near a light-rail but nothing else of interest), the
same food (scrumptious), the standard format of one day of tutorials
and two days of sessions (although with an extra developer day tacked
on, which I will describe later), an expo hall (smaller than before,
but with key participants in the ecosystem), and even community awards
(O'Reilly Media won an award as Corporate Contributor of the Year).
Monty Widenius was back as always with a MariaDB entourage, so it
seemed like old times. The keynotes seemed less well attended than the
ones from previous conferences, but the crowd was persistent and
showed up in impressive numbers for the final events--and I don't
believe it was because everybody thought they might win one of the
door prizes.

Jeremy Zawodny ready to hand out awards
Jeremy Zawodny ready to hand out awards.

Two contrasting database deployments

I checked out two well-attended talks by system architects from two

high-traffic sites: Pinterest and craigslist. The radically divergent
paths they took illustrate the range of options open to data centers
nowadays--and the importance of studying these options so a data
center can choose the path appropriate to its mission and
applications.

Jeremy Zawodny (co-author of the first edition of High Performance
MySQL
) href="http://www.percona.com/live/mysql-conference-2012/sessions/living-sql-and
-nosql-craigslist-pragmatic-approach">presented
the design of craigslist's site, which illustrates the model of
software accretion over time and an eager embrace of heterogeneity.
Among their components are:


  • Memcache, lying between the web servers and the MySQL database in
    classic fashion.

  • MySQL to serve live postings, handle abuse, data for monitoring
    system, and other immediate needs.

  • MongoDB to store almost 3 billion items related to archived (no longer
    live) postings.

  • HAproxy to direct requests to the proper MySQL server in a cluster.

  • Sphinx for text searches, with

    indexes over all live postings, archived postings, and forums.

  • Redis for temporary items such as counters and blobs.

  • An XFS filesystem for images.

  • Other helper functions that Zawodny lumped together as "async
    services."


Care and feeding of this menagerie becomes a job all in itself.
Although craigslist hires enough developers to assign them to
different areas of expertise, they have also built an object layer
that understands MySQL, cache, Sphinx, MongoDB. The original purpose
of this layer was to aid in migrating old data from MySQL to MongoDB
(a procedure Zawodny admitted was painful and time-consuming) but it
was extended into a useful framework that most developers can use
every day.

Zawodny praised MySQL's durability and its method of replication. But
he admitted that they used MySQL also because it was present when they
started and they were familiar with it. So adopting the newer entrants
into the data store arena was by no means done haphazardly or to try
out cool new tools. Each one precisely meets particular needs of the
site.


For instance, besides being fast and offering built-in sharding,
MongoDB was appealing because they don't have to run ALTER TABLE every
time they add a new field to the database. Old entries coexist happily
with newer ones that have different fields. Zawodny also likes using a
Perl client to interact with a database, and the Perl client provided
by MongoDB is unusually robust because it was developed by 10gen
directly, in contrast to many other datastores where Perl was added by
some random volunteer.

The architecture at craigslist was shrewdly chosen to match their
needs. For instance, because most visitors click on the limited set
of current listings, the Memcache layer handles the vast majority of
hits and the MySQL database has a relatively light load.


However, the MySQL deployment is also carefully designed. Clusters are
vertically partitioned in two nested ways. First, different types of
items are stored on separate partitions. Then, within each type, the
nodes are further divided by the type of query:

  • A single master to handle all writes.

  • A group for very fast reads (such as lookups on a primary key)

  • A group for "long reads" taking a few seconds


  • A special node called a "thrash handler" for rare, very complex
    queries

It's up to the application to indicate what kind of query it is
issuing, and HAproxy interprets this information to direct the query
to the proper set of nodes.

Naturally, redundancy is built in at every stage (three HAproxy
instances used in round robin, two Memcache instances holding the same
data, two data centers for the MongoDB archive, etc.).


It's also interesting what recent developments have been eschewed by
craigslist. The self-host everything and use no virtualization.
Zawodny admits this leads to an inefficient use of hardware, but
avoids the overhead associated with virtualization. For efficiency,
they have switched to SSDs, allowing them to scale down from 20
servers to only 3. They don't use a CDN, finding that with aggressive
caching and good capacity planning they can handle the load
themselves. They send backups and logs to a SAN.

Let's turn now from the teeming environment of craigslist to the
decidedly lean operation of Pinterest, a much younger and smaller
organization. As href="http://www.percona.com/live/mysql-conference-2012/sessions/scaling-pinterest">presented
by Marty Weiner and Yashh Nelapati, when they started web-scale
growth in the Autumn of 2011, they reacted somewhat like craigslist,
but with much less thinking ahead, throwing in all sorts of software
such as Cassandra and MongoDB, and diversifying a bit recklessly.
Finally they came to their senses and went on a design diet. Their
resolution was to focus on MySQL--but the way they made it work is
unique to their data and application.

They decided against using a cluster, afraid that bad application code
could crash everything. Sharding is much simpler and doesn't require
much maintenance. Their advice for implementing MySQL sharding
included:

  • Make sure you have a stable schema, and don't add features for a
    couple months.


  • Remove all joins and complex queries for a while.

  • Do simple shards first, such as moving a huge table into its own
    database.

They use Pyres, a
Python clone of Resque, to move data into shards.

However, sharding imposes severe constraints that led them to
hand-crafted work-arounds.

Many sites want to leave open the possibility for moving data between
shards. This is useful, for instance, if they shard along some
dimension such as age or country, and they suddenly experience a rush
of new people in their 60s or from China. The implementation of such a
plan requires a good deal of coding, described in the O'Reilly book href="http://shop.oreilly.com/product/9780596807290.do">MySQL High
Availability
, including the creation of a service that just
accepts IDs and determines what shard currently contains the ID.

The Pinterest staff decided the ID service would introduce a single
point of failure, and decided just to hard-code a shard ID in every ID
assigned to a row. This means they never move data between shards,
although shards can be moved bodily to new nodes. I think this works
for Pinterest because they shard on arbitrary IDs and don't have a
need to rebalance shards.

Even more interesting is how they avoid joins. Suppose they want to
retrieve all pins associated with a certain board associated with a
certain user. In classical, normalized relational database practice,
they'd have to do a join on the comment, pin, and user tables. But
Pinterest maintains extra mapping tables. One table maps users to
boards, while another maps boards to pins. They query the
user-to-board table to get the right board, query the board-to-pin
table to get the right pin, and then do simple queries without joins
on the tables with the real data. In a way, they implement a custom
NoSQL model on top of a relational database.

Pinterest does use Memcache and Redis in addition to MySQL. As with
craigslist, they find that most queries can be handled by Memcache.
And the actual images are stored in S3, an interesting choice for a
site that is already enormous.

It seems to me that the data and application design behind Pinterest
would have made it a good candidate for a non-ACID datastore. They
chose to stick with MySQL, but like organizations that use NoSQL
solutions, they relinquished key aspects of the relational way of
doing things. They made calculated trade-offs that worked for their
particular needs.

My take-away from these two fascinating and well-attended talks was
that how you must understand your application, its scaling and
performance needs, and its data structure, to know what you can
sacrifice and what solution gives you your sweet spot. craigslist
solved its problem through the very precise application of different
tools, each with particular jobs that fulfilled craigslist's

requirements. Pinterest made its own calculations and found an
entirely different solution depending on some clever hand-coding
instead of off-the-shelf tools.

Current and future MySQL

The conference keynotes surveyed the state of MySQL and some
predictions about where it will go.

Conference co-chair Sarah Novotny at keynote
Conference co-chair Sarah Novotny at keynotes.

The world of MySQL is much more complicated than it was a couple years
ago, before Percona got heavily into the work of releasing patches to
InnoDB, before they created entirely new pieces of software, and
before Monty started MariaDB with the express goal of making a better
MySQL than MySQL. You can now choose among Oracle's official MySQL
releases, Percona's supported version, and MariaDB's supported
version. Because these are all open source, a major user such as

Facebook can even apply patches to get the newest features.

Nor are these different versions true forks, because Percona and
MariaDB create their enhancements as patches that they pass back to
Oracle, and Oracle is happy to include many of them in a later
release. I haven't even touched on the commercial ecosystem around
MySQL, which I'll look at later in this article.

In his href="http://www.percona.com/live/mysql-conference-2012/sessions/keynote-mysql-
evolution">opening
keynote, Percona founder Peter Zaitsev praised the latest MySQL
release by Oracle. With graceful balance he expressed pleasure that
the features most users need are in the open (community) edition, but
allowed that the proprietary extensions are useful too. In short, he
declared that MySQL is less buggy and has more features than ever.


The href="http://www.percona.com/live/mysql-conference-2012/sessions/keynote-making-lamp-cloud">former
CEO of MySQL AB, Mårten Mickos, also found that MySQL is
doing well under Oracle's wing. He just chastised Oracle for failing
to work as well as it should with potential partners (by which I
assume he meant Percona and MariaDB). He lauded their community
managers but said the rest of the company should support them more.

Keynote by Mårten Mickos
Keynote by Mårten Mickos.

href="http://www.percona.com/live/mysql-conference-2012/sessions/keynote-new-mysql-cloud-ecosystem">Brian
Aker presented an OpenStack MySQL service developed by his current
employer, Hewlett-Packard. His keynote retold the story that had led
over the years to his developing href="https://launchpad.net/drizzle">Drizzle (a true fork of MySQL
that tries to return it to its lightweight, Web-friendly roots) and
eventually working on cloud computing for HP. He described modularity,
effective use of multiple cores, and cloud deployment as the future of
databases.

A href="http://www.percona.com/live/mysql-conference-2012/sessions/future-perfect
-road-ahead-mysql">panel
on the second day of the conference brought together high-level
managers from many of the companies that have entered the MySQL space
from a variety of directions in a high-level discussion of the
database engine's future. Like most panels, the conversation ranged
over a variety of topics--NoSQL, modular architecture, cloud
computing--but hit some depth only on the topic of security, which was
not represented very strongly at the conference and was discussed here
at the insistence of Slavik Markovich from McAfee.

Keynote by Brian Aker
Keynote by Brian Aker.


Many of the conference sessions disappointed me, being either very
high level (although presumably useful to people who are really new to
various topics, such as Hadoop or flash memory) or unvarnished
marketing pitches. I may have judged the latter too harshly though,
because a decent number of attendees came, and stayed to the end, and
crowded around the speakers for information.

Two talks, though, were so fast-paced and loaded with detail that I
couldn't possibly keep my typing up with the speaker.

One such talk was the href="http://www.percona.com/live/mysql-conference-2012/sessions/keynote-what-c
omes-next">keynote
by Mark Callaghan of Facebook. (Like the other keynotes, it should be
posted online soon.) A smattering of points from it:


  • Percona and MariaDB are adding critical features that make replication
    and InnoDB work better.

  • When a logical backup runs, it is responsible for 50% of IOPS.

  • Defragmenting InnoDB improves compression.

  • Resharding is not worthwhile for a large, busy site (an insight also
    discovered by Pinterest, as I reported earlier)


The other fact-filled talk was href="http://www.percona.com/live/mysql-conference-2012/sessions/using-nosql-in
nodb-memcached">by
Yoshinori Matsunobu of Facebook, and concerned how to achieve
NoSQL-like speeds while sticking with MySQL and InnoDB. Much of the
talk discussed an InnoDB memcached plugin, which unfortunately is
still in the "lab" or "pre-alpha" stage. But he also suggested some
other ways to better performance, some involving Memcache and others
more round-about:

  • Coding directly with the storage engine API, which is storage-engine
    independent.


  • Using HandlerSocket, which queues write requests and performs them
    through a single thread, avoiding costly fsync() calls. This can
    achieve 30,000 writes per second, robustly.

Matsunobu claimed that many optimizations are available within MySQL
because a lot of data can fit in main memory. For instance, if you
have 10 million users and store 400 bytes per user, the entire user
table can fit in 20 GB. Matsunobu tests have shown that most CPU time
in MySQL is spent in functions that are not essential for processing
data, such as opening and closing a table. Each statement opens a
separate connection, which in turn requires opening and closing the
table again. Furthermore, a lot of data is sent over the wire besides
the specific fields requested by the client. The solutions in the talk
evade all this overhead.


The commercial ecosystem

Both as vendors and as sponsors, a number of companies have always
lent another dimension to the MySQL conference. Some of these really
have nothing to do with MySQL, but offer drop-in replacements for it.
Others really find a niche for MySQL users. Here are a few that I
happened to talk to:

  • Clustrix provides a very
    different architecture for relational data. They handle sharding
    automatically, permitting such success stories as the massive scaling
    up of the social media site Massive Media NV without extra
    administrative work. Clustrix also claims to be more efficient by
    breaking queries into fragments (such as the WHERE clauses of joins)
    and executing them on different nodes, passing around only the data

    produced by each clause.

  • Akiban also offers faster
    execution through a radically different organization of data. They
    flatten the normalized tables of a normalized database into a single
    data structure: for instance, a customer and his orders may be located
    sequentially in memory. This seems to me an import of the document
    store model into the relational model. Creating, in effect, an object
    that maps pretty closely to the objects used in the application
    program, Akiban allows common queries to be executed very quickly, and
    could be deployed as an adjunct to a MySQL database.

  • Tokutek produced a drop-in
    replacement for InnoDB. The founders developed a new data structure
    called a fractal tree as a faster alternative to the B-tree structures
    normally used for indexes. The existence of Tokutek vindicates both

    the open source distribution of MySQL and its unique modular design,
    because these allowed Tokutek's founders to do what they do
    best--create a new storage engine--without needing to create a whole
    database engine with the related tools and interfaces it would
    require.

  • Nimbus Data Systems creates a
    flash-based hardware appliance that can serve as a NAS or SAN to
    support MySQL. They support a large number of standard data transfer
    protocols, such as InfiniBand, and provide such optimizations as
    caching writes in DRAM and making sure they write complete 64KB blocks
    to flash, thus speeding up transfers as well as preserving the life of
    the flash.


Post-conference events

A low-key developer's day followed Percona Live on Friday. I talked to
people in the Drizzle and
Sphinx tracks.

As a relatively young project, the Drizzle talks were aimed mostly at
developers interested in contributing. I heard talks about their
kewpie test framework and about build and release conventions. But in
keeping with it's goal to make database use easy and light-weight, the
project has added some cool features.

Thanks to a
JSON
interface
and a built-in web server, Drizzle now presents you with a
Web interface for entering SQL commands. The Web interface translates
Drizzle's output to simple HTML tables for display, but you can also
capture the JSON directly, making programmatic access to Drizzle
easier. A developer explained to me that you can also store JSON
directly in Drizzle; it is simply stored as a single text column and
the JSON fields can be queried directly. This reminded me of an XQuery
interface added to some database years ago. There too, the XML was
simply stored as a text field and a new interface was added to run the
XQuery selects.

Sphinx, in contrast to Drizzle, is a mature product with commercial
support and (as mentioned earlier in the article) production
deployments at places such as craigslist, as well as an href="http://shop.oreilly.com/product/9780596809539.do">O'Reilly
book. I understood better, after attending today's sessions, what
makes Sphinx appealing. Its quality is unusually high, due to the use

of sophisticated ranking algorithms from the research literature. The
team is looking at recent research to incorporate even better
algorithms. It is also fast and scales well. Finally, integration with
MySQL is very clean, so it's easy to issue queries to Sphinx and pick
up results.

Recent enhancements include an href="https://github.com/alexksikes/fSphinx">add-on called fSphinx
to make faceted searches faster (through caching) and easier, and
access to Bayesian Sets to find "items similar to this one." In Sphinx
itself, the team is working to add high availability, include a new
morphology (stemming, etc.) engine that handles German, improve
compression, and make other enhancements.


The day ended with a reception and copious glasses of Monty Widenius's
notorious licorice-flavored vodka, an ending that distinguishes the
MySQL conference from others for all time.

April 06 2012

Developer Week in Review: When giant corporations collide

The days of the April Fools' web joke are over, or should be. It's gotten too old, to institutionalized, and it's so widespread these days that serious news can slip through the cracks because everyone assumes it's a joke. If people want to pull hoaxes, pick a random day in the middle of the summer and do it then; you'll get much more bang for the buck because no one will be expecting it. I used to like a good fake article as much as the rest, back in the days when they would be buried somewhere in the pages of a magazine's April edition, but now it's just lame. Be assured, all the items in this edition of Developer Week in Review are 100% prank-free and were supervised by the American Humane Association.

Gentlemen, start your lawyers!

Like a large radioactive reptile, the lawsuit between Oracle and Google over the improper use of Java has been sleeping quietly in a courtroom in San Jose. But now, the slumbering monster is about to awake, potentially leaving a trail of broken companies scattered from California to Asia. After all attempts to broker a settlement between Larry's House of Java and the People's Autonomous Car and Search Engine Company failed, the judge involved has ordered the two parties to start sharpening their long-knives, in an unusually candid opinion.

It's hard to overestimate the potential impact that a ruling against Google could have on the smartphone industry. If Google was required to remove Java from Android phones, Android would essentially become useless because the entire stack that Android apps use is built on top of Java. More likely, Google would be required to shell out a significant license fee to Oracle, which added to the ones it already pays to Microsoft and (potentially) Apple, could make Android phones less and less profitable to the handset makers who actually end up paying the fees. Of course, given the glacial pace at which these proceedings move, Android may have already moved on by the time any such judgement actually comes down ...

Linux has a friend in ... Redmond?

In the past few weeks, we've made several references to Microsoft's increasing support of the open source model, and this week brought even more evidence of the sea change out of Washington state. For a technology that Steve Ballmer once described as akin to cancer, Linux is certainly getting a lot of love from Microsoft these days. The software behemoth is now in the top 20 corporate contributors to the Linux Kernel, committing more than 1% of all new lines of code last year.

It is worth bearing in mind that most of that code is in support of Microsoft technologies, such as Hyper-V, but even still, it's clear that Microsoft doesn't treat Linux like an ill-behaved street urchin anymore.

The art of game cheats

I'm not much (if anything) of a game programmer; I've always gravitated more to the web side of the force. But I certainly play my share of games. I'm currently racing my 17 year old to level 80 on "Call of Duty MW3" on the Wii (I'm [MLP]TwilightSparkle if you want to ally with a mediocre player who likes Akimbo FMG9 a bit too much for his own good ...). If you play enough multiplayer, you'll eventually come to recognize the players who have an almost psychic knowledge of where everyone is. They're the ones who always seem to come around the corner already sighted in on you. You know, the cheaters ...

Now, one game developer has stepped forward to explain some of the hacks that cheats use to become Chuck Norris clones and how they are implemented. Even if you are never going to get within 1,000 yards of a z-buffer, it's worth reading to see just how easily games can be tweaked to give unethical players an unbeatable edge.

Fluent Conference: JavaScript & Beyond — Explore the changing worlds of JavaScript & HTML5 at the O'Reilly Fluent Conference (May 29 - 31 in San Francisco, Calif.).

Save 20% on registration with the code RADAR20

Got news?

Please send tips and leads here.

Related:

December 16 2011

Developer Week in Review: HP sets webOS free

Hard to believe there's only 15 days left in 2011; it's flown by so quickly. Next week, I'll be putting out the much anticipated Developer Year In Review, highlighting the ups and downs of the industry over the last 12 months. But for the moment, enjoy these pre-holiday tidbits:

HP gets into the spirit of the season

HP WebOSEvidently, Meg Whitman was visited by three ghosts recently because she opened her window last week and shouted for the boy downstairs to run to the butcher and buy the big goose in the window so it could be delivered to Bob Cratchit's house. Except in this case, the goose was the source code to webOS, and the lucky recipient was the open source community.

It's certainly a magnanimous gesture on the part of HP, and it's likely to lead to any number of interesting spin-off projects. It will also provide an interesting contrast to the current open-source tablet darling, Android. Exactly who will administer the project and which license it will be released under is still uncertain. Hopefully, it will be a relatively permissive license so it can freely cross-pollinate.

For HP, this is definitely making the best of a bad situation. As readers may recall, I've harped on several occasions about how Oracle has been shedding itself of many of the assets it acquired when it purchased Sun. But as far as throwing away money, Oracle is bush-league compared to HP. It's taken less than two years for HP to relegate the $1.2 billion it paid for Palm to the "capital losses" column in its tax return.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

And speaking of Oracle ...

Anyone who has ever been involved in the negotiations for an outside vendor to deliver a software solution knows that it's an inexact science, at best. There always turns out to be requirements that were missed or technical complications that turn up during deployment, and customers are usually (reluctantly) willing to pay the piper because they have already committed to the solution.

Montclair State University evidently decided to try plan B when Oracle went over budget and missed deadlines on the university's new ERP system. They are taking Mr. Ellison's yacht-funding enterprise to federal court, accusing Oracle of rigging the demo and trying (in the words of the university) to extort money by threatening not to complete the work unless paid millions more in fees.

It may be dicey to figure out if Montclair understated its requirements or if Oracle low-balled the bid since I've yet to see a requirements spec for a fixed-price contract that was worth the paper it was written on. Oracle can at least take comfort from the fact that Montclair doesn't have a law school, so there won't be any pro bono faculty members on the legal team.

On the other hand, T&M has its perils, too

Some companies prefer to bid contracts as time and materials (T&M), rather than fixed price. This is a good deal for the contractor because it won't get caught underfunded if things turn out to be complicated. For the customer, it offers the benefit of being able to pull the plug if things aren't working out or to add and remove requirements without having to renegotiate. The downside for the contractor is that it can't profit from finishing early.

Of course, this all assumes that the contractor is actually working on the project. In a recent case, your tax dollars (for all you American readers) were going to pay someone to watch movies, hang out in bars, and ride roller coasters. California-based Aerospace Corp just paid the Department of Justice a nice round $2.5 million to settle allegations that not only was it billing time for an employee who was moonlighting at another firm, but that he spent his days at leisure while billing both firms.

Incredibly, this went on for five years, despite such stunts as billing for more than 24 hours of work in a single day. You almost have to admire the chutzpah of Mr. William Grayson Hunter, who also inflated his high school diploma into a doctorate from Oxford. He also managed to die of natural causes before the long arm of the law could bring him to justice, presumably with a smile on his face and a Six Flags hat on his head.

Got news?

Please send tips and leads here.

Related:

October 21 2011

Developer Week in Review: Talking to your phone

I've spent the last week or so getting up to speed on the ins and outs of Vex Robotics tournaments since I foolishly volunteered to be competition coordinator for an event this Saturday. I've also been helping out my son's team, offering design advice where I could. Vex is similar to Dean Kamen's FIRST Robotics program, but the robots are much less expensive to build. That means many more people can field robots from a given school and more people can be hands-on in the build. If you happen to be in southern New Hampshire this Saturday, drop by Pinkerton Academy and watch two dozen robots duke it out.

In non-robotic news ...

Why Siri matters

SiriIt's easy to dismiss Siri, Apple's new voice-driven "assistant" for the iPhone 4S, as just another refinement of the chatbot model that's been entertaining people since the days of ELIZA. No one would claim that Siri could pass the Turing test, for example. But, at least in my opinion, Siri is important for several reasons.

On a pragmatic level, Siri makes a lot of common smartphone tasks much easier. For example, I rarely used reminders on the iPhone and preferred to use a real keyboard when I had to create appointments. But Siri makes adding a reminder or appointment so easy that I have made it pretty much my exclusive method of entering them. It also is going to be a big win for drivers trying to use smartphones in their cars, especially in states that require hands-free operations.

I suspect Siri will also end up being a classic example of crowdsourcing. If I were Apple, I would be capturing every "miss" that Siri couldn't handle and looking for common threads. Since Siri is essentially doing natural language processing and applying rules to your requests, Apple can improve Siri progressively by adding the low-hanging fruit. For example, at the moment, Siri balks at a question like, "How are the Patriots doing?" I'd be shocked if it fails to answer that question in a year since sports scores and standings will be at the heart of commonly asked questions.

For developers, the benefits of Siri are obvious. While it's a closed box right now, if Apple follows its standard model, we should expect to see API and SDK support for it in future releases of iOS. At the moment, apps that want voice control (and they are few and far between) have to implement it themselves. Once apps can register with Siri, any app will be able to use voice.

Velocity Europe, being held Nov. 8-9 in Berlin, will bring together the web operations and performance communities for two days of critical training, best practices, and case studies.

Save 20% on registration with the code RADAR20

Can Open Office survive?

OpenOffice.org logoLong-time WIR readers will know that I'm no fan of how Oracle has treated its acquisitions from Sun. A prime example is OpenOffice. In June, OpenOffice was spun off from Oracle, and therefore lost its allowance. Now the OpenOffice team is passing around the hat, looking for funds to keep the project going.

We need to support Open Office because it's the only project that really keeps Microsoft honest as far as providing open standards access to Microsoft Office products. It's also the only way that Linux users can deal with the near-ubiquitous use of Office document formats in the real world (short of running Office in a VM or with Wine.)

The revenge of SQL

The NoSQL crowd has always had Google App Engine as an ally since the only database available to App Engine apps has been the App Engine Datastore, which (among other things) doesn't support joins. But much as Apple initially rejected multitasking on the iPhone (until it decided to embrace it), Google appears to have thrown in the towel as far as SQL goes.

It's always dangerous to hold an absolutist position (with obvious exceptions, such as despising Jar Jar Binks). SQL may have been overused in the past, but it's foolish to reject SQL altogether. It can be far too useful at times. SQL can be especially handy, as an example, when developing pure REST-like web services. It's nice to see that Google has taken a step back from the edge. Or, to put it more pragmatically, that it listens to its customer base on occasion.

Got news?

Please send tips and leads here.

Related:

October 07 2011

Top Stories: October 3-7, 2011

Here's a look at the top stories published across O'Reilly sites this week.

Oracle's Big Data Appliance: what it means
Oracle's new Big Data Appliance couldn't be a plainer validation of what's important in big data right now, or where the battle for technology dominance lies.

PhoneGap basics: What it is and what it can do for mobile developers
Joe Bowser, the developer of the Android version of PhoneGap, on the pros and cons of developing with the PhoneGap cross-platform application framework.


How data and open government are transforming NYC
New York City has become the epicenter for experiments in data-driven governance. Here, NYC officials Rachel Sterne and Carole Post discuss the city's data initiatives.

The making of a "minimum awesome product"
In this podcast, Evan Doll, the co-founder of Flipboard sat down with Joe Wikert to discuss Flipboard's focus on design and social integration.

iPad vs. Kindle Fire: Early impressions and a few predictions
Few have actually held the Kindle Fire, let alone put it through its paces, so Pete Meyers chose a novel analytical approach: Examine his own iPad habits and look for spots where the Fire can find a foothold.


Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders. Save 20% on registration with the code AN11RAD.

October 06 2011

Oracle's NoSQL

Oracle's turn-about announcement of a NoSQL product wasn't really surprising. When Oracle spends time and effort putting down a technology, you can bet that its secretly impressed, and trying to re-implement it in its back room. So Oracle's paper "Debunking the NoSQL Hype" should really have been read as a backhanded product announcement. (By the way, don't click that link; the paper appears to have been taken down. Surprise.)

I have to agree with DataStax and other developers in the NoSQL movement: Oracle's announcement is a validation, more than anything else. It's certainly a validation of NoSQL, and it's worth thinking about exactly what that means. It's long been clear that NoSQL isn't about any particular architecture. When databases as fundamentally different as MongoDB, Cassandra, and Neo4J can all be legitimately characterized as "NoSQL," it's clear that NoSQL isn't a "thing." We've become accustomed to talking about the NoSQL "movement," but what does that mean?

As Justin Sheehy, CTO of Basho Technologies, said, the NoSQL movement isn't about any particular architecture, but about architectural choice. For as long as I can remember, application developers have debated software architecture choices with gusto. There were many choices for the front end; many choices for middleware; and careers rose and fell based on those choices. Somewhere along the way, "Software Architect" even became a job title. But for the backend, for the past 20 years there has really been only one choice: a relational database that looks a lot like Oracle (or MySQL, if you'd prefer). And choosing between Oracle, MySQL, PostgreSQL, or some other relational database just isn't that big a choice.

Did we really believe that one size fits all for database problems? If we ever did, the last three years have made it clear that the model was broken. I've got nothing against SQL (well, actually, I do, but that's purely personal), and I'm willing to admit that relational databases solve many, maybe even most, of the database problems out there. But just as it's clear that the universe is a more complicated place than physicists thought it was in 1990, it's also clear that there are data problems that don't fit 20-year-old models. NoSQL doesn't use any particular model for storing data; it represents the ability to think about and choose your data architecture. It's important to see Oracle recognize this. The company's announcement isn't just a validation of key-value stores, but of the entire discussion of database architecture.

Of course, there's more to the announcement than NoSQL. Oracle is selling a big data appliance: an integrated package including Hadoop and R. The software is available standalone, though Oracle clearly hopes that the package will be running on its Exadata Database hardware (or equivalent), which is an impressive monster of a database machine (though I agree with Mike Driscoll, that machines like these are on the wrong side of history). There are other bits and pieces to solve ETL and other integration problems. And it's fair to say that Oracle's announcement validates more than just NoSQL; it validates the "startup stack" or "data stack" that we've seen in many of most exciting new businesses that we watch. Hadoop plus a non-relational database (often MongoDB, HBase, or Cassandra), with R as an analytics platform, is a powerful combination. If nothing else, Oracle has given more conservative (and well-funded) enterprises permission to make the architectural decisions that the startups have been making all along, and to work with data that goes beyond what traditional data warehouses and BI technologies allow. That's a good move, and it grows the pie for everyone.

I don't think many young companies will be tempted to invest millions in Oracle products. Some larger enterprises should, and will, question whether investing in Oracle products is wise when there are much less expensive solutions. And I am sure that Oracle will take its share of the well-funded enterprise business. It's a win all around.

Web 2.0 Summit, being held October 17-19 in San Francisco, will examine "The Data Frame" — focusing on the impact of data in today's networked economy.

Save $300 on registration with the code RADAR

Related:


Strata Week: Oracle's big data play

Here are the data stories that caught my attention this week:

Oracle's big data week

Eyes have been on Oracle this week as it holds its OpenWorld event in San Francisco. The company has made a number of major announcements, including unveiling its strategy for handling big data. This includes its Big Data Appliance, which will use a new Oracle NoSQL database as well as an open-source distribution of Hadoop and R.

Edd Dumbill examined the Oracle news, arguing that "it couldn't be a plainer validation of what's important in big data right now or where the battle for technology dominance lies." He notes that whether one is an Oracle customer or not, the company's announcement "moves the big data world forward," pointing out that there is now a de facto agreement that Hadoop and R are core pieces of infrastructure.

GigaOm's Derrick Harris reached out to some of the startups who also offer these core pieces, including Norman Nie, the CEO of Revolution Analytics, and Mike Olson, CEO of Cloudera. Not surprisingly perhaps, the startups are "keeping brave faces, but the consensus is that Oracle's forays into their respective spaces just validate the work they've been doing, and they welcome the competition."

Oracle's entry as a big data player also brings competition to others in the space, such as IBM and EMC, as all the major enterprise providers wrestle to claim supremacy over whose capabilities are the biggest and fastest. And the claim that "we're faster" was repeated over and over by Oracle CEO Larry Ellison as he made his pitch to the crowd at OpenWorld.

Web 2.0 Summit, being held October 17-19 in San Francisco, will examine "The Data Frame" — focusing on the impact of data in today's networked economy.

Save $300 on registration with the code RADAR

Who wrote Hadoop?

As ReadWriteWeb's Joe Brockmeier notes, ascertaining the contributions to open-source projects is sometimes easier said than done. Who gets credit — companies or individuals — can be both unclear and contentious. Such is the case with a recent back-and-forth between Cloudera's Mike Olson and Hortonworks' Owen O'Malley over who's responsible for the contributions to Hadoop.

O'Malley wrote a blog post titled "The Yahoo! Effect," which, as the name suggests, describes Yahoo's legacy and its continuing contributions to the Hadoop core. O'Malley argues that "from its inception until this past June, Yahoo! contributed more than 84% of the lines of code still in Apache Hadoop trunk." (Editor's note: The link to "trunk" was inserted for clarity.) O'Malley adds that so far this year, the biggest contributors to Hadoop are Yahoo! and Hortonworks.

Lines of code contributed to apache hadoop trunkLines of code contributed to Apache Hadoop Trunk (from Owen O'Malley's post, "The Yahoo! Effect")

That may not be a surprising argument to hear from Hortonworks, the company that was spun out of Yahoo! earlier this year to focus on the commercialization and development of Hadoop.

But Cloudera's Mike Olson challenges that argument — again, not a surprise, as Cloudera has long positioned itself as a major contributor to Hadoop, a leader in the space, and of course now the employer of former Yahoo! engineer Doug Cutting, the originator of the technology. Olson takes issue with O'Malley's calculations and in a blog post of his own, contends that these calculations don't accurately take into account the companies that people now work for:

Five years is an eternity in the tech industry, however, and many of those developers moved on from Yahoo! between 2006 and 2011. If you look at where individual contributors work today — at the organizations that pay them, and at the different places in the industry where they have carried their expertise and their knowledge of Hadoop — the story is much more interesting.

Olson also argues that it isn't simply a matter of who's contributing to the Apache Hadoop core, but rather who is working on:

... the broader ecosystem of projects. That ecosystem has exploded in recent years, and most of the innovation around Hadoop is now happening in new projects. That's not surprising — as Hadoop has matured, the core platform has stabilized, and the community has concentrated on easing adoption and simplifying use.

Got data news?

Feel free to email me.

Related:

October 04 2011

Four short links: 4 October 2011

  1. jfdi.asia -- Singaporean version of TechStars, with 100-day program ("the bootcamp") Jan-Apr 2012. Startups from anywhere in the world can apply, and will want to because Singapore is the gateway to Asia. They'll also have mentors from around the world.
  2. Oracle NoSQLdb -- Oracle want to sell you a distributed key-value store. It's called "Oracle NoSQL" (as opposed to PostgreSQL, which is SQL No-Oracle). (via Edd Dumbill)
  3. Facebook Browser -- interesting thoughts about why the browser might be a good play for Facebook. I'm not so sure: browsers don't lend themselves to small teams, and search advertising doesn't feel like a good fit with Facebook's existing work. Still, making me grumpy again to see browsers become weapons again.
  4. Bitbucket -- a competitor to Github, from the folks behind the widely-respected Jira and Confluence tools. I'm a little puzzled, to be honest: Github doesn't seem to have weak spots (the way, for example, that Sourceforge did).

October 03 2011

Oracle's Big Data Appliance: what it means

Today, Oracle announced their Big Data Appliance. It couldn't be a plainer validation of what's important in big data right now, or where the battle for technology dominance lies.

Oracle's appliance includes some homegrown technology, most specifically a NoSQL database of their own design, and some open source technologies: Hadoop and R. Let's take a look at what these three decisions might mean.

Oracle NoSQL Database: Oracle's core reputation is as a database vendor, and as owners of the Berkeley DB technology, they have a core NoSQL platform to build upon (Berkeley was NoSQL for years before we even had that term). Oracle have no reason to partner with or incorporate other NoSQL tech such as Cassandra or MongoDB, and now pose a significant business threat to those technologies—perhaps Cassandra more than MongoDB, due to its enterprise credentials.

Hadoop: competitive commercial big data solutions such as Greenplum and Aster Data got ahead in the market through incorporating their own MapReduce technologies. Oracle hasn't bothered to do this, and has instead standardized on Hadoop and a system of connectors to its main Oracle product. (Both Greenplum and Aster also have Hadoop connectors.) If it needed any further validation, this confirms Hadoop's arrival as the Linux of big data. It's a standard.

R: big data isn't much use until you can make sense of it, and the inclusion of R in Oracle's big data appliance bears this out. It also sets up R as a new industry standard for analytics: something that will raise serious concern among vendors of established statistical and analytical solutions SAS and SPSS.

Whether you use Oracle or not, today's announcement moves the big data world forward. We have de facto agreement on Hadoop and R as core infrastructure, and we have healthy competition at the database and NoSQL layer.

Talk about this at Strata 2012: As the call for participation for Strata 2012 (Feb 28-Mar 1, Santa Clara, CA) nears its close, Oracle's announcement couldn't be more timely. We are opening up new content tracks focusing on the Hadoop ecosystem and on R. Submit your proposal by the end of this week.

September 29 2011

Developer Week in Review: Android proves fruitful for Microsoft

The ball has finally dropped at Apple, and we know that October 4 is the big day that iOS 5 and some undisclosed subset of iPhone devices will be unveiled. Oddly, developers still haven't received the Gold Master of iOS 5, which means that Apple is cutting things close if it wants to give people time to update apps in the store, not to mention those of us who have to revise books once the NDA lifts on iOS 5.

So, while we wait for Godot Tim Cook, let's see what other mischief is afoot.

Royalties for Redmond

As we've reported previously, one of the big winners in the growth of Android has been Microsoft, as phone manufactures have been lining up to pay royalties to Redmond to avoid patent lawsuits. Samsung joined the fray this week, agreeing to pony up a reported $5 per phone to stay out of court.

In light of this, Google's purchase of Motorola Mobility is looking less and less wise. The widely held view was that the sale was intended to shield Android-based phones behind Motorola's rich patent portfolio, but every major player is caving into Microsoft anyway.

Between the squeeze play on Android and the long-standing siphoning of Linux revenues from companies such as Novell, Microsoft seems to be following a business plan reminiscent of a certain Monty Python sketch.

Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders.

Save 20% on registration with the code AN11RAD

SPARC? Oh yeah, I remember that ...

SPARC T4Back when dinosaurs ruled the earth, the absolutely hottest thing you could have on your desk was a Sun-4. The SPARC-based systems were leaps and bounds ahead of anything else in their price range, except perhaps for some esoteric hardware from Silicon Graphics (remember them?)

Time has not been kind to the SPARC, alas. Sun's hardware market share shrank as people discovered that Linux on cheap hardware could give a better bang for the buck, and the entire venture was eventually swallowed by Oracle. The conventional wisdom was that Oracle bought Sun largely for its hardware line, and there was some confirmation of that this week. While much of the rest of Sun's holdings have been left to languish or spun off entirely, Larry's gang has evidently been busy with hardware. The SPARC T4 is the result.

The problem is, while the T4 brings some modern features like out-of-order execution to the SPARC line, these are things that other processor families have had for a decade or more. While it may staunch the flow of former SPARC customers defecting to x86 systems, it's unlikely to gain many new converts. And as any Harvard MBA can tell you, a business model based on not losing existing customers is not a formula for success in the long term.

Might want to rethink those voting machines (and the people who use them)

We've been hearing for years that direct recording electronic voting machines are potentially hackable. With a powder-keg election forthcoming, it was therefore not reassuring news this week that researchers at Argonne National Laboratory were able to totally subvert the voting counts on Diebold voting machines, simply by installing a $10 circuit between a ribbon cable and the connector. Since Diebold machines are not tamper resistant, this means that pretty much anyone with the technical savvy to create the device could hijack the polls.

I see this as part of a larger problem in the computer industry — an almost blind belief that technology can solve social problems in isolation. People seem to think that making government data transparent or turning to social networking can solve society's ills. In reality, the things that need to be re-engineered are the people. The best software in the world won't make people give up irrational belief systems, or stop hating others (be they red state or blue) because they're different. And as long as hate, intolerance and ignorance run wild, technology will be as likely to be used as a weapon as a solution.

Got news?

Please send tips and leads here.

Related:

September 23 2011

Developer Week in Review: webSOS

On the developer front, if the growing tide of rumors is correct, there will be some iOS stuff to report next week.

Meanwhile:

Last one out turn off the lights

HP WebOSHP has flung the axe, and it has taken out a large swath of the ill-fated webOS crew. HP is confirming that development will cease by the end of the year, reducing the number of viable mobile operating systems down to two again (Blackberry is heading the way of webOS, and Windows Mobile has an uphill battle at this point).

Is hegemony in the mobile space a good thing? Maybe, maybe not. It's good for mobile developers, as it reduces the number of potential platforms you need to consider. It could be bad for consumers, as it reduces the pressure on the remaining players to innovate. However, given that neither HP nor Microsoft nor RIM was pushing the envelope much with their products, that might not be a valid concern. And, frankly, Google and Apple do a pretty good job of stealing ideas from each other — witness the new Android-like notification framework in iOS5.

Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders.

Save 20% on registration with the code AN11RAD

An (un)sign of the times

JavaOne of the joys of Java development is dealing with signed jars. For the uninitiated, Java Archives (jars) can be signed, "proving" that the contents inside are valid and untampered. Among other things, it is how the Java Web Start framework decides which Java programs can be automatically downloaded and started from a web page. Getting your jar file signed correctly is a delicate dance, and getting it wrong means that the applications will just plain not work.

Seemingly out of the blue, Oracle has started to remove the old Sun signatures from some core Java libraries that many developers depend on. The end result of this is that, going forward, it will become more difficult to deploy applications that use these frameworks. Oracle is saying it was done for security reasons, but as with many moves by Oracle lately, the end result has been to upset the developer community.

Creating the next generation of coders?

One of the paradoxical phenomena that seems to be occurring in society is that, even as technology is becoming more and more a part of people's lives, programming is being marginalized in the public schools. Instead, kids are taught how to use Excel or Powerpoint (God knows, my kid is a Powerpoint wiz!).

In the UK, they've decided to turn things around by making software design a part of the curriculum. You can make a strong argument that software engineering brings in skills from a lot of other disciplines like math and science, so it makes a good integrated teaching experience. On the other hand, my experience has been that public schools are uniquely bad at teaching coding because they try to teach it by rote, when it is at heart a creative process. It's like trying to teach painting by telling the students exactly where to place every brush stroke. Only time will tell if the UK can do it any better.

Got news?

Please send tips and leads here.

Related:

September 09 2011

Developer Week in Review: iPhone 5 is still on hold

Ah, a new school year. It's the time when my wife disappears into her office, not to be seen again until the late spring unless she sees her shadow. My son is grumbling about 60-question math homework assignments, and all the melancholy I feel during the summer about being the only family member on the clock fades away since I actually have the lightest schedule now. Revenge is sweet ...

If you've been in a late-summer haze, here's a few items you may have missed.

Bigfoot sighted using iPhone 5

iOS 5The predictions in August were that the iPhone 5 (or 4S, or whatever it's going to be called) would be announced in early September. Then it was going to be mid-September, and now people are talking about early October. Now, I'm as much of an Apple fanboy as the next guy, but this obsession about the new phone seems to border on the absurd. I've only had my iPhone 4 for a year — I'm not even sure I would upgrade to a 5 unless it cures cancer or something.

The only real reason to speculate about the iPhone 5 ship date is that it will probably coincide with the general release of iOS 5, which definitely is something to talk about, if only to other people who have signed the developer NDA. I mean the ... no, I can't talk about that. But the ... no, can't mentioned that either. Anyway, it's wicked cool, trust me.

Your comprehensive legal roundup

HTCLast week, everyone sued everyone. This item will repeat for the foreseeable future.

Of particular interest is that HTC is using patents acquired from Google to strike back at Apple. The patent war is becoming reminiscent of the Cuban Missile Crisis — I expect to hear a statement from Oracle HQ any day reporting a suspected transfer of patents from Apple to Google and vowing a blockade unless the lawyers turn back at once.

The Makers are coming, the Makers are coming!

For those who live on the East Coast, your annual chance to get your geek on is coming up next weekend. Maker Faire New York will be returning for a second year at the NY Hall of Science, and it's well worth the trip. I went with my son last year, and we'll be back this year as well.

It's a great chance to see programming integrating with the physical world on a much more practical (or impractical) level than developers are accustomed to. If you've spent your life designing ecommerce websites, it can be refreshing to see a pair of honking-big computer-controlled Tesla coils blaring out music. It's also a Mecca for embedded computing and micro controllers, so if you like programming on the small scale, you'll see a lot to enjoy. If you happen to run into me there, say hi!

Got news?

Please send tips and leads here.

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code ORM30

Related:

August 03 2011

Developer Week in Review: Lion drops pre-installed MySQL


A busy week at Casa Turner, as the infamous Home Renovations of Doom wrap up, I finish the final chapters of "Developing Enterprise iOS Applications" (buy a copy for all your friends, it's a real page turner!), pack for two weeks of vacation with the family in California (Palm Springs in August, 120 degrees, woohoo!), and celebrate both a birthday and an anniversary.



But never fear, WIR fans, I'll continue to supply the news, even as my MacBook melts in the sun and the buzzards start to circle overhead.

The law of unintended consequences

Lion ServerIf you decide to install Lion Server, you may notice something missing from the included software: MySQL. Previous releases of OS X server offered pre-installed MySQL command line and GUI tools, but they are AWOL from Lion. Instead, the geek-loved but less widely used Postgres database is installed.

It seems pretty obvious to the casual observer why Apple would make this move. With Oracle suing Google over Java, and Oracle's open source philosophy in doubt, I know I wouldn't want to stake my bottom line on an Oracle package bundled with my premiere operating system. Apple could have used one of the non-Oracle forks of MySQL, but it appears they decided to skirt the issue entirely by going with Postgres, which has a clear history of non-litigiousness.

Meanwhile, Oracle had better be asking themselves if they can afford to play the games they've been playing without alienating their market base.

South Korea fines Apple 3 million won, which works out to ...

Apple has bee been hit with a penalty from the South Korean government that's a result of the iPhone location-tracking story that broke earlier this year. Now, Apple may have more money than the U.S. Treasury sitting in petty cash right now, but it will be difficult for them to recover from such a significant hit to their bottom line: a whopping 3 million won, which works out to a staggering ... um ... $2,830. Never mind.

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science -- from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 20% on registration with the code STN11RAD

Java 7 and the risks of X.0 software

Java 7 was recently released to the world with great fanfare and todo. This week, we got a reminder why using an X.0 version of software is a risky endeavor. It turns out that the optimized compiler is really a pessimized compiler, and that programs compiled with it stand a chance of crashing. Even better, there's a chance they'll just go off and do the wrong thing.

Java 7 seems to be breaking new ground in non-deterministic programming, which will be very helpful for physics researchers working with the Heisenberg uncertainty principle. What could be more appropriate for simulating the random behavior of particles than a randomly behaving compiler?

Got news?

Please send tips and leads here.

Related:

July 28 2011

Developer Week in Review: Linux turns the big 3.0

I have been informed by the contractors, currently starting in on bathroom renovation No. 3 at our house, that my official designation is "Houston." This is because, pretty much every day, they call me at work and say "We have a problem." If you think patching bugs in legacy code written by someone who has left your company is bad, try getting work done on a 215-year-old house.



While the rest of the O'Reilly family is out cavorting in the wilds of Portland at OSCON, a few of us must tend the fires back East, and keep the rest of the world informed on what's going on. Such as ...



The new Linux is out, the new Linux is out!

LinuxIt used to be, when COBOL developers roamed the Earth, that a new release of the Linux kernel was a cause for much excitement, especially something as momentous as a new major version. Can anyone every remember when Linux 1.x became Linux 2.0? Here's a hint, it was 15 years ago.



But times have changed. Very few people install a Linux kernel directly anymore. Most get them through the distribution they have chosen. And for people who have embedded versions of Linux, they may not have the slightest idea what version of the kernel they're running. Ask a random sampling of HTC Android users what kernel is installed, and you'll probably get a blank stare (here's a cheat sheet, if you're interested.)

Adding to the ho-hum nature of the 3.0 release is that fact that there's really nothing special in it, by Linus' own admission. He just figured it was time to stop endlessly adding to the 2.0 version tree, and get a clean start on the 3's. With the 20th anniversary of the famous Linux Letter coming up in late August, now is probably as a good a time as ever to put the terrible twos to bed for good.

Here's a fun question to ponder, though: How many build scripts that assume "2.6.X" or "2.X.X" as a Linux version number are about to break?

Hoist with their own petard

It behooves people to remember, in this day and age, that the things they say and write may come back to haunt them. Thus, when trying to make the case that the Android operating system is an unholy misuse of Java, it would have been good for Oracle to remember that there was a letter floating around from 2007 in which Sun CEO Jonathan Schwartz offers Google a warm greeting, and pledged to work aggressively to cooperate with Google on Android.



There's a certain sweet symmetry in Oracle's battle against the search engine giant being potentially derailed by material from deep in the web's archives. It was legal eagles at Groklaw who uncovered the letter, but I'm sure there was much Googling involved in finding it.



Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science -- from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 20% on registration with the code STN11RAD

You have to admire Lodsys' ambition

Not content with suing the smaller fry of the iOS and Android ecosystem, patent "leveragers" (I'd use a less kind word there, but my editor would just change it ...) Lodsys has taken their patent fight to some truly big fish. In new legal action, Rovio (the makers of "Angry Birds"), EA, Take Two Interactive ("Grand Theft Auto") and Atari have found themselves in the defendant box.



The continuing suits, which revolve around in-app purchases, could prove an interesting line in the sand. Apple has licensed rather than litigated in the past, but the signs so far are that Apple (which already paid Lodsys once for the use of the patents) has decided that enough is enough. By dragging big players such as EA into the fray, Lodsys may be making the same mistake that SCO did when they dragged IBM and other large corporations into their Linux litigation. Big companies have large legal teams, and at some point, Lodsys may find themselves worse off than if they had just taken their money and run.


Got news?

Please send tips and leads here.

Related:

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl