Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

December 02 2013

Four short links: 2 December 2013

  1. CalTech Machine Learning Video Library — a pile of video introductions to different machine learning concepts.
  2. Awesome Pokemon Hack — each inventory item has a number associated with it, they are kept at a particular memory location, and there’s a glitch in the game that executes code at that location so … you can program by assembling items and then triggering the glitch. SO COOL.
  3. Drone Footage of Bangkok Protests — including water cannons.
  4. The Mature Optimization Handbook — free, well thought out, and well written. My favourite line: In exchange for that saved space, you have created a hidden dependency on clairvoyance.

June 05 2012

The software professional vs the software artist

I hope that James Turner's post on "The overhead of insecure infrastructure" was ironic or satiric. The attitude he expresses is all too common, and frankly, is the reason that system administrators and other operations people can't keep their systems secure.

Why do we have to deal with vulnerabilities in operating systems and applications? It's precisely because of prima donna software developers who think they're "artists" and can't be bothered to take the time to do things right. That, and a long history of management that was more interested in meeting ship dates than shipping secure software; and the never ending and always escalating battle between the good guys and the bad guys, as black hats find new vulnerabilities that no one thought of a week ago, let alone a few years ago.

Yes, that's frustrating, but that's life. If a developer in my organization said that he was too good and creative to care about writing secure code, he would be out on his ear. Software developers are not artistes. They are professionals, and the attitude James describes is completely unprofessional and entirely too common.

One of the long-time puzzles in English literature is Jonathan Swift's "A Modest Proposal for Preventing the Children of Poor People From Being a Burden on Their Parents or Country, and for Making Them Beneficial to the Publick." It suggests solving the problem of famine in Ireland by cannibalism. Although Swift is one of English literature's greatest satirists, the problem here is that he goes too far: the piece is just too coldly rational, and never gives you the sly look that shows something else is going on. Is Turner a latter-day Swift? I hope so.


May 18 2012

Top Stories: May 14-18, 2012

Here's a look at the top stories published across O'Reilly sites this week.

A federal judge learned to code
The judge presiding over the Oracle/Google case learned Java, and that skill came in handy when coding specifics arose during the trial. It's proof that coding is a part of cultural competence, even if you never do it professionally.

The chicken and egg of big data solutions
So, here we are with all of this disruptive big data technology, but we seem to have lost the institutional wherewithal to do anything with it in a lot of large companies, at least until package solutions come along.

DIY learning: Schoolers, Edupunks, and Makers challenge education
Schoolers, Edupunks and Makers are showing us what's possible when learners, not institutions, own the education that will define their lives.

John Allspaw on DevOps
John Allspaw discusses DevOps in high-volume web companies and the importance of cooperation between development and operations.

JavaScript and Dart: Can we do better?
O'Reilly editor Simon St. Laurent talked with Google's Seth Ladd about the challenges of improving the web. How can we build on JavaScript's ubiquity while addressing performance, team, and scale issues?

Velocity 2012: Web Operations & Performance — The smartest minds in web operations and performance are coming together for the Velocity Conference, being held June 25-27 in Santa Clara, Calif. Save 20% on registration with the code RADAR20.

April 06 2012

State of the Computer Book Market, part 4: The Languages

In this fourth post (see posts one, two and three) on the State of the Computer Book Market, we will look at programming languages and drill in a little on each language area.

Overall, the market for programming languages was up slightly at 1.15% in 2011 when compared with 2010. There were 6,435,247 units sold in 2011 versus 6,361,178 units sold in 2010, which is an increase of 74,069 additional units. Java was the biggest language again and experienced healthy growth of 13.46% or 33,930 more units in 2011 than 2010, while PHP occupied the opposite end with the biggest decrease at -26,994 fewer units year-over-year or a -22.50% rate of change.

Before we begin to drill in on the languages, we thought it would be best to explain our "language dimension." When we group books by their language dimension, we categorize them by the language used in their code examples. So Flash Programming with Java would be in our Flash atomic category, but the language dimension would be Java. Similarly, our Head First Design Patterns book contains examples written in Java, so it too carries the "Java" tag on the language dimension.

To provide some perspective, 2010 and 2011 have been a mixed-bag of results for tech publishing, meaning there were some bright spots with some of the technologies and some definite low areas too. The chart directly below does not include books that are method-oriented, about project management, about consumer operating systems, or books without language-oriented material. So this is a different view of the market than the overall view found in post 1 of this series. You can see all languages on a week-by-week basis while showing that the the languages closely mirrored the overall market found in post 1. In other words, the graph for Languages shows that 2011 was a roller-coaster for languages too.


Java continues to be the number one language from book units sold and dollars. There is some shuffling going on with the languages tough. JavaScript is very hot now, as is R. Likely a result of Android and big data driving folks to these languages. Java grew 13.46% or roughly 33,930 additional units in 2011 and JavaScript doubled the growth rate of Java at 27.18% and an additional 61,758 units in 2011. PHP and C++ switched places in 2011 where PHP produced 126,994 units in 2011 and fell to #6 and C++ moved into 5th with 19,207 additional units. Only three languages kept their 2010 ranking, Java at #1, Objective C at #4, and Python at #8. As you can see in the 2010-2011 Top 20 languages chart below, Java has a significant lead in the language race with JavaScript moving up quickly.

2010-2011 Top 20 Languages


If you look at the chart below, you will see which languages were responsible for the most units sold between 2004 and 2011. The chart is basically the sum of units each year for the respective language. The top ten languages generated unit sales of 10,283,109 for the 8-year period, while the second ten generated 2,910,636 in the same period. The top ten languages represented roughly 72% of units sold during this period. With so many languages to choose from, makes me wonder if any language will every see the numbers that Java experienced in the early 2000 timeframe.


A treemap view of the programming Languages


In the treemap view above, which compares the last quarter of 2011 with the last quarter of 2010, you'll notice balance of green, red and black areas. This pretty much indicates that the last quarter of 2011 was roughly the same as the last quarter of 2010 from the programming languages perspective. Most of the bright green, which indicates rapid growth, is in the bottom right hand corner and small in size. This means there was some growth with small languages in the fourth quarter of 2011.

Before we dive in, let's look at the high-level picture for the grouping of languages. I have grouped these languages by total number of units sold between 2004-2011. As you can see in the table below, the Large and Major languages were up collectively, and everything from Mid-Major and below was down. However books where the language was the focus, ended up 74,688 ahead of 2010 even though the Mid-Major and below lost -15,120 units compared to last year.

Group Unit Range Y2010 Units Y2011 Units 10MketShar 11MketShar Large 50,000 — 200,000 1,323,608 1,389,650 81.09% 79.72% Major 10,000 — 49,999 220,350 244,116 13.50% 14.00% Mid-Major 3,000 — 9,999 77,969 68,980 4.78% 3.96% Mid-Minor 1,600 — 2,999 17,822 17,739 1.09% 1.02% Minor 400 — 1,599 9,364 9,546 0.55% 0.52% Linelist 100 — 399 15,210 10,350 0.42% 0.23% TheRest < 100 2,365 995 0.14% 0.06%

For the sake of grouping and presenting this information in a more readable format, we have classified the categories for the languages in this way with the following headers:


  • Language: Name or short name of the language


  • 2010 Units: Units sold in 2010
  • 2011 Units: Units sold in 2011


  • 2010 Titles: Number of Titles making Bookscan 3000 in 2010
  • 2011 Titles: Number of Titles making Bookscan 3000 in 2011


  • 10Mkt Share: 2010 Market Share
  • 11Mkt Share: 2011 Market Share

The following table contains data for the Large languages. As you can see, 6 of the 11 top languages experienced growth in 2011 and were led by JavaScript impressive growth and Java's continued strength. Objective-C has experienced a second year of declining units but seems to have slowed the decline. What is up with C++? Why is this language still growing? Is there something going on with devices that is fueling the growth of C++?

Large Programming Languages — 50,000 — 200,000 units in 2011

*Large* U N I T S T I T L E S M A R K E T S H A R E Language 2010 Units 2011 Units 2010 Titles 2011 Titles 10Mkt Share 11Mkt Share Java 219,031 251,710 374 420 13.42% 14.45% JavaScript 165,910 228,527 180 243 10.16% 13.12% C# 208,450 181,497 271 305 12.77% 10.42% Objective C 151,229 140,679 89 138 9.27% 8.08% C++ 110,624 129,373 196 201 6.78% 7.43% PHP 146,219 119,203 173 180 8.96% 6.85% VBA 64,443 92,954 71 77 3.95% 5.34% Python 69,805 78,200 95 109 4.28% 4.49% SQL 57,184 57,425 96 108 3.50% 4.30% ActionScript 71,797 56,075 99 114 4.40% 3.22% .net Languages 58,916 54,007 83 83 3.61% 3.10%

Here are the top titles for the Large languages:

O'Reilly Head First Java, Second Edition O'Reilly Learning PHP, MySQL, and JavaScript, First Edition Dummies Android Application Development For Dummies APress Beginning iPhone 4 Development: Exploring the iOS SDK O'Reilly JavaScript: The Good Parts

You'll notice in the Major languages that Ruby had the biggest growth among this grouping. And the R language had significant growth and surpassed Perl in sales units. Perl used to be one of the largest languages around and is now ranked #19 among the languages. PowerShell experienced it second consecutive year of growth in 2011.

Major Programming Languages — 10,000 — 49,999 units in 2011

*Major* U N I T S T I T L E S M A R K E T S H A R E Language 2010 Units 2011 Units 2010 Titles 2011 Titles 10Mkt Share 11Mkt Share Visual Basic 47,925 42,272 89 87 2.94% 2.43% Ruby 24,434 37,760 49 56 1.50% 2.17% C 38,786 36,828 92 82 2.38% 2.11% Powershell 19,861 28,520 26 30 1.22% 1.64% Shell Script 25,068 24,004 20 22 1.54% 1.38% R 11,431 20,072 18 26 0.70% 1.15% Transact SQL 17,507 17,308 28 27 1.07% 0.99% Perl 16,658 15,786 32 28 1.02% 0.91% Processing 9,053 10,791 9 11 0.55% 0.62% SAS 9,627 10,775 27 27 0.59% 0.62%

Here are the top titles for the Major languages.

Prentice Hall The C Programming Language Prentice Hall Practical Guide to Linux Commands, Editors, and Shell Programming Dummies SQL For Dummies SAS Publishing The Little SAS Book: A Primer, Fourth Edition O'Reilly SQL Pocket Guide

Mid-Major Programming Languages — 3,000 — 9,999 units in 2011

The news in this set of languages is that 3 of the 11 languages saw growth in 2011 so the grouping as a whole is off by about 18.35%. This is partially due to a couple of the languages moving up to the prior category — most notably R and SAS..

*Mid-Major* U N I T S T I T L E S M A R K E T S H A R E Language 2010 Units 2011 Units 2010 Titles 2011 Titles 10Mkt Share 11Mkt Share Pl/Sql 10,774 9,322 24 35 0.63% 0.52% Smalltalk 9,294 8,716 2 2 0.54% 0.48% MatLab 8,332 8,504 22 27 0.49% 0.47% SPSS 8,973 7,598 16 18 0.53% 0.42% Basic 5,540 6,168 7 10 0.32% 0.34% VBScript 10,849 5,940 12 12 0.64% 0.33% Assembly 4,391 3,645 18 20 0.26% 0.20% Lua 6,309 3,636 7 8 0.37% 0.20% MDX 5,140 3,626 8 8 0.30% 0.20% nxt-g 1,172 3,535 4 5 0.07% 0.20% F# 4,939 3,285 7 6 0.29% 0.18%

Here are the top titles for the Mid-Major languages.

Dummies Beginning Programming For Dummies Microsoft Press Microsoft Visual Basic 2010 Step by Step Sams Sams Teach Yourself Visual Basic 2010 in 24 Hours Complete Starter Kit Oxford University Press Getting Started with MATLAB: A Quick Introduction for Scientists and Engineers Dummies SPSS For Dummies

Mid-Minor — 1,600 — 2,999 units in 2011

The news in this category is the growth of functional languages, like Scheme, Scala, Haskell, Alice and Lisp. These languages showed a nice 51.38% growth in 2010 and another 11.17% growth in 2011.

*Mid-Minor* U N I T S T I T L E S M A R K E T S H A R E Language 2010 Units 2011 Units 2010 Titles 2011 Titles 10Mkt Share 11Mkt Share Scala 2,531 2,844 5 5 0.15% 0.16% Haskell 1,051 2,129 5 6 0.06% 0.12% Lisp 1,684 2,038 4 6 0.10% 0.11% Bash 1,715 1,950 2 2 0.10% 0.11% Scheme 1,479 1,891 8 6 0.09% 0.10% VHDL 1,785 1,802 18 15 0.10% 0.10% Alice 2,713 1,745 10 9 0.16% 0.10% UnrealScript 3,028 1,736 3 3 0.18% 0.10% Blitzmax 1,836 1,604 2 2 0.11% 0.09%

Here are the top titles for the Mid-Minor languages.

Artima Programming in Scala: A Comprehensive Step-by-step Guide O'Reilly bash Pocket Reference No Starch Press Land of Lisp: Learn to Program in Lisp, One Game at a Time! No Starch Press Learn You a Haskell for Great Good!: A Beginner's Guide Sams Mastering Unreal Technology, Volume I: Introduction to Level Design with Unreal Engine 3

Minor Languages — 1,000 — 1,599 units in 2011

This category of languages saw 4 of the 7 languages sell more units in 2011. There was roughly a 1.91% increase in units sold year-over-year. The biggest gains were Puppet and Stata. Even if there was only one book per language in this grouping, it would be hard to financially justify a print product because of economies of scale; unless of course the publisher of a title is selling lots of copies direct on their website or direct to corporations or colleges. The combination of digital and print may help some of the titles below this point actually make more financial sense. My experience says anything below this point is not financially beneficial to the publisher or author, yet there may be other reasons they are in print.

*Minor* U N I T S T I T L E S M A R K E T S H A R E Language 2010 Units 2011 Units 2010 Titles 2011 Titles 10Mkt Share 11Mkt Share Clojure 1,332 1,532 2 4 0.08% 0.08% Puppet 659 1,518 1 3 0.04% 0.08% Stata 856 1,511 6 4 0.05% 0.08% Labview 1,682 1,443 3 2 0.10% 0.08% Scratch 1,112 1,263 2 2 0.07% 0.07% Groovy 2,523 1,205 7 9 0.15% 0.07% AWK 1,200 1,074 2 2 0.07% 0.06%

Here are the top titles for the Minor languages.

Stata Press A Gentle Introduction to Stata, Third Edition Apress Pro Puppet Manning The Joy of Clojure: Thinking the Clojure Way Course Technology Scratch Programming for Teens Oxford University Press Hands-On Introduction to LabVIEW for Scientists and Engineers

Linelist — 100 — 999 units in 2011

This grouping of languages saw 12 of the 28 languages sell more units in 2011, although the sales volume is fairly insignificant. But there was significant erosion in this group as it collectively saw a -46.96% decrease in units sold year-over-year. I am not going to list the bestsellers, because they are not exactly bestsellers, by any measure, in this grouping. Here is the list of languages selling more than 100 units and less than 1,000 in order by the most units to the fewest:

d, erlang, opencl c, applescript, mathematica, latex, tcl, jscript, minitab, apex, autolisp, pure data, fbml, hla, peoplecode, opengl shader, spin, fortran, pig, silverlight, mel, sparql, kml, linden script, lingo, coffeescript, sml, cobol.

TheRest Programming Languages — < 100 units in 2011

Lastly, the following languages sold fewer than 100 units in 2011. Here is the list in descending order:

abap, blitz3d, octave, nxt, gml, pascal, prolog, ml, chef, x++, inform, cfscript, cfml, racket, boo, jcl, siebel escript, idl, javafx, mvpl, m, verilog, jsl, nxc, limbo, mysql spl, rexx, rpg.

The following languages did not see any units sold in 2011. Here is the list in alphabetical order:

ada, awd, bondi, c/al, cachesql, cl, cs2, delphi, directx, dsl, e, egl, eiffel, go, haxe, maxscript, mumps, natural, ocaml, oopic, opl, pda languages, phrogram, pl/1, q, qbasic, realbasic, s, spark, squeak, windows script, xquery.

Next up, post 5 in this series will look at digital sales.

July 01 2011

Music and lyrics and code

If the popularity of Geek Choir at various tech-related conferences is any indication, there's a substantial correlation between producing music — whether vocally or with an instrument — and coding.

Michael Brewer (@operatic), application programmer specialist at the University of Georgia and a speaker at OSCON 2011, got the official Geek Choir sessions started at the Open Source Bridge and O'Reilly OSCON conferences. In a recent interview, he discussed how the choir came about and how music and coding complement each other.

How do music and technical aptitude intersect?

MichaelBrewer.jpgMichael Brewer: Since Geek Choir got accepted, I've been hearing a lot of anecdotal evidence of a high crossover between music and geek aptitude. Of course, people have been talking about the math-music connection since "Gödel, Escher, Bach." Recent studies have again shown connections between early exposure to music and math ability, although it's not exactly what we think of as the "Mozart Effect."

I tend to view it as a combination of pattern recognition and the ability to organize and reproduce thoughts about larger, more abstract concepts and their executions. Also, the production of sound using tools at hand — including vocal chords — is similar, in a sense, to producing code with software or hacking other projects.

We are a species that bonds with our tools in unusual ways. Correlation doesn't prove causation, though — there's a bit of a chicken-and-egg problem in figuring out if geeks are good at music or if musicians are good at being geeks.

Geek Lifestyle at OSCON 2011 — From fine-tuning your setup to taking the geek approach to growing your own food, we'll celebrate and explore hacker culture in all its richness in the Geek Lifestyle track at OSCON (July 25-29 in Portland, Ore.)

Save 20% on registration with the code OS11RAD

What is a "Geek Choir," and how did it get started?

Michael Brewer: I attended several OSCONs in the early-to-mid 2000s. I noticed several music jam sessions, as well as the popularity of the Gibson guitar booth in the exhibit hall one year. Folks were jamming on those guitars for hours.

At the first Open Source Bridge Conference, I suggested a Geek Choir session for the "unconference" on the last day. We started with fairly few people, but once we started singing — and people in the halls heard us and "voted with their feet" — we more than tripled our attendance in 15 minutes. The next year, Geek Choir made it into the OSBridge main conference. We had a very successful and enjoyable session, mixing experienced singers with absolute newbies.

In your OSCON session description, it says there's no advanced prep for the session choirs — why did you decide to go that route, and what benefits does a no-prep environment create?

Michael Brewer: It makes it easier on the newbies if everyone is getting introduced to the music at the same time. Also, it means that I have to be sure in my preparation to select music that is both accessible for inexperienced singers and worthwhile for experienced musicians. It's a good engineering challenge.

What are some tips for putting together a Geek Choir?

Michael Brewer: Here's a few:

  • Be welcoming and respectful. Everyone can contribute, even if they have never sung in public before or don't read music.
  • Choose — or compose — music that can be done by a mix of voices, both in terms of range and skill level. Parts can be done, but they have to be fairly straightforward to pick up. Shape note songs are good for this, as they were specifically engineered to (a) be easily learned by the (somewhat) untrained American choirs of the late 1700s and early 1800s, and (b) be performed with mixed genders on the various lines — sopranos and tenors would both sing the melody, for example, and altos and basses the bass parts, at comfortable octaves for each.
  • Stay in the public domain when you can. There are some tremendous repositories of music, including the Choral Public Domain Library and the International Music Score Library Project.
  • Have a great time!

There are a variety of open source tools for arranging music. Which do you recommend?

Michael Brewer: In general, there are two types of music composition software: music sequencers, which work with manipulating and combining blocks of sounds (sequences) into larger musical works, and notation software, which deals primarily in written or printed music.

Wonderful music is created with either. I generally work with notation software, so I'm much more familiar with notation editors. In this arena, everyone is chasing the main two commercial products — Finale (which I use) and Sibelius. For a long time, the open source tools weren't really comparable, in terms of ease of use, but MuseScore has really closed the gap. There's also LilyPond. I haven't worked with it yet, but I've heard good things about it.

What similarities, if any, do you see in the communal qualities of music and the communities that grow around open source projects?

Michael Brewer: There are several:

  • Both are groups of people coming together to create something, be it software or music.
  • There is artistry in the finished product for both. Code is most definitely art.
  • People vote with their feet for both, in terms of joining and leaving.
  • Coming together to work on common tasks builds connections and solidarity among the members. They tend to view themselves as a collective, giving themselves an identity as part of a larger whole.


June 08 2011

Four short links: 8 June 2011

  1. Who Writes Wikipedia -- reported widely as "bots make most of the contributions to Wikipedia", but which really should have been "edits are a lousy measure of contributions". The top bots are doing things like ensuring correctly formatted ISBN references and changing the names of navboxes--things which could be done by humans but which it would be a scandalous waste of human effort if they were. We analyse edits because it's easy to get data on edits; analysis of value is a different matter.
  2. How I Failed and Finally Succeeded at Learning How to Code (The Atlantic) -- great piece on teaching and learning programming, focusing on Project Euler. Kids are naturally curious. They love blank slates: a sandbox, a bag of LEGOs. Once you show them a little of what the machine can do they'll clamor for more. They'll want to know how to make that circle a little smaller or how to make that song go a little faster. They'll imagine a game in their head and then relentlessly fight to build it. Along the way, of course, they'll start to pick up all the concepts you wanted to teach them in the first place. And those concepts will stick because they learned them not in a vacuum, but in the service of a problem they were itching to solve.
  3. The Believing Brain -- Belief comes quickly and naturally, skepticism is slow and unnatural, and most people have a low tolerance for ambiguity.
  4. 3D Printed Rocket -- stainless steel rocket engine.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!