Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

April 23 2012

The rewards of simple code

Based on his experiences as chief architect on the Bugzilla Project and his current employment at Google as a software engineer, Max Kanat-Alexander (@mkanat) has formulated a method and philosophy for designing effective code that is, at its core, all about simplicity. He wrote a book about these ideas, "Code Simplicity," in which he explains how developers can optimize the structural plans for their software projects.

In the following interview, Kanat-Alexander discusses the negative factors that can lead to messy code and why a large number of developers on a project doesn't have to undermine the quality of software design.

What are the factors that lead to "non-simple" code?

Max Kanat-Alexander: When I look over all the projects that I've worked on, everything I've read in the field of software development, and all my experiences interviewing developers over the years, most of the non-simple code I see results entirely from human factors. Some people say that the tools are making their lives hard — and they often are — but, ultimately, complexity comes down to whether or not the developer writing the code has a deep understanding of both the specifics of his or her job and the fundamental principles of software development.

It's those fundamental principles of development that I'm working toward with this book. You know how sometimes a senior developer just knows what they're supposed to do, but they can't really explain why to other people? Partially, that's what brings us so much non-simple code in the world, because we can't pass down that knowledge effectively to less senior developers. That's one part of what the book is here to solve: boiling down all of that knowledge and experience into broad, solid truths that anybody can immediately understand just by reading them.

But there's even stuff that senior developers forget or simply don't know. When you're relying entirely on experience to teach people, they're not all going to have the same experiences. So there's been no guarantee that a developer will ever get all the basic principles they need to be a truly excellent programmer.

How does the number of developers involved in a software project affect the code's simplicity?

Max Kanat-Alexander: There's the famous quote from "The Mythical Man-Month" that adding developers to a late project makes it later. Complexity is the primary reason for that. But a project doesn't have to be complex just because it has several developers on it.

It's true that adding new people is going to introduce some complexity, so you should be prepared for that. You'll want to make sure you have a solid foundation of simplicity in your existing system before you start throwing lots of new people at it, and you can't throw them at it faster than that foundation will support. Otherwise, the new folks will develop complexity into it.

But there's no grand rule that says "a project must be complex if it has X developers." Theoretically, with the right development practices and the right design of your system, you could have a nearly infinite number of people working on one system and still prevent complexity. I think your work might slow down enormously due to all the communication that was going back and forth, but the system could be kept simple if you all really worked at it.

I'm in favor of small teams, myself, but that doesn't mean it's impossible to build simple systems with large teams. It's just much harder than doing it with small teams, and it's probably going to happen at a much slower pace.

So, to answer your question, having lots of people doesn't naturally lend itself to complexity, but in the practical world of everyday programming, it sure makes it harder to keep things simple.

What is the wrong way to code?

Max Kanat-Alexander: The wrong way would be to write software without understanding it. The right way would be to write software with as much understanding about it as you can possibly get.

The more you know about what you're doing, the better you can do it. And that's what the book is there to give you: the foundations of a science so that you can have the most important understandings there are about software development, and then build on those ideas in your mind and in your day-to-day development work.

What immediate steps can developers take to make their code simple and more efficient?

Max Kanat-Alexander: Let's talk first about the word "immediate." Most developers get into trouble with their code because they either don't plan for the future at all, or they make some sort of bad prediction about the future because they're impatient and don't want to wait for the future to arrive so they can actually see it. So, if you want to do something immediately to make your code simpler, it would be to stop thinking so immediately. Start thinking about how much work you're going to be doing maintaining the system in the future. Maybe it's worth a little, or even a lot, of work right now so that your system stays easy to maintain. I know that it's hard to refactor sometimes, and that you really just want to get that feature out there, but it's worthwhile to be a little more responsible about the state of your code base. You want to make it simpler before you start adding a bunch of new complexity with some great new feature. It's the "future you" that's going to benefit from all this, just as much as your teammates. Make "future you" happy.

So, with that in mind, we can tackle the word "efficient." If we're talking about being efficient in terms of speed that the program runs at or efficient in terms of space that something takes up, the thing you want to do first is make sure the system is simple and easy to maintain. Because even if you can optimize something now, that doesn't mean that will be the optimal solution forever. And if you add complexity to the system prematurely just to optimize something, you may not be able to keep the system running optimally over time. So, it's really simplicity that leads to long-term efficiency, even though sometimes short-term efficiency gains come from adding complexity. I'm not saying you should never optimize — you should, when it matters. But you have to understand the trade-off that you're making, and you have to think a bit about how the future's going to mess with your optimizations.

Do you have your own philosophy or set of principles about good software design?

Max Kanat-Alexander: I have this broad set of opinions about software development, and then underlying that, there's a set of hard facts that I've researched and discovered. The book focuses on those facts, with an occasional small bit of my viewpoint thrown in about those facts.

I could write several other books about my personal thoughts on software, and I'm sure there's a lot more to discover about it, too. But I thought it was more important to develop and deliver a scientific basis for software development first. Once that's done, we can start to have lots of interesting theoretical discussions about what ideas or opinions might be best.

Code Simplicity — This concise guide helps you understand the fundamentals of good software design through principles you can apply to any programming language or project.

This interview was edited and condensed.


December 09 2011

Developer Week in Review: Developers are our most important asset?

I'm about halfway through the Steve Jobs biography (currently in the middle of the NeXT years), and I am continually struck by the dissonance of Jobs' incredible insight about some things and total blindness to others. It reminds me of an observation I used to make about some people: There's a reason Wisdom and Intelligence were two different stats in Dungeons & Dragons.

Not all of us are born to ascend to such lofty heights of fame, but it's nice to know that ...

Developers: Not just nameless cogs in the machine?

People who make a living making software have had reason to be nervous over the last decade. The trend seems to have been a race to the bottom of the salary scale, with generic talent in developing countries valued on par with highly experienced developers at home. Even when companies were willing to acknowledge that an external developer might not be as productive as an experienced one who has been working for the company for many years, the argument was made that since you could hire four or five foreign developers for the price of the stateside talent, the economics still made sense.

Now, an interesting essay in Forbes makes the argument that really good developers aren't twice or three times as valuable as the average, but 10 times or more. The essay's author, Venkatesh Rao, puts forward the proposition that smart companies such as Google recognize the value of retaining their high-end talent and deliberately shower them with lavish perks to keep them from straying. Rao argues that because developers require so little support infrastructure (as opposed to a biologist, for example, who needs staff and a lab), highly productive software engineers have a direct multiplier effect on the bottom like.

I've been noticing the beginnings of a backlash against the offshoring fad, as it seems have others. The question, of course, is whether average companies are able to look past the immediate cash-flow factor and evaluate the value of their staffs based on things such as the end quality of the product and (to quote Mr. Jobs) how insanely great the results are.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

Data centers and the boonies economy

The news has been alight recently with stories of large companies (Apple, Google, Facebook, Microsoft) setting up massive data centers in out-of-the-way locations. The reasons for this are several, and they've resulted in a perfect storm of motivations for placing data centers in rural locations.

First off, data centers require a lot of square footage, and a million square feet in the greater San Francisco Bay area is obviously a lot more expensive than in the wilds of Washington state. Construction costs are likely to be lower as well. Tax rates tend to be lower, and small country towns are more likely to offer incentivized tax rates to bring in jobs.

Second, data centers don't really require a lot of high-end staffing, beyond a resident engineer or two to keep things humming. Security personnel and maintenance staff are non-skilled positions that can be hired as easily in the mountains of the Ozarks as in downtown New York. And once again, they are likely to work much cheaper because the cost of living in rural locations is so low.

Many rural locations also are close to low-cost power generation sources, such as dams. Since electricity is a major cost in data center budgets, getting your power locally can take a large bite out of operating budgets. In addition, there is a belt of climate that runs through areas such as Colorado that offers the ability to take advantage of open air cooling, rather than having to run costly air conditioning all the time.

There's also something to be said for geographic diversity. If The Big One ever hits San Francisco, it will take down pretty much any data center in the affected area. Scattering your eggs into multiple baskets is common sense.

The reason that this works for data centers, and less so for things such as manufacturing, is that all a data center needs to function is good connectivity. With so much dark fiber strung across the country, it's not that expensive to bring multiple "fat pipes" to even the most remote locations, especially when you factor in all the savings. Data centers don't need good rail infrastructure, highways, geographic centrality, or any of the other factors that drive location decisions for physical manufacturing.

Your mobile news roundup

Ignoring for the moment the continual cacophony of lawsuits and counter-suits that seem to be business as usual these days, there were actually some recent news items of note in the mobile space.

There's no publicly available API for developers to add their own mojo to Apple's Siri, but that hasn't stopped enterprising hackers from discovering a way to do a man-in-the-middle intercept and add their own functionality. Before anyone gets in a tizzy, it only works if the mobile user explicitly opts in by installing a self-signed SSL certificate so that HTTPS connections to the hacked Siri proxy succeed. It's not something that could be done behind your back. Once in place, you can insert your own Siri functionality by writing code in Ruby on the proxy server. I've tried it, and it's surprisingly easy to make Siri jump through hoops. Apple will probably close this loophole soon, but for the moment it offers a tantalizing look at how powerful general access to a voice interface could be. The word is that Apple is hiring two engineers to work on APIs for Siri. Evidently, Apple sees the potential, too.

In Android-land, Google announced that the ten billionth app had been downloaded from the Android Market. By comparison, the 15 billionth iTunes app purchase occurred this summer, as announced at WWDC. There's no question that Android's velocity is greater than iOS, due largely to the huge number of phones running Android now, most of which are significantly cheaper than an iPhone. Is it time to wonder if the iPhone is going to end up being the Betamax of phones, eventually done in by a flood of inexpensive competitors? Or will it end up being more like the MacBook, the choice of anyone who can afford them?

In Redmond, Microsoft is ramping up a developer infrastructure for its Windows 8 platform, and the company has decided to follow the app store model as well. It appears that Microsoft will be offering better terms for developers, something that shouldn't be surprising, given Microsoft's legendary loyalty to those who choose to follow the way of Windows.

Got news?

Please send tips and leads here.


November 10 2011

Developer Week in Review: Adobe raises the white flag on mobile Flash

To err is human, to err publicly is just plain embarrassing. I ran an item in last week's review that turned out to be kinda stale news. How stale? Well, it dated back to the last Bush administration. That stale ...

Moving forward, I'll try to avoid posting "classic" developer news, and keep things on the cutting edge. Such as:

Flash - 0, HTML5 - 1

Flash and HTML5For a long time, it appeared that Adobe Flash was going to become the de facto mobile application development platform. Apple's intransigence to adopt Flash on mobile Safari was considered a major knock against Apple, and when Apple opened the door to AIR-based iOS native apps, it was seen as Apple caving in to the desire for Flash developers to be able to deliver their apps onto iOS.

Somewhere in there, however, HTML5 came along and stole Adobe's lunch money. Adobe appears to have moved on to Kübler-Ross stage five, and has accepted that HTML5 has trounced Flash, at least in the mobile arena. The company has signaled to their employees that moving forward, Flash will not be supported on mobile platforms.

This is a much bigger story than just mobile, however. Mobile web traffic now accounts for 7% of the total, and is growing at a rate of nearly 1% every three months. As tablets become more popular, this number may skyrocket. Web content providers are unlikely to commit to developing web pages that can't be used well by such a growing demographic, and publishers/developers are likely to shift from Flash to HTML5 for RIA development. Adobe has a leg up on other tool chain providers because it has rich integration into tools such as Photoshop and Illustrator, but it will have to fight to keep this position.

On the mobile app side, Adobe can try to promote the AIR-to-native path, but it's going to be competing with a growing number of "write-once, run-everywhere" companies such as Appcelerator, as well as companies that choose to simply develop natively.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

Working toward the one-language-per-developer ratio

Frequent WIR readers will know that I'm no fan of language dilution, the process wherein new languages are developed and promoted with such frequency that software engineering becomes a Tower of Babble. It seems like a necessary step in the developing hubris of an organization that it decides to have the one true language that will make life a programmer's paradise. Google has done this several times, most recently trying to replace JavaScript. Now, the normally staid Eclipse foundation has joined the fray with Xtend.

Extend joins C# in the "looks just like Java, if you squint" language camp. The good news is that as a JVM-based language, it can share libraries with Java, so it's not starting from scratch. New code created in Xtend can be used by Java developers. But still, do we need another pretty-close-to-Java language? I spend my days coding alternately in Java and Objective-C, and the cognitive dissonance set up as I switch back and forth can generate major cranial pain. Is it 'this' or 'self'? Do I send a message with a dot or by putting it inside brackets? It's much easier to switch between languages that share nothing in common because it's the small differences that screw you up. Xtend is going to be another language close enough to one I already know to make me go nuts remembering the deltas between the two.

Because it's the only shape that can't fall into a manhole, that's why!

Work long enough in the industry, and you'll end up interviewing for a company that thinks trivia and brainteasers are a good way to test applicants. Increasingly, companies seem to think that tests and code challenges are the best way to find the "best of the best." Neil McAllister has an interesting essay in InfoWorld questioning if this really leads to the desired outcome.

I tend to agree, somewhat. Tests that require an applicant to pull obscure or advanced knowledge out of his or her head aren't good tests because they are essentially memory exercises. The best "challenge-style" test I ever had was when I applied for a job at ITA, now (ironically) a part of test-junkie Google. They sat me down in front of a system with carte-blanche Internet access and the ability to install any tools I wanted, and to use any language. Then, they presented me with a heuristic challenge: as I remember, it was to find all the possible anagrams of varying lengths you could find in a provided dictionary.

What I liked about this test was that the company seemed interested in my process, rather than my ability to immediately churn out the right answer. I sat with my minder for several hours, refining the code, adding features that he requested — much more like pair programming than a pure test. At the end of the day, they knew how I worked, how I found things I didn't know or remember, and my coding methodology. It was time-intensive, but much more useful, to my mind, than knowing why manhole covers are round.

Got news?

Please send tips and leads here.


July 21 2011

FOSS isn't always the answer

There's been some back and forth between various members of the technical press about whether the open source movement has lost its idealism, and the relative virtues of shunning or accepting proprietary software into your life. The last thing to cross my screen was an essay by Bruce Byfield that includes this gem:

In my mind, to buy and use proprietary products, except in the utmost necessity is something to be ashamed about. And that's how I've felt the few times I've bought proprietary myself. It seems an undermining of FOSS' efforts to provide an alternative.

Over the years, I've become less patient with the stridency of the FOSS movement, or at least some of the more pedantic wings of it. It is certainly not my place to tell anyone that they should buy or not buy any kind of software. However, the repeated assertions by members of the FOSS movement that proprietary software is somehow dirty or a corruption of principles has begun to stick in my craw.

There are plenty of places where FOSS makes all the sense in the world, and those are the places that FOSS has succeeded. No one uses a closed source compiler anymore, Eclipse is one of the leading IDEs for many languages, and Linux is a dominant player in embedded operating systems. All these cases succeeded because, largely, the software is secondary to the main business of the companies using it (the major exception being Linux vendors who contribute to the kernel, but they have a fairly unique business model.)

Where FOSS breaks down pretty quickly is when the software is not a widely desired tool used by the developer community. Much as you can quickly get to the Wikipedia philosophy page by repeated clicking on the first link of articles, you can quickly get to the requirement for a utopian society once you start following the line of assumptions needed to make consumer-level FOSS work.

The typical line of thought runs like this: Let's say we're talking about some truly boring, intricate, detail-laden piece of software, such as something to transmit dental billing records to insurers (this type of stuff is going to be required under the new health care laws.) Clearly, people don't write these things for kicks, and your typical open-source developer is unlikely to be either a dentist, or a dental billing specialist.

OSCON 2011 — Join today's open source innovators, builders, and pioneers July 25-29 as they gather at the Oregon Convention Center in Portland, Ore.

Save 20% on registration with the code OS11RAD

So, if all software should be free and open source, who is going to write this code? One argument is that the dentist, or a group of dentists, should underwrite the production of the code. But dentistry, like most things in western society, tends to be a for-profit competitive enterprise. If everyone gets the benefit of the software (since it's FOSS), but a smaller group pays for it, the rest of the dentists get a competitive advantage. So there is no incentive for a subset of the group to fund the effort.

Another variant is to propose that the software will be developed and given away, and the developers will make their living by charging for support. Leaving alone the cynical idea that this would be a powerful incentive to write hard-to-use software, it also suffers from a couple of major problems. To begin with, software this complex might take a team of 10 people one or more years to produce. Unless they are independently wealthy, or already have a pipeline of supported projects, there's no way they will be able to pay for food (and college!) while they create the initial product.

And once they do, the source is free and available to everyone, including people who live in areas of the world with much lower costs (and standards) of living. What is going to stop someone in the developing world from stepping in and undercutting support prices? It strikes me as an almost automatic race to the bottom.

And this assumes that software development is the only cost. Let's think about a game such as "Portal 2" or one of the "Call of Duty" titles. They have huge up-front costs for actors, motion capture, and so on. And they have very little in the way of potential revenue from support, as well. So do the FOSS proponents believe that modern computer games are all evil, and we should all go back to "NetHack" and "Zork"?

Let me be clear here: I am in no way trying to undermine anyone who wants to develop or use FOSS. But I have spent most of my adult life writing proprietary software — most of it so specialized and complicated that no open source project would ever want to take it on — and I find the implication that the work I do is in some way cheap or degrading to be a direct insult. When it has made sense, I have contributed work to open source projects, sometimes to a significant degree. But when it made sense, and when not, should remain my choice.

In many ways, the web is a perfect example of the marketplace of ideas. No one knows (or in most cases, cares) whether the technology under the covers is FOSS or proprietary. Individuals make the same measured decisions when selecting software for personal or business use. If there is a FOSS package that meets all the requirements, it tends to be selected. If it suffers in comparison to proprietary counterparts, it it may still be selected if the need to modify or extend the package is important, or if the price differential is just too hard to justify. But in many cases, proprietary software fills niches that FOSS software does not. If individual activists want to "wear a hair shirt" and go without functionality in the name of FOSS, that's their decision. But I like linen, thank you.

If there are people out there who are willing to engage in a reasoned, non-strident discussion of this issue, I'd love to talk it out. But they need to accept the ground rules that most of us live in a capitalist society, we have the right to raise and provide for a family, and that until we all wake up in a FOSS developer's paradise, we have to live and work inside of that context. Drop me a note or post a comment. I'd love to hear how a proprietary-free software world could work.


November 30 2010

Free to Choose ebook deal reveals the programmer zeitgeist

Allen Noren, who runs, including all our online e-commerce, sent around a list of the top titles resulting from our Free to Choose Cyber-Monday promotion. I was so struck by the titles on the list that I thought I'd like to share it more widely. While it's clearly a self-selected list -- the people who order directly from O'Reilly are more likely to be our core audience of cutting edge "alpha geeks," while people who buy in stores are more likely to buy consumer titles like the Missing Manuals -- it still gives a fascinating view of what's on the minds of that audience today.

Here's the list:

Here are a few of the things that jump out at me:

  • Python and Javascript have become the foundational languages for developers. We have been watching these languages for some time. HTML5 and node.js make Javascript even more important for web developers, while Python has a particularly large following in data science. It's particularly interesting, and important, that using Python to collect data from sensors (Real World Instrumentation with Python) made it onto the list.
  • The "big data" themes we've been sounding in conferences like Strata are resonating in our publishing business, as six of the top titles focus on data science. The rise of data science, coupled with the rise in data entrepreneurship, may well be the most important trend in computing.
  • HTML 5 matters. HTML5, coupled with Javascript, turns the browser into a full-fledged application platform that spans everything from phones to tablets to desktops. It is the big Web story for the next few years. The only question is whether some sort of "embrace and extend" strategy will harm portability and lead us back into browser-dependency hell.
  • Microsoft and Adobe aren't going away any time soon. It's easy to write both companies off: Microsoft for losing ground to Apple in the consumer audio, phone, and laptop markets, and Adobe for being banned from Apple's mobile devices. But they've both proven extremely adaptable. Microsoft's Kinect shows they can still produce a winner, and their quick turnabout on hacking Kinect demonstrates an agility that is rare even in much smaller companies. Adobe quickly made alliances with Google, and is developing tools to generate HTML5 from Flash.

It's no surprise that these themes are related: HTML5 drives the importance of JavaScript, big data drives the importance of Python, and both are driving the changes to which Microsoft and Adobe are reacting.

Would love to hear your thoughts.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
Get rid of the ads (sfw)

Don't be the product, buy the product!