Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

May 23 2012

Clojure's advantage: Immediate feedback with REPL

Chas Emerick (@cemerick) is the co-author of "Clojure Programming" along with Brian Carper and Christophe Grand. He maintains a busy blog at and he also produces "Mostly Lazy … a Clojure podcast".

I asked Chas to enumerate some of the topics that would make a difference to developers: something that would attract attention to Clojure as a language, so we wouldn't spend time talking about yet another syntax. One of the first things he immediately mentioned was REPL. Writing code in Clojure is often about making changes and immediately seeing your results. Clojure has emphasized this shell-like approach to development and created an environment that allows for immediate evaluation of code and incorporating changes into a running process.

Clojure differs from other languages in that this interactive shell isn't an afterthought. Ruby's IRB or Java's beanshell are similar attempts at interactivity, but they are not primary features of each language. With Clojure, REPL is built in, and you can connect to any running Clojure process and modify and execute code. In this interview we discuss some of the possibilities that this introduces for Clojure developers.

The full interview is embedded below and available here. For the entire interview transcript, click here.

Highlights from interview:

On what is unique about REPL

"...what's really unique about Clojure is that most people's workflow when developing and using Clojure is tightly tied to the REPL, using the dynamic interactive development capabilities that the REPL provides to really boost your productivity and give you a very immediate sense of control over both the Clojure runtime, the application, programming or service that you're building." [Discussed 00:52]

How does REPL affect your development?

"Generally what you do in Clojure is you start up a JVM instance that is running the Clojure runtime and starts a REPL that you connect to ... then you stay connected to that running Clojure runtime for hours, days. I've had Clojure environments running for weeks in an interactive development setting, where you gradually massage the code that's running within that runtime that corresponds to files you have on disc.

"You can choose what code to load into that environment at any time, massage the data that's being processed by your application. It's a great feedback mechanism and gives you an immediate, fine‑grain sense of control over what you're doing [Discussed 01:18]

On using REPL to deploy production patches

"It's true, you can start up a REPL running on Heroku right now, connect to it and modify your application. Everything will work as you would expect if you happen to be running the application using Foreman locally.

"You don't want to be, in general, modifying your production environments from an interactive standpoint. You want to have a repeatable process ... Depending on the circumstances, it can be reasonable in a 'fire drill' situation to push a critical, time‑sensitive patch out to production ... 99% of the time you probably shouldn't reach for it, but it's very good to know that it's there." [Discussed 05:44]

On using REPL for direct access to production statistics

"JMX is great in terms of providing a structured interface for doing monitoring. But you need to plan ahead of time for the things you're going to monitor for and make sure you have the right extensions to monitor the things that you care about.

"Having a REPL available to connect to in every environment — whether it's development, user acceptance, functional testing or production — means that when you need to, you can get in there and write a one‑off function. Pull some data from this database, see what's going on with this function, capture some data that you wouldn't normally be capturing, stuff that may be far too large to log or to practically get access through JMX. It's a great tool." [Discussed 05:44]

OSCON 2012 — Join the world's open source pioneers, builders, and innovators July 16-20 in Portland, Oregon. Learn about open development, challenge your assumptions, and fire up your brain.

Save 20% on registration with the code RADAR


November 30 2011

Developer Week in Review: Siri is the talk of the town

After a one-week hiatus, during which research was undertaken in waistline enhancement via the consumption of starch and protein materials, we're back to see what's been happening in the non-turkey-related fields.

Imitation is the sincerest form of flattery

SiriIt's an interesting time for the voice-enabled smartphone field. On the one hand, some industry pundits with vested interests are claiming that people don't want to talk to their phones and don't want them to be assistants. Perhaps they have forgotten that the original smartphones were offshoots of the PDA market, and that PDA doesn't stand for "public display of affection" in this case.

At the other extreme, we have Microsoft stating that Apple's Siri is just a knock-off of Windows Tellme, a claim that has been placed into question by several head-to-head comparisons of features.

Of most interest to the developer community are reports that the latest iOS beta release contains additional hooks to allow applications to integrate into Siri's voice recognition functionality. I talked about the possibility that Apple would be expanding the use of Siri into third-party apps a few weeks ago, and the new features in the beta seem to confirm that voice is going to be made available as a feature throughout applications. This would be a real game changer, in everything from games to GPS applications on the iOS platform.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

Computer science for the masses

Two interesting pieces of news this time around on the educational front. In the higher learning arena, Stanford is expanding its free online computer science courseware with several new classes, including one on machine learning. Although you can't earn a free degree this way, you can get computer-graded test results to go along with the recorded lectures. This material will be very useful, even to grizzly old veterans such as myself, who may have a hole or two in their theoretical underpinnings. For a bright high school student who has exhausted his or her school's CS offerings, it could also serve as a next step.

Meanwhile, in the UK, the government seems to be moving toward having all students learn the basics of programming. I worry about this, on two fronts. First, it is unclear if the majority of students really need to learn software engineering or would benefit from it. Force-feeding coding skills into students who may not have the aptitude or proclivity to want to learn them seems unwise to me and is likely to slow down the students who might actually have a desire to learn the subject. Second, I have my doubts that a government-designed software engineering curriculum would actually be any good.

Is there anything JavaScript can't do?

JavaScriptJavaScript is often derided by "serious" computer professionals as a poorly designed toy language unfit for "real" software engineering. Yet, those who spend time using it know that you can produce some impressive results with it.

For example, there is now a JavaScript implementation of the OpenPGP message specification, which would allow JavaScript code to send and receive encrypted messages. And if you really want to go out on a limb, you could always develop a Java Virtual Machine byte code interpreter written entirely in JavaScript (somewhere, James Gosling is crying ...).

There's no question that JavaScript has its weak points, but its near-ubiquity makes it an incredibly useful spanner to carry around in your tool belt. Developers, sneer at your own risk. Like cockroaches, JavaScript may be around well after some more traditional languages have turned to dust.

Got news?

Please send tips and leads here.


July 07 2011

A rough guide to JVM languages

O'Reilly is celebrating the release of Java 7, and our inaugural OSCON Java conference: July 25-27 in Portland, Ore.

The possibility of using alternative languages on the JVM has always been an appealing side story to Java. Jython and JRuby were early pioneers in implementing a dynamic language on top of a statically typed VM. Now that Java 7 directly includes support in the JVM for dynamic languages, the coexistence is official.

Pick any of the languages here and you can expect the support of the robust JVM threading and garbage collection, plus access to a broad array of application libraries.

Up and coming


The pragmatist among JVM languages, Scala is a general-purpose programming language. It was conceived and is generally viewed as a "better Java," and of all alternative JVM languages has the best acceptance in an enterprise setting.

Scala combines the familiar feel of object-oriented Java with strong language support for concurrency, XML and functional programming features: many of the tools that contemporary complex and scalable systems require. Scala also takes a lot of the awkwardness out of Java code, through features such as type inference and traits.

object HelloWorld {
    def main(args: Array[String]) {
      println("Hello, World!")

Scala-related content at OSCON Java:

OSCON Java 2011
To celebrate the release of Java 7, the first 77 people registering today for OSCON Java with code JAVA7 will get the pass at a discounted price of $700 (applicable to OSCON Java package only).

Register now with code JAVA7


Clojure is a functional programming language based on Lisp. Through careful design, Clojure is simpler to read and use than Lisp, and it interacts cleanly with the Java world. Its functional nature makes programs very concise and composable.

In common with Scala, Clojure is designed with concurrency in mind. Its variables are immutable, and the use of software transactional memory and agents help manage shared mutable state in a more sustainable way than locking.

The experience of Clojure's inventor Rich Hickey is reason enough alone to give it a whirl. I once happily spent an hour listening to him describe the design and implementation of sequences in the language.

A "Lisp that could," Clojure is finding increasing (and surprising, to most observers) traction and acceptance. One contributing factor to this is an advanced build and package management infrastructure in leiningen and Clojars. Salesforce-owned hosted platform provider Heroku recently added Clojure as its third supported environment, following Ruby and Node/JavaScript.

A small but important omission from the Clojure ecosystem is a port of the book The Little Schemer.

(println "Hello, World!")

Clojure-related content at OSCON Java:

Tried and tested


A mashup of ideas from Python, Ruby and Smalltalk, Groovy gained early traction in the world of JVM languages, reaching its stable release in December 2007. Having been around a while, Groovy strongly acknowledges the Java world into which it was born, and is a good choice for Java developers wishing to use a more agile and dynamic language.

One of Groovy's jewels is Grails, a high-productivity web development environment inspired by Ruby on Rails. The Groovy world has also spawned Gradle, a modern project automation system for Java and JVM languages, providing an alternative to the established Ant and Maven projects.

println "Hello, World!"

Groovy-related content at OSCON Java:


Another veteran of the JVM language scene, Rhino is an implementation of the JavaScript programming language in Java. Part of the Mozilla project, it is typically used by developers to add user scriptability to their applications using JavaScript, though it is also used in situations where a system is predominantly implemented in JavaScript.

First released in 1999, Rhino has been around the block a few times. As a JavaScript engine, it is facing increasing competition from the C++-based V8, but retains the immense advantage of Java interoperability.

print('Hello, World!');

Remastered classics


A port of the Python language to the JVM, Jython offers some advantages over and above using Python, including Java's multi-threading and the ability to statically compile into Java classes. As a Python terse object-oriented language, programmers can quickly prototype in Jython, creating hybrid Java-Jython systems. Jython also offers developers an alternative to Rhino's JavaScript for embedding scriptability in their applications.

Jython also lets web developers bring popular Django web application framework into a Java setting.

print "Hello, World!"


The Java implementation of Ruby is no second-class citizen. JRuby has been recorded as better performing than Ruby 1.8, and strives for C-Ruby compatibility. Users of JRuby can benefit from using the Ruby on Rails web framework, or add Ruby scriptability to their applications.

One appealing aspect of building a system in JRuby or Jython is that it gives developers the option of reimplementing performance-critical code in Java without having to switch platforms as a project matures. Twitter's migration from Ruby to the JVM is a case in point.

puts "Hello, World!"

JRuby-related content at OSCON Java:

The Wild West

The JVM remains a fertile ground for language experimentation, as several of the languages featured in OSCON's Emerging Languages track demonstrate, including Seph and Gosu. As a platform, the JVM bootstraps language inventors and experimenters today much as lex and yacc did two decades ago. At OSCON Java, aspiring language creators should pop along to Charles Nutter's JVM Bytecode for Dummies.

There's still time left to register for OSCON Java. See you in Portland, 25-27 July!


July 05 2011

Four short links: 5 July 2011

  1. Conference Organisers Handbook -- accurate guide to running a two-day 300-person conference. Compare Yet Another Perl Conference guidelines.
  2. Twitter Shifting More Code to JVM -- interesting how, at scale, there are some tools and techniques of the scorned Enterprise that the web cool kids must turn to. Some. Business Process Workflow XML Schemas will never find love.
  3. Louis von Ahn on Duolingo -- from the team that gave us "OCR books as you verify you are a human" CAPTCHAs comes "learn a new language as you translate the web". I would love to try this, it sounds great (and is an example of what crowdsourcing can be).
  4. Fully Bayesian Computing (PDF) -- A fully Bayesian computing environment calls for the possibility of defining vector and array objects that may contain both random and deterministic quantities, and syntax rules that allow treating these objects much like any variables or numeric arrays. Working within the statistical package R, we introduce a new object-oriented framework based on a new random variable data type that is implicitly represented by simulations. Perl made text processing easy because strings were first-class objects with a rich set of functions to operate on them; Node.js has a sweet HTTP library; it's interesting to see how much more intuitive an algorithm becomes when random variables are a data type. (via BigData)

June 06 2011

Why OSCON Java?

What is OSCON Java? It's a good question. There are many Java conferences on every continent except Antarctica. Why is O'Reilly throwing its hat in the ring?

The Java community has always been a broad, fractious, interesting mess, capable of doing surprising things with little warning, and that's precisely why we're attracted to it. It's undeniable that Java is huge; it's been in one of the top two slots on Tiobe's Programming Community Index since Tiobe started in 2002. It's always been one of the largest components of the technical book market. Java's 2010 book sales represent a resurgence since 2008, but even in its weakest years, Java has always been one of the largest components of the book market. Beyond being huge, Java is one of the key languages of the open source movement. While there has been plenty of discussion over the years of the JDK's status as open source software, there has been no shortage of open source projects. SourceForge lists more than 25,000 Java projects, more than any other language.

A fertile ecosystem

Over the years, the Java community has birthed many of the most important ideas in software development, or was the conduit through which these ideas received broader audiences. Java was the first language to address interactivity and programmability in web pages: inadequately, as it turned out, but applets were still the first step on the road from a purely static web to the modern interactive web. The MVC design pattern was the basis for the Swing GUI framework, and was later adapted for many web frameworks. When it first came out, EJB didn't represent a new idea — the ideas came from products reaching back to the '60s, like IBM's CICS — but it enabled many people to write platform-independent database-backed software and push the web forward. And yes, the mistakes of EJB were one of the inspirations behind Ruby on Rails. Another response to EJB's complexity, the Spring Framework, grew from a relatively simple implementation of the Inversion of Control (IOC) design pattern to one of the dominant forces in Enterprise Java development.

It's amazing how much of the revolution in software development tooling originated in Java. Ant replaced Unix's make(1) utility, which had long outlived its usefulness. In turn, Ant has been superseded by Maven and Gradle. The ideas behing Extreme Programming, which morphed into the less strident Agile methodology, entered the world through Java. Java saw the first unit testing tools: according to legend, Erich Gamma and Kent Beck wrote the first version of JUnit while they were together on a long flight. Continuous Integration, another widely adopted agile practice, first appeared in the Java community with CruiseControl, which in turn has largely been supplanted by Jenkins.

Java has had a tremendous impact on the recent history of computing. But conferences aren't about the past, they're about the present and the future. Large as the Java community is, it's been some time since Java was the language that the "cool kids" used. At the second JavaOne, one of the Moscone Center staff said that it was the youngest crowd he'd ever seen there. That's certainly no longer true. We've all grayed. That may be why Oracle is presenting Sting and Tom Petty on stage at this year's JavaOne.

It's fair to say that Java stagnated. I was surprised to find that Enterprise Java looks basically the same as it did several years ago. EJB has been massively improved (EJB 3.0 and 3.1 are light years beyond the mess of EJB 2), and Spring has grown from a spunky upstart project to a mature part of the community, with major corporate sponsorship by VMWare. But we're still talking about JSF, Java EE, and enterprise web frameworks (well over a hundred at last count). The enterprise world is still clicking along, making lots of money, but what's changed? What's interesting? What's new?

Of course, there's plenty new. First, look at data. I'm not surprised at the rise of "big data" and data products over the past few years, and I am pleased at the role Java has played in all aspects of that movement. The Hadoop ecosystem is large and rich, but it's not just Hadoop. Cassandra is also a Java project. And while Cassandra is similar in many respects to HBase (part of the Hadoop project), Neo4J is another new Java database that has a completely different model, designed for storing data as graphs. Nor is it just map/reduce and databases. The Incanter statistics library is an important part of the Clojure world, Clojure itself being one of the many languages that have sprung up around the JVM.

There's more to come: I recently had a conversation with one of the authors of Storm, a very new (still unreleased) project designed to facilitate massively parallel realtime processing. It's written in Java, though its initial release will support client programming in many languages.

Another area in which Java has had a huge impact has been mobile. Yes, there is plenty of tension between Oracle and Google, but one can't deny the importance of Android in increasing interest (and drawing younger developers) to Java. An article recently pointed out that really innovative apps tend to appear first on Android, and then move to the iPhone. That matches my experience; Android's openness (which, yes, is debated) often allows developers to combine features in exciting and original ways, such as using NFC to read older RFID tags, a hack that Brian Jepson demonstrated at Google IO. And I expect the new Accessory API and toolkit to revolutionize what we can do with our mobile devices.

OSCON Java 2011, being held July 25-27 in Portland, Ore., is focused on open source technologies that make up the Java ecosystem. (This event is co-located with OSCON.)

Save 20% on registration with the code OS11RAD

The jewel of the JVM

Finally, it would be amiss to write about what's exciting in Java without mentioning the explosion of new programming languages that run on the JVM. At a JavaOne fireside talk a few years back, James Gosling said that the JVM was the "crown jewel" of the Java world — not the language itself. He's been proven right. There have always been many strange JVM languages kicking around; Wikipedia lists 67. I believe that's an understatement by a factor of two or three, though most of these languages are oddball curiosities. However, we finally have some JVM languages that are really sticking, and having a long-lasting impact. I've already mentioned Clojure and Groovy. JRuby and Scala are also in it for the long haul. It's important to understand that the Java community isn't just solely the Java language, it's about the fabulous work that's gone into the JVM, and the rich ecosystem of libraries that make the JVM a desirable target for innovation.

Java, for all its deficiencies, is widely used: in big data, mobile, enterprise, web, and cloud computing. I don't think there's any other language that can claim to be so important in so many fields. As Bjarne Stroustrup said, "There are only two kinds of languages: the ones people complain about and the ones nobody uses." He also said, "There are more useful systems developed in languages deemed awful than in languages praised for being beautiful." Java has plenty of people willing to complain; as I wrote in 2006, people have been dancing on Java's grave ever since Gosling first announced it at Usenix in 1995. But those complaints have never stopped good people from doing useful and important work in Java.

OSCON Java highlights

OSCON Java and its related conferences, OSCON Data and OSCON, will track the most recent developments in the Java world. I expect some of the highlights to be:

  • Languages: All the regulars are represented, but I'm particularly interested in Neal Ford's talk on functional thinking.
  • Tools: I look forward to hearing about Gradle, the Groovy DSL for building, testing, and deploying software; and Jenkins, the continuous integration framework.
  • Mobile: You're not surprised that we have Android. What's more surprising is that we have a session on cross-platform mobile development using tools like GWT and PhoneGap. (There is more Android content in the main OSCON event.)
  • Robotics: One area that Java hasn't hitherto penetrated has been robotics, but we've got a session about Robotics Programming — with live robots!
  • Cloud computing: The cloud has changed the nature of system administration and management. It's not a bunch of guys who mumble at consoles anymore, it's developers who build software to manage thousands of hosts at a time. BrowserMob will be showing how they've developed tools in Java to manage the cloud in bulk.

We've got a great array of keynoters, including Raffi Krikorian talking about the Twitter's transition from Rails to Java; you can expect an inspirational and thoughtful address from Kathy Sierra; and we'll have Josh Bloch, Martin Odersky, and others on the main stage to puzzle, delight, and challenge us.

There will be no shortage of interesting talks, hallway conversations, and birds of a feather sessions. OSCON Java is also under the broader umbrella of OSCON, which also includes OSCON Data, so you can also check out what's happening in those conferences as well.

OSCON Java is a celebration of the rich open source Java ecosystem, an ecosystem that's been built up over the past 15 years of work on Java by the many participants that make up the community. We're thrilled to be a part of it. Java developers have produced great things in the past, you are producing great things now, and we look forward to what you produce in the future.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!