Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

May 23 2012

Clojure's advantage: Immediate feedback with REPL

Chas Emerick (@cemerick) is the co-author of "Clojure Programming" along with Brian Carper and Christophe Grand. He maintains a busy blog at and he also produces "Mostly Lazy … a Clojure podcast".

I asked Chas to enumerate some of the topics that would make a difference to developers: something that would attract attention to Clojure as a language, so we wouldn't spend time talking about yet another syntax. One of the first things he immediately mentioned was REPL. Writing code in Clojure is often about making changes and immediately seeing your results. Clojure has emphasized this shell-like approach to development and created an environment that allows for immediate evaluation of code and incorporating changes into a running process.

Clojure differs from other languages in that this interactive shell isn't an afterthought. Ruby's IRB or Java's beanshell are similar attempts at interactivity, but they are not primary features of each language. With Clojure, REPL is built in, and you can connect to any running Clojure process and modify and execute code. In this interview we discuss some of the possibilities that this introduces for Clojure developers.

The full interview is embedded below and available here. For the entire interview transcript, click here.

Highlights from interview:

On what is unique about REPL

"...what's really unique about Clojure is that most people's workflow when developing and using Clojure is tightly tied to the REPL, using the dynamic interactive development capabilities that the REPL provides to really boost your productivity and give you a very immediate sense of control over both the Clojure runtime, the application, programming or service that you're building." [Discussed 00:52]

How does REPL affect your development?

"Generally what you do in Clojure is you start up a JVM instance that is running the Clojure runtime and starts a REPL that you connect to ... then you stay connected to that running Clojure runtime for hours, days. I've had Clojure environments running for weeks in an interactive development setting, where you gradually massage the code that's running within that runtime that corresponds to files you have on disc.

"You can choose what code to load into that environment at any time, massage the data that's being processed by your application. It's a great feedback mechanism and gives you an immediate, fine‑grain sense of control over what you're doing [Discussed 01:18]

On using REPL to deploy production patches

"It's true, you can start up a REPL running on Heroku right now, connect to it and modify your application. Everything will work as you would expect if you happen to be running the application using Foreman locally.

"You don't want to be, in general, modifying your production environments from an interactive standpoint. You want to have a repeatable process ... Depending on the circumstances, it can be reasonable in a 'fire drill' situation to push a critical, time‑sensitive patch out to production ... 99% of the time you probably shouldn't reach for it, but it's very good to know that it's there." [Discussed 05:44]

On using REPL for direct access to production statistics

"JMX is great in terms of providing a structured interface for doing monitoring. But you need to plan ahead of time for the things you're going to monitor for and make sure you have the right extensions to monitor the things that you care about.

"Having a REPL available to connect to in every environment — whether it's development, user acceptance, functional testing or production — means that when you need to, you can get in there and write a one‑off function. Pull some data from this database, see what's going on with this function, capture some data that you wouldn't normally be capturing, stuff that may be far too large to log or to practically get access through JMX. It's a great tool." [Discussed 05:44]

OSCON 2012 — Join the world's open source pioneers, builders, and innovators July 16-20 in Portland, Oregon. Learn about open development, challenge your assumptions, and fire up your brain.

Save 20% on registration with the code RADAR


May 03 2012

Editorial Radar: Functional languages

Functional Languages are driving a broader set of choices for programmers. O'Reilly editors Mike Loukides and Mike Hendrickson sat down recently to talk about the advantages of functional programming languages and how functional language techniques can be deployed with almost any language. (The full conversation is embedded below.)

Andy Hunt and Dave Thomas have long recommend learning a new language each year, especially those languages that teach new concepts [discussed at the 02:02 mark]. Functional languages have made that easier. They behave in a different way than the languages many of us grew up on — procedural like C or languages derived from C. Plus, the polyglot programming movement has driven the interest in functional languages as one of the languages you might want to learn.

Programmers need to understanding the advantages of using a functional language, such as productivity, power of expressiveness, reliability, stateful objects, concurrency, natural concurrency, modularity, and composability [05:37]. Though a search still exists for a magic bullet [06:29] to make it easier for programers to better solve the problem of concurrency. CPU speeds have been stuck at roughly the same level for the last four to five years. Programmers have been given is more transistors on a chip, hence more CPUs and more cores to work with making concurrency one of the most difficult issues facing computer scientists today. Enter functional programming with improved debugging and the ability to write more reliable code in a concurrent environment.

Additional highlights from this conversation include:

  • Print book sales of functional languages are growing, especially books on R programming. And while Loukides doesn't consider R to be a functional language, some debate exists about its classification. Though it's clear the data science movement has driven the use of R because it's well designed for statistics and dealing with data. [Discussed at the 00:29 mark]
  • We'll see F# grow in the Microsoft development environment while Scala and Clojure are dominating the open source space. Erlang will also be around for a long time for building highly reliable concurrent systems. [Discussed at the 03:01 mark]
  • Since the publication of Doug Crockford's JavaScript: The Good Parts, coders have discovered the functional language abilities of JavaScript and Java. Google's release of Maps and Gmail revolutionized how JavaScript is used. Some of today's best examples include Node for high-performance websites and D3 for creating exotic and beautiful data visualizations. [Discussed at the 08:15 mark]
  • While JavaScript isn't a functional language, it's designed loosely, so it's easy to use as a functional language. You might also be interested in how functional programming techniques can be used in C++ — a blog post written by John Carmack. [Discussed at the 10:36 mark]
  • Java isn't intended as a functional language. Though Dean Wampler's Functional Programming for Java Developers provides an approachable introduction to functional programming for anyone using an object-oriented language. [Discussed at the 11:41 mark]
  • The use of a functional language or functional language techniques can make your code more robust and easier to debug. [Discussed at the 12:09 mark]

You can view the entire conversation in the following video:

Tune in next month for a discussion of NoSQL and web databases.

Fluent Conference: JavaScript & Beyond — Explore the changing worlds of JavaScript & HTML5 at the O'Reilly Fluent Conference (May 29 - 31 in San Francisco, Calif.).

Save 20% on registration with the code RADAR20


July 07 2011

A rough guide to JVM languages

O'Reilly is celebrating the release of Java 7, and our inaugural OSCON Java conference: July 25-27 in Portland, Ore.

The possibility of using alternative languages on the JVM has always been an appealing side story to Java. Jython and JRuby were early pioneers in implementing a dynamic language on top of a statically typed VM. Now that Java 7 directly includes support in the JVM for dynamic languages, the coexistence is official.

Pick any of the languages here and you can expect the support of the robust JVM threading and garbage collection, plus access to a broad array of application libraries.

Up and coming


The pragmatist among JVM languages, Scala is a general-purpose programming language. It was conceived and is generally viewed as a "better Java," and of all alternative JVM languages has the best acceptance in an enterprise setting.

Scala combines the familiar feel of object-oriented Java with strong language support for concurrency, XML and functional programming features: many of the tools that contemporary complex and scalable systems require. Scala also takes a lot of the awkwardness out of Java code, through features such as type inference and traits.

object HelloWorld {
    def main(args: Array[String]) {
      println("Hello, World!")

Scala-related content at OSCON Java:

OSCON Java 2011
To celebrate the release of Java 7, the first 77 people registering today for OSCON Java with code JAVA7 will get the pass at a discounted price of $700 (applicable to OSCON Java package only).

Register now with code JAVA7


Clojure is a functional programming language based on Lisp. Through careful design, Clojure is simpler to read and use than Lisp, and it interacts cleanly with the Java world. Its functional nature makes programs very concise and composable.

In common with Scala, Clojure is designed with concurrency in mind. Its variables are immutable, and the use of software transactional memory and agents help manage shared mutable state in a more sustainable way than locking.

The experience of Clojure's inventor Rich Hickey is reason enough alone to give it a whirl. I once happily spent an hour listening to him describe the design and implementation of sequences in the language.

A "Lisp that could," Clojure is finding increasing (and surprising, to most observers) traction and acceptance. One contributing factor to this is an advanced build and package management infrastructure in leiningen and Clojars. Salesforce-owned hosted platform provider Heroku recently added Clojure as its third supported environment, following Ruby and Node/JavaScript.

A small but important omission from the Clojure ecosystem is a port of the book The Little Schemer.

(println "Hello, World!")

Clojure-related content at OSCON Java:

Tried and tested


A mashup of ideas from Python, Ruby and Smalltalk, Groovy gained early traction in the world of JVM languages, reaching its stable release in December 2007. Having been around a while, Groovy strongly acknowledges the Java world into which it was born, and is a good choice for Java developers wishing to use a more agile and dynamic language.

One of Groovy's jewels is Grails, a high-productivity web development environment inspired by Ruby on Rails. The Groovy world has also spawned Gradle, a modern project automation system for Java and JVM languages, providing an alternative to the established Ant and Maven projects.

println "Hello, World!"

Groovy-related content at OSCON Java:


Another veteran of the JVM language scene, Rhino is an implementation of the JavaScript programming language in Java. Part of the Mozilla project, it is typically used by developers to add user scriptability to their applications using JavaScript, though it is also used in situations where a system is predominantly implemented in JavaScript.

First released in 1999, Rhino has been around the block a few times. As a JavaScript engine, it is facing increasing competition from the C++-based V8, but retains the immense advantage of Java interoperability.

print('Hello, World!');

Remastered classics


A port of the Python language to the JVM, Jython offers some advantages over and above using Python, including Java's multi-threading and the ability to statically compile into Java classes. As a Python terse object-oriented language, programmers can quickly prototype in Jython, creating hybrid Java-Jython systems. Jython also offers developers an alternative to Rhino's JavaScript for embedding scriptability in their applications.

Jython also lets web developers bring popular Django web application framework into a Java setting.

print "Hello, World!"


The Java implementation of Ruby is no second-class citizen. JRuby has been recorded as better performing than Ruby 1.8, and strives for C-Ruby compatibility. Users of JRuby can benefit from using the Ruby on Rails web framework, or add Ruby scriptability to their applications.

One appealing aspect of building a system in JRuby or Jython is that it gives developers the option of reimplementing performance-critical code in Java without having to switch platforms as a project matures. Twitter's migration from Ruby to the JVM is a case in point.

puts "Hello, World!"

JRuby-related content at OSCON Java:

The Wild West

The JVM remains a fertile ground for language experimentation, as several of the languages featured in OSCON's Emerging Languages track demonstrate, including Seph and Gosu. As a platform, the JVM bootstraps language inventors and experimenters today much as lex and yacc did two decades ago. At OSCON Java, aspiring language creators should pop along to Charles Nutter's JVM Bytecode for Dummies.

There's still time left to register for OSCON Java. See you in Portland, 25-27 July!


July 01 2011

Radar's top stories: June 27-July 1, 2011

Here's a look at the top stories published on Radar this week.

Get started with Hadoop
Focusing on the Hadoop Distributed File System (HDFS) and MapReduce, this in-depth piece offers tips for organizations that are looking to evaluate Hadoop and deploy an initial cluster.
Clojure: Lisp meets Java, with a side of Erlang
Stuart Sierra digs into Clojure: what it is, how it works, and why it's attracting Java developers.
Two lessons from Pottermore: Direct sales and no DRM
It's not surprising that J.K. Rowing is forging ahead with a well thought-out direct sales plan for Harry Potter ebooks, but it's a shock that publishers aren't doing the same things for their titles.
What CouchDB can do for HTML5, web apps and mobile
Bradley Holt talks about what CouchDB offers web developers, how the database works with HTML5, and why CouchApps could catch on.
How Netflix handles all those devices
Matt McCarthy explains how WebKit and A/B testing play important roles on Netflix's many apps. Plus: Platform lessons Netflix has learned that apply to other developers and companies.

OSCON Java 2011, being held July 25-27 in Portland, Ore., is focused on open source technologies that make up the Java ecosystem. Save 20% on registration with the code OS11RAD

June 28 2011

Clojure: Lisp meets Java, with a side of Erlang

ClojureClojure seems to be the hot new language of the moment, attracting both Java developers and hackers who remember Lisp with fond memories. To see what all the buzz was about, I talked to Stuart Sierra (@stuartsierra), a Clojure developer and author of "Practical Clojure." Sierra will be speaking about Clojure in next month's OSCON Java track.

What is Clojure? How does it differ from functional languages like Erlang?

Stuart SierraStuart Sierra: Clojure is a new programming language that runs on the Java Virtual Machine. It's in the Lisp family of languages and it's a general purpose language, but it emphasizes a functional programming style with tools to manage shared state in large multi-threaded programs.

It's similar to Erlang with its emphasis on functional programming, but whereas Erlang is designed for systems distributed across many machines, Clojure focuses on many threads within a single machine. Erlang is trying to solve the very difficult problem of making everything parellelizable across multiple machines. Clojure is doing something different. It's trying to make sure that many threads in the same machine can safely share state.

How similar is Clojure to the syntax of traditional Lisp-like languages?

Stuart Sierra: Clojure is probably a little bit closer to Common Lisp than Scheme in style. It would not be particularly difficult for someone with Lisp experience to pick it up, although they would have to learn about some of the differences in the syntax. Clojure has a little bit more syntax than Common Lisp or Scheme, so it's actually a little easier to read and it has fewer parentheses than Lisp-like languages typically do. We've done examples where we put Java code and the equivalent Clojure code side-by-side and there are the same number or fewer parenthesis in the Clojure code because it's so much shorter.

OSCON Java 2011, being held July 25-27 in Portland, Ore., is focused on open source technologies that make up the Java ecosystem. (This event is co-located with OSCON.)

Save 20% on registration with the code OS11RAD

Some developers have a visceral negative reaction to Lisp. Why do you think that is?

Stuart Sierra: I think the problem people often have with Lisp is based on first encountering it in an academic setting, and the particular Lisp language they were using was designed for academic purposes. Scheme, in particular, came out of academia, and that focus has often made it more difficult to use for real-world applications. Clojure, on the other hand, is a very pragmatic language. It was designed by a career programmer, Rich Hickey, who wanted to solve real problems that he had with other programming languages. So if you can get over the initial similarities with Lisps that you might have seen before, I think you'll find that Clojure is a very productive language to work with.

Is Clojure being used in large-scale production?

Stuart Sierra: Yes, there are many "success stories" listed on the Clojure website and more on our wiki. I work for a company, Relevance, that does consulting for enterprise software, and we've had several client projects written in Clojure that have been quite successful. It's also been used in production by Akamai, financial companies, big data companies that are doing large data processing, and in a variety of other places.

The Java Virtual Machine (JVM) seems to be the language platform of choice for developing languages. What makes the JVM so attractive, and how does Clojure leverage it?

Stuart Sierra: The JVM makes an excellent platform for language development, and that's why we've seen so many new languages show up. It's actually much better as a target for languages than some lower-level assembly-type languages for the simple reason that it provides a way for all of those different languages to interoperate. So now you can have Clojure code, JRuby code, Scala code and Java code all making use of the same libraries and all calling back and forth to one another. So a Java method call in Clojure looks like a Clojure function call, but you're calling a Java method call by name. That compiles to a normal Java method call that has all of the performance benefits that the JVM can provide. You can define classes in Clojure that will be available to Java programs and accessible to Java programmers through the standard calling conventions that they're used to. All of Clojure's built-in data types implement the standard Java collection interfaces, so they can be passed directly into Java code without changes.

What's the IDE situation for Clojure?

Stuart Sierra: IDE support is evolving. There are plug-ins available now in varying states of sophistication for all of the major IDEs: Eclipse, IntelliJ, NetBeans. A lot of people are also using Emacs or another sophisticated text editor with good support for writing Lisp-like languages.


January 11 2011

Backtype: Using big data to make sense of social media

Strata Conference 2011 To prepare for O'Reilly's upcoming Strata Conference, we're talking with some of the leading innovators working with big data and analytics. Today, we talk with Backtype's lead engineer, Nathan Marz.

Backtype is an "intelligence platform," a suite of tools and insights that help companies quantify and understand the impact of their social media efforts. Marz works on the back end, figuring out ways to store and process terabytes of data from Twitter, Facebook, YouTube, and millions of blogs.

The platform runs on Hadoop, and makes use of Cascading, a Java API for creating complex workflows for processing data. Marz likes working with the Java-based tool for abstracting details of Hadoop because, "I find that when you're using a custom language you end up having a lot of complexity in your program that you don't anticipate, especially when you try to do things that are more dynamic."

Big data tools and applications will be examined at the Strata Conference (Feb. 1-3, 2011). Save 30% on registration with the code STR11RAD.

Marz has written an abstraction on top of Cascading called Cascalog, a Clojure-based query language for Hadoop inspired by Datalog. "The cool thing about Clojure is that it fully integrates with the Java programming language," Marz said. "I think one of the problems with Lisps in the past has been a lack of library support. But by being on top of the JVM, that problem is solved with Clojure." He's generally optimistic about what the functional and declarative paradigms can offer in the big data space, saying his programs are more concise and written closer to how he thinks.

Marz says he's happy with the development activity around Cascalog since he released it in April 2010 and is working on a few enhancements, including making it more expressive by adding optimized joins as well as making the query planner more intelligent by being more aggressive with, for example, push-down filtering.

You'll find the full interview in the following video:

December 17 2010

Strata Gems: Clojure is a language for data

We're publishing a new Strata Gem each day all the way through to December 24. Yesterday's Gem: Who needs disks anyway?.

Strata 2011

The Clojure programming language has been rising in popularity in
recent months. A Lisp-like language, it brings functional programming to the Java virtual machine (JVM)
platform. One of the distinctives of Clojure is that data is expressed in the same way as code, making
it ideal for writing powerful and concise domain-specific languages.

Clojure's inventor, Rich Hickey, has ensured that its integration with the world of Java is as painless as possible. And for those who fear Lisp-like languages, Clojure also bends a little to be friendlier. The result is that Clojure joins two worlds previously estranged: powerful functional programming with widespread and mature APIs.

In the world of big data, this means that Clojure can be used with Cascading, an API for programmatically creating Hadoop processing pipelines. Nathan Marz of Backtype used Clojure's power to create an entire query language for Hadoop, Cascalog.

Particular features of Clojure make it suitable for parallel data processing: immutable data types and built-in constructs for concurrency.

Cloud and big data go hand-in-hand. For working with the cloud, the jclouds project provides Java with a unified API to multiple cloud vendors, include Azure, Amazon and Rackspace. The jclouds API is often used with Clojure, exemplified by the Pallet project. Implemented in Clojure, Pallet automates the provisioning and control of cloud machine instances.

If you're looking to learn a new programming language and expand the way you think about coding, give Clojure a whirl. The excellent Java support means you won't be left isolated.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!