Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

April 03 2013

Four short links: 3 April 2013

  1. Capn Proto — open source faster protocol buffers (binary data interchange format and RPC system).
  2. Saddle — a high performance data manipulation library for Scala.
  3. Vegaa visualization grammar, a declarative format for creating, saving and sharing visualization designs. (via Flowing Data)
  4. dumpmon — Twitter bot that monitors paste sites for password dumps and other sensitive information. Source on github, see the announcement for more.

October 17 2012

Tools for test-driven development in Scala

Scala, a language designed for well-structured and readable programs, is richly provisioned with testing frameworks. The community has adopted test-driven development (TDD) and behavior-driven development (BDD) with zeal. These represent the baseline for trustworthy code development today.

TDD and BDD expand beyond the traditional model of incorporating a test phase into the development process. Most programmers know that ad hoc debugging is not sufficient and that they need to run tests on isolated functions (unit testing) to make sure that a change doesn’t break anything (regression testing). But testing libraries available for Scala, in supporting TDD and BDD, encourage developers to write tests before they even write the code being tested.

Tests can be expressed in human-readable text reminiscent of natural language (although you can’t stretch the comparison too far) so that you are documenting what you want your code to do while expressing the test that ensures that code ultimately will meet your requirements.

Daniel Hinojosa, author of Testing in Scala, describes the frameworks and their use for testing, TDD, and BDD in this interview.

Highlights from our discussion include:

  • The special advantages of test frameworks for Scala. [Discussed at the 0:10 mark]
  • The two main testing frameworks, ScalaTest and Specs2. It’s worth studying both of these frameworks, but you’ll probably ultimately stick to one based on programming style and how you want to do mocking. [Discussed at the 2:12 mark]
  • Mocking simply means removing operations that will take a long time or require outside support, such as a database. When testing, you want to fool your code into believing that the operation took place while actually simulating it. This is especially critical for TDD, because tests are so extensive and run so regularly. [Discussed at the 04:01 mark]
  • How the new ScalaMock library extends the abilities to mock parts of the system. This is an emerging technology. [Discussed at the 7:36 mark]
  • Generating random input test data. You can actually make your code more robust by throwing garbage values at it rather than by planning what data to input, because a programmer usually fails to anticipate some of the data that will be encountered in production use. For instance, you might not realize how large the input data will be, or might forget to include negative numbers. Scala gives you a full range of control ranging from specifying precise values to allowing completely random input. [Discussed at the 8:24 mark]
  • Looking toward the future of Scala testing. [Discussed at the 10:38 mark]

You can view the entire conversation in the following video:

Related:

June 14 2012

Why use Scala

Scala is not strictly a functional programming language — it was designed from the ground up to be an object-oriented and functional hybrid. So programmers need to choose which methodology to use, but both are available. I recently talked with Alex Payne (@al3x), co-author of the book "Programming Scala," about the advantages of using Scala.

Highlights from the full video interview include:

  • Why should you learn and use Scala? [Discussed at the 0:22 mark]
  • Advantages of combining object-oriented and functional capabilities in a programming language. [Discussed at the 1:24 mark]
  • Back end service development. [Discussed at the 2:56 mark]
  • Could Scala replace Java? [Discussed at the 4:45 mark]
  • Toolkit in use at Simple. [Discussed at the 6:00 mark]
  • How to build a better software development environment. [Discussed at the 7:25 mark]

You can view the entire conversation in the following video:

OSCON 2012 — Join the world's open source pioneers, builders, and innovators July 16-20 in Portland, Oregon. Learn about open development, challenge your assumptions, and fire up your brain.

Save 20% on registration with the code RADAR


Related:


May 03 2012

Editorial Radar: Functional languages

Functional Languages are driving a broader set of choices for programmers. O'Reilly editors Mike Loukides and Mike Hendrickson sat down recently to talk about the advantages of functional programming languages and how functional language techniques can be deployed with almost any language. (The full conversation is embedded below.)

Andy Hunt and Dave Thomas have long recommend learning a new language each year, especially those languages that teach new concepts [discussed at the 02:02 mark]. Functional languages have made that easier. They behave in a different way than the languages many of us grew up on — procedural like C or languages derived from C. Plus, the polyglot programming movement has driven the interest in functional languages as one of the languages you might want to learn.

Programmers need to understanding the advantages of using a functional language, such as productivity, power of expressiveness, reliability, stateful objects, concurrency, natural concurrency, modularity, and composability [05:37]. Though a search still exists for a magic bullet [06:29] to make it easier for programers to better solve the problem of concurrency. CPU speeds have been stuck at roughly the same level for the last four to five years. Programmers have been given is more transistors on a chip, hence more CPUs and more cores to work with making concurrency one of the most difficult issues facing computer scientists today. Enter functional programming with improved debugging and the ability to write more reliable code in a concurrent environment.

Additional highlights from this conversation include:

  • Print book sales of functional languages are growing, especially books on R programming. And while Loukides doesn't consider R to be a functional language, some debate exists about its classification. Though it's clear the data science movement has driven the use of R because it's well designed for statistics and dealing with data. [Discussed at the 00:29 mark]
  • We'll see F# grow in the Microsoft development environment while Scala and Clojure are dominating the open source space. Erlang will also be around for a long time for building highly reliable concurrent systems. [Discussed at the 03:01 mark]
  • Since the publication of Doug Crockford's JavaScript: The Good Parts, coders have discovered the functional language abilities of JavaScript and Java. Google's release of Maps and Gmail revolutionized how JavaScript is used. Some of today's best examples include Node for high-performance websites and D3 for creating exotic and beautiful data visualizations. [Discussed at the 08:15 mark]
  • While JavaScript isn't a functional language, it's designed loosely, so it's easy to use as a functional language. You might also be interested in how functional programming techniques can be used in C++ — a blog post written by John Carmack. [Discussed at the 10:36 mark]
  • Java isn't intended as a functional language. Though Dean Wampler's Functional Programming for Java Developers provides an approachable introduction to functional programming for anyone using an object-oriented language. [Discussed at the 11:41 mark]
  • The use of a functional language or functional language techniques can make your code more robust and easier to debug. [Discussed at the 12:09 mark]

You can view the entire conversation in the following video:

Tune in next month for a discussion of NoSQL and web databases.

Fluent Conference: JavaScript & Beyond — Explore the changing worlds of JavaScript & HTML5 at the O'Reilly Fluent Conference (May 29 - 31 in San Francisco, Calif.).

Save 20% on registration with the code RADAR20

Related:

December 26 2011

Four short links: 26 December 2011

  1. Pattern -- a BSD-licensed bundle of Python tools for data retrieval, text analysis, and data visualization. If you were going to get started with accessible data (Twitter, Google), the fundamentals of analysis (entity extraction, clustering), and some basic visualizations of graph relationships, you could do a lot worse than to start here.
  2. Factorie (Google Code) -- Apache-licensed Scala library for a probabilistic modeling technique successfully applied to [...] named entity recognition, entity resolution, relation extraction, parsing, schema matching, ontology alignment, latent-variable generative models, including latent Dirichlet allocation. The state-of-the-art big data analysis tools are increasingly open source, presumably because the value lies in their application not in their existence. This is good news for everyone with a new application.
  3. Playtomic -- analytics as a service for gaming companies to learn what players actually do in their games. There aren't many fields untouched by analytics.
  4. Write or Die -- iPad app for writers where, if you don't keep writing, it begins to delete what you wrote earlier. Good for production to deadlines; reflective editing and deep thought not included.

July 12 2011

Four short links: 12 July 2011

  1. Slopegraphs -- a nifty Tufte visualization which conveys rank, value, and delta over time. Includes pointers to how to make them, and guidelines for when and how they work. (via Avi Bryant)
  2. Ask Me Anything: A Technical Lead on the Google+ Team -- lots of juicy details about technology and dev process. A couple nifty tricks we do: we use the HTML5 History API to maintain pretty-looking URLs even though it's an AJAX app (falling back on hash-fragments for older browsers); and we often render our Closure templates server-side so the page renders before any JavaScript is loaded, then the JavaScript finds the right DOM nodes and hooks up event handlers, etc. to make it responsive (as a result, if you're on a slow connection and you click on stuff really fast, you may notice a lag before it does anything, but luckily most people don't run into this in practice). (via Nahum Wild)
  3. scalang (github) -- a Scala wrapper that makes it easy to interface with Erlang, so you can use two hipster-compliant built-to-scale technologies in the same project. (via Justin Sheehy)
  4. Madlib -- an open-source library for scalable in-database analytics. It provides data-parallel implementations of mathematical, statistical and machine learning methods for structured and unstructured data. (via Mike Loukides)

July 07 2011

A rough guide to JVM languages

O'Reilly is celebrating the release of Java 7, and our inaugural OSCON Java conference: July 25-27 in Portland, Ore.

The possibility of using alternative languages on the JVM has always been an appealing side story to Java. Jython and JRuby were early pioneers in implementing a dynamic language on top of a statically typed VM. Now that Java 7 directly includes support in the JVM for dynamic languages, the coexistence is official.

Pick any of the languages here and you can expect the support of the robust JVM threading and garbage collection, plus access to a broad array of application libraries.

Up and coming

Scala

The pragmatist among JVM languages, Scala is a general-purpose programming language. It was conceived and is generally viewed as a "better Java," and of all alternative JVM languages has the best acceptance in an enterprise setting.

Scala combines the familiar feel of object-oriented Java with strong language support for concurrency, XML and functional programming features: many of the tools that contemporary complex and scalable systems require. Scala also takes a lot of the awkwardness out of Java code, through features such as type inference and traits.

object HelloWorld {
    def main(args: Array[String]) {
      println("Hello, World!")
    }
  }

Scala-related content at OSCON Java:

OSCON Java 2011
To celebrate the release of Java 7, the first 77 people registering today for OSCON Java with code JAVA7 will get the pass at a discounted price of $700 (applicable to OSCON Java package only).

Register now with code JAVA7

Clojure

Clojure is a functional programming language based on Lisp. Through careful design, Clojure is simpler to read and use than Lisp, and it interacts cleanly with the Java world. Its functional nature makes programs very concise and composable.

In common with Scala, Clojure is designed with concurrency in mind. Its variables are immutable, and the use of software transactional memory and agents help manage shared mutable state in a more sustainable way than locking.

The experience of Clojure's inventor Rich Hickey is reason enough alone to give it a whirl. I once happily spent an hour listening to him describe the design and implementation of sequences in the language.

A "Lisp that could," Clojure is finding increasing (and surprising, to most observers) traction and acceptance. One contributing factor to this is an advanced build and package management infrastructure in leiningen and Clojars. Salesforce-owned hosted platform provider Heroku recently added Clojure as its third supported environment, following Ruby and Node/JavaScript.

A small but important omission from the Clojure ecosystem is a port of the book The Little Schemer.

(println "Hello, World!")

Clojure-related content at OSCON Java:



Tried and tested


Groovy

A mashup of ideas from Python, Ruby and Smalltalk, Groovy gained early traction in the world of JVM languages, reaching its stable release in December 2007. Having been around a while, Groovy strongly acknowledges the Java world into which it was born, and is a good choice for Java developers wishing to use a more agile and dynamic language.

One of Groovy's jewels is Grails, a high-productivity web development environment inspired by Ruby on Rails. The Groovy world has also spawned Gradle, a modern project automation system for Java and JVM languages, providing an alternative to the established Ant and Maven projects.

println "Hello, World!"

Groovy-related content at OSCON Java:

Rhino

Another veteran of the JVM language scene, Rhino is an implementation of the JavaScript programming language in Java. Part of the Mozilla project, it is typically used by developers to add user scriptability to their applications using JavaScript, though it is also used in situations where a system is predominantly implemented in JavaScript.

First released in 1999, Rhino has been around the block a few times. As a JavaScript engine, it is facing increasing competition from the C++-based V8, but retains the immense advantage of Java interoperability.

print('Hello, World!');

Remastered classics

Jython

A port of the Python language to the JVM, Jython offers some advantages over and above using Python, including Java's multi-threading and the ability to statically compile into Java classes. As a Python terse object-oriented language, programmers can quickly prototype in Jython, creating hybrid Java-Jython systems. Jython also offers developers an alternative to Rhino's JavaScript for embedding scriptability in their applications.

Jython also lets web developers bring popular Django web application framework into a Java setting.

print "Hello, World!"

JRuby

The Java implementation of Ruby is no second-class citizen. JRuby has been recorded as better performing than Ruby 1.8, and strives for C-Ruby compatibility. Users of JRuby can benefit from using the Ruby on Rails web framework, or add Ruby scriptability to their applications.

One appealing aspect of building a system in JRuby or Jython is that it gives developers the option of reimplementing performance-critical code in Java without having to switch platforms as a project matures. Twitter's migration from Ruby to the JVM is a case in point.

puts "Hello, World!"

JRuby-related content at OSCON Java:

The Wild West

The JVM remains a fertile ground for language experimentation, as several of the languages featured in OSCON's Emerging Languages track demonstrate, including Seph and Gosu. As a platform, the JVM bootstraps language inventors and experimenters today much as lex and yacc did two decades ago. At OSCON Java, aspiring language creators should pop along to Charles Nutter's JVM Bytecode for Dummies.


There's still time left to register for OSCON Java. See you in Portland, 25-27 July!



Related:


June 22 2011

Four short links: 22 June 2011

  1. DOM Snitch -- an experimental Chrome extension that enables developers and testers to identify insecure practices commonly found in client-side code. See also the introductory post. (via Hacker News)
  2. Spark -- Hadoop-alike in Scala. Spark was initially developed for two applications where keeping data in memory helps: iterative algorithms, which are common in machine learning, and interactive data mining. In both cases, Spark can outperform Hadoop by 30x. However, you can use Spark's convenient API to for general data processing too. (via Hilary Mason)
  3. Bagel -- an implementation of the Pregel graph processing framework on Spark. (via Oliver Grisel)
  4. Week 315 (Matt Webb) -- read this entire post. It will make you smarter. The company’s decisions aren’t actually the shareholders’ decisions. A company has a culture which is not the simple sum of the opinions of the people in it. A CEO can never be said to perform an action in the way that a human body can be said to perform an action, like picking an apple. A company is a weird, complex thing, and rather than attempt (uselessly) to reduce it to people within it, it makes more sense - to me - to approach it as an alien being and attempt to understand its biology and momentums only with reference to itself. Having done that, we can then use metaphors to attempt to explain its behaviour: we can say that it follows profit, or it takes an innovative step, or that it is middle-aged, or that it treats the environment badly, or that it takes risks. None of these statements is literally true, but they can be useful to have in mind when attempting to negotiate with these bizarre, massive creatures. If anyone wonders why I link heavily to BERG's work, it's because they have some incredibly thoughtful and creative people who are focused and productive, and it's Webb's laser-like genius that makes it possible. They're doing a lot of subtle new things and it's a delight and privilege to watch them grow and reflect.

January 18 2011

Four short links: 18 January 2011

  1. How Facebook Ships Code -- all engineers go through 4 to 6 week "Boot Camp" training where they learn the Facebook system by fixing bugs and listening to lectures given by more senior/tenured engineers. estimate 10% of each boot camp's trainee class don't make it and are counseled out of the organization. Reminded me of Zappos paying people to leave. (via Hacker News)
  2. EU Funds Scala -- it's a research project at a university, and just got a big pile of funding from the EU.
  3. Biotic Games -- they make Pong, Pacman, Pinball, etc. from biotech. (via Andy Baio)
  4. Asleep and Awake (BERG London) -- It’s glowing rectangles all the way down: those backlit screens that suck your attention. Matt J described it nicely a few years ago: the iPhone is a beautiful, seductive but jealous mistress that craves your attention, and enslaves you to its jaw-dropping gorgeousness at the expense of the world around you. Reminded me of Jesse Robbins's great line, "mobile is the opposite of mindful".

December 22 2010

Developer Year in Review: Programming Languages

Continuing our look at the year in development, let's move on to the exciting land of languages. We'll finish off next week with operating systems.

Java: Strategic asset or red-headed stepchild?

Watching Oracle's machinations around Java can be more than a little confusing. One minute, they're talking about forking it into free and commercial versions, a potential slap in the face to the open source community. Then they refused to let Apache's Harmony project have access to key testing suites to certify the Java alternative. But then Oracle ended the year on their hands and knees begging Apache to stay in the JCP (and failing).

Meanwhile, we saw yet another "that's not really Java" lawsuit. This time Oracle was suing Google over the Android implementation. Evidently, having Bill Gates and Steve Ballmer as dire enemies wasn't good enough for Larry Ellison, so he's trying to add Sergey Brin and Larry Page to his list as well.

On a side note, has anyone noticed how Java basically took over the mobile space? Of the three major smartphone platforms (sorry Windows, you have a ways to go before you make that list again ...), two of them run Java of some sort. If you add in J2ME, which is inside many of the "clamshell" phones, Java is the dominant player in mobile.

It was also a good year for the JVM, as JVM-powered languages such as Closure, Groovy and Scala leveraged the omnipresence of Java to gain traction.

I see your 8 cores, and raise you 8

Functional programming considers to gain in popularity in the years ahead, mainly as programmers try to come to terms with how to leverage all the multi-threaded power available to them in modern hardware. Along with the aforementioned Scala, Erlang and Haskell have also seen commercial deployments increase.

Francesco Cesarini gave a great talk at OSCON on how Erlang can help developers. Unfortunately, there was no transcript, because it had no side effects. (Trust me, the functional programmers in the readership are falling over laughing.)

In other language news ...

Perl: Perl 6 still lags "Duke Nukem Forever" as far as being promised software still awaiting final shipment, but only by three years.

PHP: With Salesforce.com adding PHP to their language arsenal, you can now run PHP on all the major cloud-based platforms (the others being Amazon, Windows and Google.)

Ruby: No new major version of Ruby this year, nor any earth-shattering news, but it continues to be the language that all the cool kids use.

Python: Release 3.2 is on track for a Q1 2011 release. "Python" is also a lousy word to put into a Google News search, unless you enjoy reading about people smuggling snakes through customs and DPW workers making unexpected discoveries in sewers.

That's it for this week. I'll take a look at the year in operating systems in the next edition. Suggestions are always welcome, so please send tips or news here.



Related:




Four short links: 22 December 2010

  1. ietherpad -- continuation of the etherpad startup. Offers pro accounts, and promise an iPad app to come. (via Steve O'Grady on Twitter)
  2. Scala Collections Quickref -- quick reference card for the Scala collections classes. (via Ian Kallen on Twitter)
  3. Raw Data and the Rise of Little Brother -- Turns out, despite the great push for citizen journalism, citizens are not, on average, great at “journalism.” But they are excellent conduits for raw material — those documents, videos, or photos.
  4. Theseus 1.0 -- impressive source maze builder in Ruby contributed to the public domain. (via Hacker News)

May 25 2010

Four short links: 25 May 2010

  1. Lending Merry-Go-Round -- these guys have been Australia's sharpest satire for years, filling the role of the Daily Show. Here they ask some strong questions about the state of Europe's economies ... (via jdub on Twitter)
  2. What's Powering the Guardian's Content API -- Scala and Solr/Lucene on EC2 is the short answer. The long answer reveals the details of their setup, including some of their indexing tricks that means Solr can index all their content in just an hour. (via Simon Willison)
  3. What I Learned About Engineering from the Panama Canal (Pete Warden) -- I consider myself a cheerful pessimist. I've been through enough that I know how steep the odds of success are, but I've made a choice that even a hopeless fight in a good cause is worthwhile. What a lovely attitude!
  4. Mapping the Evolution of Scientific Fields (PLoSone) -- clever use of data. We build an idea network consisting of American Physical Society Physics and Astronomy Classification Scheme (PACS) numbers as nodes representing scientific concepts. Two PACS numbers are linked if there exist publications that reference them simultaneously. We locate scientific fields using a community finding algorithm, and describe the time evolution of these fields over the course of 1985-2006. The communities we identify map to known scientific fields, and their age depends on their size and activity. We expect our approach to quantifying the evolution of ideas to be relevant for making predictions about the future of science and thus help to guide its development.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl