Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

December 25 2011

2011 showed us new ways of seeing

The sense of new images hitting our eyes from everywhere is one of the most remarkable aspects of 2011

Reasons to be cheerful in 2011? Let's see.

It was a year when eyes opened a bit wider, when images from Earth and space and the enigmatic microverse of quantum physics expanded our field of vision – and the spread of new means of communication made those images more accessible and shareable than ever before.

David Attenborough showed us what life is like under the surface of the Arctic's frozen sea. In a different way, images from Egypt and Libya showed us the previously hidden and denied passions of entire peoples. Meanwhile from Cern came visualisations of the elusive and mighty Higgs boson.

The silly slurs on Frozen Planet for filming polar bear cubs in a zoo drew attention to how extreme our visual information now is. This detail simply could not have been filmed in the wild, but so many marvels were captured that people were actually surprised by the so-called "faking". That's what the BBC gets for raising expectations. Its polar documentary was full of images such as wolves hunting, seen from above, that were never possible in the past. Similarly, pictures from the frontier of science, showing such wonders as Earth-like planets, go beyond previous astronomy and make outer space seem close.

This sense of new images hitting our eyes from everywhere is one of the most remarkable aspects of 2011. I know it has nothing to do with art as such. Yet art created one or two remarkable images of its own. In particular, Urs Fischer's melting candle sculptures moved me deeply and are themselves images of science – instances of entropy.

Uh-oh, entropy – the universe running down. The news in 2011 was sometimes exhilarating but mostly terrifying. In the media that circulate such news faster than ever before, the content was often disturbing. But the images that showed us the ever-changing world, and the ways they reached us, were eye-popping. So a reason to be cheerful is that new ways of seeing are being born in our time. © 2011 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

October 13 2011

Strata Week: Simplifying MapReduce through Java

Here are a few of the data stories that caught my attention this week:

Crunch looks to make MapReduce easier

Despite the growing popularity of MapReduce and other data technologies, there's still a steep learning curve associated with these tools. Some have even wondered if they're worth introducing to programming students.

All of this makes the introduction of Crunch particularly good news. Crunch is a new Java library from Cloudera that's aimed at simplifying the writing, testing, and running of MapReduce pipelines. In other words, developers won't need to write a lot of custom code or libraries, which as Cloudera data scientist Josh Willis points out, "is a serious drain on developer productivity."

He adds that:

Crunch shares a core philosophical belief with Google's FlumeJava: novelty is the enemy of adoption. For developers, learning a Java library requires much less up-front investment than learning a new programming language. Crunch provides full access to the power of Java for writing functions, managing pipeline execution, and dynamically constructing new pipelines, obviating the need to switch back and forth between a data flow language and a real programming language.

The Crunch library has been released under the Apache license, and the code can be downloaded here.

Web 2.0 Summit, being held October 17-19 in San Francisco, will examine "The Data Frame" — focusing on the impact of data in today's networked economy.

Save $300 on registration with the code RADAR

Querying the web with Datafiniti

DatafinitiDatafiniti launched this week into public beta, calling itself the "first search engine for data." That might just sound like a nifty startup slogan, but when you look at what Datafiniti queries and how it works, the engine begins to look profoundly ambitious and important.

Datafiniti enables its users to enter a search query (or make an API call) against the web. Or, that's the goal at least. As it stands, Datafiniti lets users make calls about location, products, news, real estate, and social identity. But that's a substantial number of datasets, using information that's publicly available on the web.

Although Datafiniti demands you enter SQL parameters, it tries to make the process of doing so fairly easy, with a guide that pops up beneath the search box to help you phrase things properly. That interface is just one of the indications that Datafiniti is making a move to help democratize big data search.

The company grew out of a previous startup named 80Legs. As Shion Deysarker, founder of Datafiniti told me, it was clear that the web-crawling services provided by 80Legs were really just being utilized to ask specific queries. Things like, what's the average listing price for a home in Houston? How many times has a brand name been mentioned on Twitter or Facebook over the last few months? And so on.

Deysarker frames Datafiniti in terms of data access, arguing that until now a few providers have controlled the data. The startup wants to help developers and companies overcome both access and expense issues associated with gathering, processing, curating and accessing datasets. It plans to offer both subscription-based and unit-based pricing.

Keep tabs on the Large Hadron Collider from your smartphone

LHSee screenshotNew apps don't often make it into my data news roundup, but it's hard to ignore this one: LHSee is an Android app from the University of Oxford that delivers data directly from the ATLAS experiment at CERN. The app lets you see data from collisions at the Large Hadron Collider.

The ATLAS experiment describes itself as an effort to learn about "the basic forces that have shaped our Universe since the beginning of time and that will determine its fate. Among the possible unknowns are the origin of mass, extra dimensions of space, unification of fundamental forces, and evidence for dark matter candidates in the Universe."

The LHSee app provides detailed information into how CERN and the Large Hadron Collider work. It also offers a "Hunt the Higgs Boson" game as well as opportunities to watch 3-D collisions streamed live from CERN. The app is available for free through the Android Market.

Got data news?

Feel free to email me.


September 29 2011

Strata Week: Facebook builds a new look for old data

Here are a few of the data stories that caught my attention this week.

Facebook data and the "story of your life"

Last week at its F8 developer conference, Facebook announced several important changes, including updates to its Open Graph that enable what it calls "frictionless sharing" as well as a more visual version of users' profiles — the "Timeline." As is always the case with a Facebook update, particularly one that involves a new UI, there have been a number of vocal responses. And not surprisingly, given Facebook's history, there have been a slew of questions raised about how the changes will impact users' privacy.

Facebook Timeline

Some of those concerns stem from the fact that now, with the Timeline, a person's entire (Facebook) history can be accessed and viewed far more easily. On stage at F8, CEO Mark Zuckerberg described the Timeline as a way to "curate the story of your life." But whether or not you view the new visual presentation of your Facebook profile with such grand, sweeping terms, it's clear that the new profile is a way for Facebook to re-present user data. Some of this data may be things that would have otherwise been forgotten — not just banal status updates, but progress in games, friendships made, relationships broken and so on. As Facebook describes it:

The way your profile works today, 99 percent of the stories you share vanish. The only way to find the posts that matter is to click 'Older Posts' at the bottom of the page. Again. And again. Imagine if there was an easy way to rediscover the things you shared, and collect all your best moments in a single place.

That new way was helped, in part, by Facebook's hiring earlier this year of data visualization experts Nicholas Felton and Ryan Case. But turning old Facebook data into new user profiles has caused some consternation, including the insistence by Silicon Filter's Frederic Lardinois that "sorry, Facebook, but the stuff I share on your site is not the story of my life."

But it wasn't an announcement from the stage at F8 that raised the most questions about Facebook data this week. Rather, it was a post by Nik Cubrilovic arguing that "logging out of Facebook is not enough." Cubrilovic discovered that even if you log out of Facebook, its cookies persisted. "With my browser logged out of Facebook," he wrote, "whenever I visit any page with a Facebook like button, or share button, or any other widget, the information, including my account ID, is still being sent to Facebook. The only solution to Facebook not knowing who you are is to delete all Facebook cookies."

Facebook responded to Cubrilovic and addressed the issue so that upon logout the cookie containing a user's ID is destroyed.

Web 2.0 Summit, being held October 17-19 in San Francisco, will examine "The Data Frame" — focusing on the impact of data in today's networked economy.

Save $300 on registration with the code RADAR

Faster than the speed of light?

Just as some tech pundits were busy debating whether the latest changes to Facebook had "changed everything," an observation by the physicists at CERN also prompted many to say the same thing — "this changes everything" — in terms of what we know about physics and relativity.

But not so fast. The scientists at the particle accelerator in Switzerland have been measuring how neutrinos change their properties as they travel. According to their measurements (some three years' worth of data and 15,000 calculations), the neutrinos appeared to have traveled from Geneva to Gran Sasso, Italy, faster than the speed of light. According to Einstein, that's not possible: nothing can exceed the speed of light.

CERN researchers released the data in hopes that other scientists can help shed some light on the findings. The scientists at Fermilab are among those who will pour over the information.

For those who need a little brushing up on their physics, an article at CNET has a good illustration of the experiment. High school physics teacher John Burk also has a great explanation of the discovery and the calculations behind it as well as insights into why the discovery and the discussions demonstrate good science (but in some cases, lousy journalism).

Data and baseball (and Brad Pitt)

MoneyballThe film "Moneyball," based on the 2003 bestseller by Michael Lewis, was released this past week. It's the story of Oakland Athletics general manager Billy Beane — played in the film by Brad Pitt — who helped revive the franchise by using data analysis to assemble his team.

Of course, stats and baseball have long gone hand in hand, but Beane argued that the on-base percentage (OPB) was a better indicator of a player's strengths than just batting average. By looking at the numbers that other teams weren't, Beane was able to assemble a team made up of players that other teams viewed as less valuable. (And by extension, of course, this helped Beane assemble that team at a much lower price.)

Thanks, in part, to Pitt's star power, a story about data making a difference is a Hollywood hit, and the movie's release has spurred others to ask if that sort of strategy can work elsewhere. In a post called "Moneyball for Startups," SplatF's Dan Frommer asked if it would be applicable to tech investing, a question that received a number of interesting follow-up discussions from around the web.

In a recent webcast, "Codermetrics" author Jonathan Alexander examined software teams through a "Moneyball" lens.

Got data news?

Feel free to email me.


June 13 2010

The Guardian's Science Weekly podcast: The man behind the Large Hadron Collider

We were honoured to have theoretical physicist Professor Sir Chris Llewellyn Smith in the studio with us.

Chris is a former director general of Cern and was instrumental in creating the Large Hadron Collider. He's now chair of the council of SESAME (Synchrotron-light for Experimental Science and Applications in the Middle East).

Chris updates us on progress in the field of nuclear fusion, and reveals some fascinating details about his time at Cern.

Our own science correspondent Ian Sample has written a book about the LHC's quest for the Higgs boson, Massive: The Hunt for the God Particle. He was happy to tell us all about it.

Producer Andy visited the new Skin exhibition at London's Wellcome Collection, where he got to wear a "social condom".

We also highlight an experiment in science journalism we are carrying out on our website. Story Tracker. It might just revolutionise the way we cover major science stories.

After reading nearly all of the documents at the centre of the University of East Anglia hacked emails furore, Guardian environment writer Fred Pearce wrote a book that is billed as the definitive account of the scandal. The Climate Files is out this week. Listen to James Randerson's interview with Fred in full in the latest Science Weekly Extra podcast.

Finally, next weekend is Science Hack Weekend: Get Excited and Make Things with Science! at the Guardian's offices in King's Cross, London. Bring your own bunsen burner. (Actually, don't.)

What is Science Hack Weekend? According to the organisers:

"A bunch of geeks get together in the same physical space to collaborate and create awesome things, usually by mashing up APIs. A Hack Day is usually 48 hours long and involves a sleepover ...although not much sleeping happens when everyone is either hacking or playing Werewolf."

Follow the podcast on our Science Weekly Twitter feed and receive updates on all breaking science news stories from Guardian Science.


Join our Facebook group.

Listen back through our archive.

Subscribe free via iTunes to ensure every episode gets delivered. (Here is the non-iTunes URL feed).

April 01 2010

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
Get rid of the ads (sfw)

Don't be the product, buy the product!