Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

June 29 2012

Four short links: 29 June 2012

  1. Personalization (Chris Lehmann) -- We should be careful about how we use that term, and we should be very skeptical of how well computerized programs can really personalize for kids. Most of what I see - especially from curriculum and assessment vendors - involves personalization of pace while still maintaining standardization of content. This.
  2. Unveiling Quadrigram (Near Future Laboratory) -- a Visual Programming Environment to gather, shape and share living data. By living data we mean data that are constantly changing and accumulating. They can come from social network, sensor feeds, human activity, surveys, or any kind of operation that produce digital information.
  3. Tim O'Reilly at MIT Media Lab (Ethan Zuckerman) -- a great recap of a Tim talk. There's an interesting discussion of the unmeasured value created by peer-to-peer activities (such as those made dead simple by the Internet), which is one of the new areas we're digging into here at O'Reilly.
  4. The State vs the Internet (David Eaves) -- we've all seen many ways in which the Internet is undermining the power of nation states. A session at Foo asked how it was going to end (which would give way first?), and this is an excellent recap. It could be that the corporation is actually the entity best positioned to adapt to the internet age. Small enough to leverage networks, big enough to generate a community that is actually loyal and engaged.

August 12 2011

Four short links: 12 August 2011

  1. Hippocampus Text Adventure -- written as an exercise in learning Python, you explore the hippocampus. It's simple, but I like the idea of educational text adventures. (Well, educational in that you learn about more than the axe-throwing behaviour of the cave-dwelling dwarf)
  2. Pandas -- BSD-licensed Python data analysis library.
  3. Building Lanyrd -- Simon Willison's talk (with slides) about the technology under Lanyrd and the challenges in building with and deploying it.
  4. Electronic Skin Monitors Heart, Brain, and Muscles (Discover Magazine blogs) -- this is freaking awesome proof-of-concept. Interview with the creator of a skin-mounted sensor, attached like a sticker, is flexible, inductively powered, and much more. This represents a major step forward in possibilities for personal data-gathering. (via Courtney Johnston)

January 21 2011

Four short links: 21 January 2011

  1. Proof-of-Concept Android Trojan Captures Spoken Credit-Card Numbers -- Soundminer sits in the background and waits for a call to be placed [...] the application listens out for the user entering credit card information or a PIN and silently records the information, performing the necessary analysis to turn it from a sound recording into a number. Very clever use of sensors for evil! (via Slashdot)
  2. Cloud9 IDE -- open source IDE for node.js. I'm using it as I learn node.js, and it's sweet as, bro.
  3. The Quantified Self Conference -- May 28-29 in Mountain View. (via Pete Warden)
  4. Bram Cohen Demos P2P Streaming -- the creator of BitTorrent is winding up to release a streaming protocol that is also P2P. (via Hacker News)

November 24 2010

Four short links: 24 November 2010

  1. What Android Is (Tim Bray) -- a good explanation of the different bits and their relationship.
  2. Cell Phone Photo Helped in Oil Spill (LA Times) -- a lone scientist working from a cell phone photo who saved the day by convincing the government that a cap it considered removing was actually working as designed. (via BoingBoing)
  3. Penki -- iPhone app that lets you paint 3D messages which are revealed in long-exposure photographs. (via Aaron Straup Cope on Delicious)
  4. I'm Working at Microsoft and We're Donating Imagery to OpenStreetMap! (Steve Coast) -- MSFT hired the creator of OSM and he says Microsoft is donating access to its global orthorectified aerial imagery to help OpenStreetMappers make the map even better than it already is.

October 18 2010

Seeing green from space

In addition to utilizing the global sensor network to access realtime, current, and future weather, we can also use these sources to project the effects of the weather across a wide spectrum of human activities. As such, satellite remote sensing has become an indispensable tool for researchers in the Earth observation community.

Remote sensing is just what the name implies: a suite of tools for accessing information about a subject without actually "touching" it. Remote sensing devices range from your own eyes to satellites in orbit hundreds of miles above the surface.

Many of these Earth-orbiting satellites are in continuous data acquisition and transmission mode, capturing everything from ocean temperatures, to land reflectance at the surface of the Earth, to global chlorophyll production. Each of these satellites is equipped with a variety of instruments, which collect very specific segments of information contained in the various bands of the electromagnetic spectrum. Users then, depending on their area of interest, will take the digital data and construct profiles of their study area, analyzing individual or composite band data and building time series profiles so that these databases can start to tell a story.

One of the most common multispectral analyses uses information derived in the near infrared and visible (red) spectral regions, called the Normalized Difference Vegetation Index, or NDVI, which can be viewed as a "greenness index." The higher the value on the scale, the more photosynthetically active the surface vegetation is, which can be used as an indicator of vegetation health. Whether the objective is to assess the health of crops in a specific growing region or across an entire continent, this index is a good indicator of how a crop region may be progressing, and where appropriate, crop failures can start to be identified. This year, the NDVI was used as an important proxy for agricultural health in India as the image below shows.

Within the graphic, the smaller image to the left shows the NDVI in July of 2009, and the smaller image on the right is the index one year later.

India is a country where agriculture and related industries make up a large part of domestic economic activity, and is therefore largely dependent on the health of the annual monsoon rains. This is not just for those directly involved in agriculture -- the nation's agrarian base consists of millions of independent farmers, who are the primary consumers of the goods purchased by the secondary industries, such as automobiles and motor scooters. So a poor monsoon not only means the potential for food shortages and less revenue for farmers, it also means less income to support other segments of India"s economy.

In 2009, the Monsoon was officially declared a "failure" (see image below left) as seasonal rainfall totals for the country came in 22 percent below normal. In the midst of a poor global macroeconomic picture, the lack of rains last year could not be repeated. This year has produced a much better monsoon (below right), and fortunately, Weather Trends clients were able to make longer range decisions with this forecast in mind. While 2010 is not a complete recovery, as India's north eastern states are still low, the pattern has been much more beneficial to the agricultural sector in the central and southern states.

Just receiving more rain does not necessarily mean economic recovery, so we look to the NDVI to measure the change. As we can see, the year-over-year images reflect the better ground conditions with the "greenness" across central and southern India indicating better crop potential.

The NASA Ocean Color Web is a treasure trove of research-grade data that can be used to analyze these environmental variables, and combinations of these data sources can lead to the construction of new indices that may be used an stand-alone analyses, or for incorporation into longer time series models.


Reposted byFreeminder23 Freeminder23

September 08 2010

Sensor networks and the future of forecasting

We can't control the climate, but are there ways to mitigate and avoid the negative effects extreme weather brings? I believe the starting point for potential solutions lies in data acquisition and environmental sensor networks (ESN).

Current technologies and sensors, ranging from cell phones to satellites, allow a "global environmental cyberinfrastructure" to be more than a topic for discussion at academic conferences. Researchers have studied connections and system interactions for some time, but now a broader segment of society is becoming aware of the precarious relationship between weather, climate and humanity. This awareness is sometimes motivated by the need to help. Other times there's a profit incentive. The reason doesn't matter if the result is a better low-cost global sensor network that can be tapped by anyone with a signal.

A systems approach to identifying natural hazards, coupled with a communications framework that can easily make data available to the public, is the crucial cornerstone of a functional environmental sensor network. The global monitoring of short- and long-range weather patterns and the linking of sensor-network data could allow forecasters to identify potential problems before they manifest.

The weather link between Russia and Pakistan

Weather has caused great disruption to many lives in both Russia and Pakistan in recent months. While these are separate circumstances, they share common physical factors. The following is a look at how events in one part of the world influence weather elsewhere.

The Russian heat wave

Global wheat prices spiked in early August. Much of that activity stemmed from potential crop losses in Russia, and it was helped by ubiquitous stories of parched fields and decimated crops. To be fair, part of the price spike came on the heels of a Russian export ban. Nonetheless, this story's origin is tied to weather.

The two maps below show the monthly year-over-year (2010 vs. 2009) changes in maximum temperature and precipitation for July. This weather was known to people in the agricultural and weather communities. The stage was set long ago for potential problems in western Russia.

(Click to enlarge)

(Click to enlarge)

A persistent high-pressure system centered over eastern Europe has combined with changes in the jet stream's pattern to keep temperatures high. When this high is as entrenched as it has been, it serves to do two things:

  1. It diverts a jet stream that would normally steer cooler air into parts of central Russia and northeastern Africa.
  2. It blocks moist air from the southeast, which exacerbates the dryness. This is a reason why parts of central Africa are seeing better moisture in recent months.

  3. There's an active west-to-east jet stream that travels above western Russia. This stream typically exhibits a seasonal shift to the east, and in the process allows moister air from the west to migrate into the region. The jet did not shift in July, and the result was a prolonged period of moisture-free air. When this combines with a strong high, the region experiences weather like it's seen over the last month. And when there is a high-pressure system in one region, there is often a corresponding low elsewhere.

    Flooding in Pakistan

    The low in this case has been over the mountainous region of northern Pakistan. This cold low has been the catalyst for a good portion of the excess rains. So, while located in distinct climate zones, the heat in Russia has a connection to the floods in Pakistan.

    But there's more to this puzzle. Every year, the annual Indian monsoon is anticipated throughout India and Pakistan, as much of the commercial activity that takes place in both countries is agrarian in nature. The monsoon was deficient in 2009, leading to short crops in many sectors. The arrival of the rainy season this year carried a heightened importance.

    The onset of the 2010 monsoon was healthy and most regions have been receiving beneficial moisture totals. But the placement of another area of high pressure over northeast India has, thus far, kept India's northern states dry. In the process, this high has been diverting even more moisture, which flows from southeast to northwest into central/western India and along into Pakistan.

    The first map below from NOAA's Earth System Research Laboratory depicts the storm tracks from the last week (as viewed via anomalies in outgoing longwave radiation), where the excess moisture is visible directly over Pakistan. The second map below, from the NOAA Climate Prediction Center, highlights wind anomalies over the last week. We see from this map that the stronger winds originating from the southeast were actively driving the moisture into areas that needed it the least.

    (Click to enlarge)

    (Click to enlarge)

    Pakistan's drainage infrastructure, which is silt-laden, has made the rainfall situation worse. When excess rains fall, it takes much longer to drain than necessary.

    Unfortunately, any excess rains in the coming weeks will likely be met with more problems for civilians. As of mid-August, estimates put flood-related deaths above 1,300, and more than 15 million people have been affected by the floods. Both of these figures are expected to rise.

    Satellites and risk management

    Could these separate-but-related crises have been foreseen? If so, what measures could have been initiated to mitigate some of the fallout?

    The map below is an indicator of vegetation health as derived via satellite for the wheat regions to the north of the Caspian and Black Seas. This particular graphic depicts the Normalized Difference Vegetation Index (NDVI) for the region, shown as an anomaly vs. the five-year average (for mid-to-late July). The index assigns a value for crop health, and based on the color scale shown in the legend, it is clear that the region has been exhibiting severe vegetation stress.

    The important thing to note here is that wheat prices started to increase in July, then exhibited a violent spike in early August. As this map is from mid-to-late July, we can see that by using tools such as satellite indices in conjunction with a long-range weather forecast, the current impact in wheat prices and ensuing financial turmoil could have been anticipated. To a certain degree, it could have been mitigated through a proactive physically-based risk management strategy. In addition, the same-satellite derived images that were capturing the movement of monsoon rains across India's agricultural regions could have been used to view the excess moisture in regions where the Pakistan floods originated.

    We can, of course, explore the questions surrounding how much lead time is necessary to avoid a crisis (remember Katrina). Nonetheless, it is clear that many were not aware of these disastrous systems until it was too late.

    (Click to enlarge)

    More coming soon

    This column is a starting point for discussions that examine climate, weather, sensors, networks, and their influence on society. The last couple of years have seen a heightened interest in this area from the research side, evidenced by new topics and sessions presented at the annual conferences of the American Geophysical Union, the American Meteorological Society, and others. I hope to bring some of these discussions to a broader audience while also helping readers understand how closely related their lives and decisions are to weather and climate.


July 05 2010

Four short links: 5 July 2010

  1. The Open Spending Data that Isn't (OKFN) -- the UK government mandated councils release details of expenditure over 500 pounds in size. Councils have been sending data to a proprietary service and claiming this is releasing it. Everyone needs to realise that government must always wholesale its data (offer bulk downloads), even when it doesn't retail that data (offer useful visualisation or analysis tools for it).
  2. SenseAware -- sensors for shipping that wirelessly report back where they are, whether there's light (i.e., has the container been opened), what the temperature is. (via data4all on Twitter)
  3. Open Science, Open Data, Open Methods (Ben Goldacre) -- open data is sometimes no use unless we also have open methods. (via OKFN)
  4. Sones -- cross-platform open source graph database built on Mono.

May 04 2010

Four short links: 4 May 2010

  1. Comparing genomes to computer operating systems in terms of the topology and evolution of their regulatory control networks (PNAS) -- paper comparing structure and evolution of software design (exemplified by the Linux operating system) against biological systems (in the form of the e. coli bacterium). They found software has a lot more "middle manager" functions (functions that are called and then in turn call) as opposed to biology, where "workers" predominate (genes that make something, but which don't trigger other genes). They also quantified how software and biology value different things (as measured what persists across generations of organisms, or versions of software): Reuse and persistence are negatively correlated in the E. coli regulatory network but positively correlated in the Linux call graph[...]. In other words, specialized nodes are more likely to be preserved in the regulatory network, but generic or reusable functions are persistent in the Linux call graph. (via Hacker News)
  2. Virtual Keyboards in Google Search -- rolling out virtual keyboards across all Google searches. Very nice solution to the problem of "how the heck do I enter that character on this keyboard?". (via glynmoody on Twitter)
  3. Information and Quantum Systems Lab at HP -- working on the mathematical and physical foundations for the technologies that will form a new information ecosystem, the Central Nervous System for the Earth (CeNSE), consisting of a trillion nanoscale sensors and actuators embedded in the environment and connected via an array of networks with computing systems, software and services to exchange their information among analysis engines, storage systems and end users. (via dcarli on Twitter)
  4. Turkit -- Java/JavaScript API for running iterative tasks on Mechanical Turk. (via chrismessina on Twitter)

March 19 2010

Four short links: 19 March 2010

  1. Tsung -- GPLed multi-protocol (HTTP, PostgreSQL, MySQL, WebDAV, SOAP, XMPP) load tester written in Erlang.
  2. Myth of China's Manufacturing Prowess -- The latest data shows [...] that the United States is still the largest manufacturer in the world. In 2008, U.S. manufacturing output was $1.8 trillion, compared to $1.4 trillion in China (UN data. China’s data do not separate manufacturing from mining and utilities. So the actual Chinese manufacturing number should be much smaller). Also contains pointers to an interesting discussion of lack of opportunities for college grads in China.
  3. OpenSSO and the Value of Open Source -- Oracle are removing all open source downloads and wiki mentions, leaving only the enterprise OpenSSO product on their web site. A Norwegian company has stepped in and will continue the open source project. This is essentially a fork, but for the forces of good. (via normnz on Twitter)
  4. The Internet of Things -- 5m video on sensor networks, etc. (via imran on Twitter)

January 12 2010

Four short links: 12 January 2010

  1. Tldr -- an application for navigating through large-scale online discussions. The application visualizes structures and patterns within ongoing conversations to let the user browse to content of most interest. In addition to visual overviews, it also incorporates features such as thread summarization, non-linear navigation, multi-dimensional filtering, and various other features that improve the experience of participating in large-discussions.
  2. City Senses -- Chris Heathcote points out that Pachube can be used as a sensor for the world, not just for energy. His pilot projects are good, and I'd like to see more. The biggest change I'm predicting for the next 10 years is the rise of sensor networks: smart stuff, aware environments, and loquacious locations.
  3. Aggressive Patent Trolls -- 1/3 of Union Square Ventures startups are threatened by patent trolls. (via Hacker News)
  4. Anopticon -- Italian project to document the location of CCTV cameras, many of which were set up illegally. The site is in Italian, but there's some information on this article in The Register. Information is power, and this is a clever way for citizens to regain power in the privacy war. (via Nick Clark)

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!