Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

June 01 2012

Developer Week in Review: The overhead of insecure infrastructure

I'm experiencing a slow death by pollen this week, which has prompted me to ponder some of the larger issues of life. In particular, I was struck by the news that an FPGA chip widely used in military applications has an easily exploitable back door.

There is open discussion at the moment about whether this was a deliberate attempt by a certain foreign government (*cough* China *cough*) to gain access to sensitive data and possibly engage in Stuxnet-like mischief, or just normal carelessness on the part of chip designers who left a debugging path open and available. Either way, there's a lot of hardware out there walking around with its fly down, so to speak.

As developers, we put a lot of time and effort into trying to block the acts of people with bad intent. At my day job, we have security "ninjas" on each team that take special training and devote a fair amount of their time to keeping up with the latest exploits and remediations. Web developers constantly have to guard against perils such as cross-site scripting and SQL injection hacks. Mobile developers need to make sure their remote endpoints are secure and provide appropriate authentication.

The thing is, we shouldn't have to. The underlying platforms and infrastructures we develop on top of should take care of all of this, and leave us free to innovate and create the next insanely great thing. The fact that we have to spend so much of our time building fences rather than erecting skyscrapers is a sign of how badly this basic need has been left unmet.

So why is the development biome so under protected? I think there are several factors. The first is fragmentation. It's easier to guard one big army base than 1,000 small ones. In the same way, the more languages, operating systems and packages that are in the wild, the more times you have to reinvent the wheel. Rather than focus on making a small number of them absolutely bulletproof (and applying constant vigilance to them), we jump on the flavor of the day, regardless of how much or little effort has been put into reducing the exposed security footprint of our new toy.

The fact that we have independent, massive efforts involved in securing the base operating systems for MacOS, Windows, Linux, BSD, etc, is nothing short of a crime against the development community. Pretty it up any way that suits you with a user interface, but there should (at this point in the lifecycle of operating systems) only be a single, rock-solid operating system that the whole world uses. It is only because of greed, pettiness, and bickering that we have multiple, fragile operating systems, all forgetting to lock their car before they go out to dinner.

Languages are a bit more complex, because there is a genuine need for different languages to match different styles of development and application needs. But, again, the language space is polluted with far too many "me-too" wannabes that distract from the goal of making the developer's security workload as low as possible. The next time you hear about a site that gets pwned by a buffer overrun exploit, don't think "stupid developers!", think "stupid industry!" Any language that allows a developer to leave themselves vulnerable to that kind of attack is a bad language, period!

The other major factor in why things are so bad is that we don't care, evidently. If developers refused to develop on operating systems or languages that didn't supply unattackable foundations, companies such as Apple and Microsoft (and communities such as the Linux kernel devs) would get the message in short order. Instead, we head out to conferences like WWDC eager for the latest bells and whistles, but nary a moment will be spent to think about how the security of the OS could be improved.

Personally, I'm tired of wasting time playing mall security guard, rather than Great Artist. In a world where we had made security a must-have in the infrastructure we build on, rather than in the code we develop, think of how much more amazing code could have been written. Instead, we spend endless time in code reviews, following best practices, and otherwise cleaning up after our security-challenged operating systems, languages and platform. Last weekend, we honored (at least in the U.S.) those who have given their life to physically secure our country. Maybe it's time to demand that those who secure our network and computing infrastructures do as good a job ...

OSCON 2012 — Join the world's open source pioneers, builders, and innovators July 16-20 in Portland, Oregon. Learn about open development, challenge your assumptions, and fire up your brain.

Save 20% on registration with the code RADAR20


Related:


April 16 2012

Four short links: 16 April 2012

  1. Peter Thiel's Class 4 Notes -- in perfect competition, marginal revenues equal marginal costs. So high margins for big companies suggest that two or more businesses might be combined: a core monopoly business (search, for Google), and then a bunch of other various efforts (robotic cars, TV, etc.). Cash builds up because it turns out that it doesn’t cost all that much to run the monopoly piece, and it doesn’t make sense to pump it into all the side projects. In a competitive world, you would have to be funding a lot more side projects to stay even. In a monopoly world, you should pour less into side projects, unless politics demand that the cash be spread around. Amazon currently needs to reinvest just 3% of its profits. It has to keep running to stay ahead, but it’s more easy jog than intense sprint. I liked the whole lecture, but this bit really stood out for me.
  2. Kickstarter Disrupting Consumer Electronics (Amanda Peyton) -- good point that most people wouldn't have thought that consumer electronics would lend itself to the same funding system as CDs of a one-act play about artisanal beadwork comic characters. Consumer electronics as a market has been ripe for disruption all along. That said, it’s ridiculously not obvious that disruption would come from the same place that allows an artist with a sharpie, a hotel room and a webcam a way to make the art she wants.
  3. OmniOS -- OmniTI's JEOS. Their team are engineers par excellence, so this promises to be good.
  4. Understanding Amazon's Ebook Strategy (Charlie Stross) -- By foolishly insisting on DRM, and then selling to Amazon on a wholesale basis, the publishers handed Amazon a monopoly on their customers—and thereby empowered a predatory monopsony. So very accurate.

September 20 2011

Four short links: 20 September 2011

  1. Plan 9 on Android -- replacing the Java stack on Android fans with Inferno. Inferno is the Plan 9 operating system originally from Bell Labs.
  2. SmartOS -- Joyent-created open source operating system built for virtualization. (via Nelson Minar)
  3. libtcod -- open source library for creating Rogue-like games. (via Nelson Minar)
  4. Wikipedia Miner -- toolkit for working with semantics in Wikipedia pages, e.g. find the connective topics that link two chosen topics. (via Alyona Medelyan)

July 14 2011

Why files need to die

Filing Cabinet by Robin Kearney, on FlickrFiles are an outdated concept. As we go about our daily lives, we don't open up a file for each of our friends or create folders full of detailed records about our shopping trips. Create, watch, socialize, share, and plan — these are the new verbs of the Internet age — not open, save, close and trash.

Clinging to outdated concepts stifles innovation. Consider the QWERTY keyboard. It was designed 133 years ago to slow down typists who were causing typewriter hammers to jam. The last typewriter factory in the world closed last month, and yet even the shiny new iPad 2 still uses the same layout. Creative alternatives like Dvorak and more recently Swype still struggle to compete with this deeply ingrained idea of how a keyboard should look.

Today we use computers for everything from booking travel to editing snapshots, and we accumulate many thousands of files. As a result, we've become digital librarians, devising naming schemes and folder systems just to cope with the mountains of digital "stuff" in our lives.

The file folder metaphor makes no sense in today's world. Gone are the smoky 1970s offices where secretaries bustled around fetching armfuls of paperwork for their bosses, archiving cardboard files in dusty cabinets. Our lives have gone digital and our data zips around the world in seconds as we buy goods online or chat with distant relatives.

A file is a snapshot of a moment in time. If I email you a document, I'm freezing it and making an identical copy. If either of us wants to change it, we have to keep our two separate versions in sync.

So it's no wonder that as we try and force this dated way of thinking onto today's digital landscape, we are virtually guaranteed the pains of lost data, version conflicts and failed uploads.

It's time for a new way to store data – a new mental model that reflects the way we use computers today.

OSCON Data 2011, being held July 25-27 in Portland, Ore., is a gathering for developers who are hands-on, doing the systems work and evolving architectures and tools to manage data. (This event is co-located with OSCON.)

Save 20% on registration with the code OS11RAD

Flogging a dead horse

Microsoft, Apple and Linux have all failed to provide ways to work with our data in an intuitive way. Many new products have emerged to try and ease our pain, such as Dropbox and Infovark, but they're limited by the tired model of files and folders.

The emergence of Web 2.0 offered new hope, with much brouhaha over folksonomies. The idea was to harness "people power" by getting us to tag pictures or websites with meaningful labels, removing the need for folders. But Flickr and Delicious, poster boys of the tagging revolution, have fallen from favor and as the tools have stagnated and enthusiasm for tagging has dwindled.

Clearly, human knowledge is needed for computers to make sense of our data – but relying on human effort to digitize that knowledge by labeling files or entering data can only take us so far. Even Wikipedia has vast gaps in its coverage.

Instead, we need computers to interpret and organize data for us automatically. This means they'll store not only our data, but also information about that data and what it means – metadata. We need them to really understand our digital information as something more than a set of text documents and binary streams. Only then will we be freed from our filing frustrations.

I am not a machine, don't make me think like one

In all our efforts to interact with computers, we're forced to think like a machine: What device should I access? What format is that file? What application should I launch to read it? But that's not how the brain works. We form associations between related things, and that's how we access our memories:

Associative recall in the brain

Wouldn't it be nice if we could navigate digital data in this way? Isn't it about time that computers learned to express the world in our terms, not theirs?

It might seem like a far-off dream, but it's achievable. To do this, computers will need to know what our data relates to. They can learn this by capturing information automatically and using it to annotate our data at the point it is first stored — saving us from tedious data entry and filing later.

For example, camera manufacturers have realized that adding GPS to cameras provides valuable metadata for each photograph. Back at your PC, your geo-tagged images will be automatically grouped by time and location with zero effort.

Our digital lives are full of signals and sensors that can be similarly harnessed:

  • ReQall uses your calendar and to-do list activity to help deliver information at the right time.
  • RescueTime tracks the websites and programs you use to understand your working habits.
  • Lifelogging projects like MyLifeBits go further still, recording audio and video of your life to provide a permanent record.
  • A research project at Ryerson University demonstrates the idea of context-aware computing — combining live, local data and user information to deliver highly relevant, customized content.

Semantics: Teaching computers to understand human language

Metadata annotation via sensors and semantic annotation

As this diagram shows, hardware and software sensors can only tell half the story. Where computers stand to learn the most is by analyzing the meanings behind the 1s and 0s. Once computers understand our language, our documents and correspondence are no longer just isolated files. They become source material, full of facts and ready to be harvested.

This is the science of semantics — programs that can extract meaning from the written word.

Here's some of what we can do with semantic technology today:

Today, most semantic research is done by enterprises that can afford to spend time and money on enterprise content management (ECM) and content analytics systems to make sense of their vast digital troves. But soon consumers will reap the benefits of semantic technology too, as these applications show:

  • While surfing the web, we can chat and interact around particular movies, books or activities using the browser plug-in GetGlue, which scans the text in the web pages you visit to identify recognized social objects.
  • We will soon have our own intelligent agents, the first of which is Siri, an iPhone app that can book movie tickets or make restaurant reservations without us having to fill in laborious online forms.

This ability for computers to understand our content is critical as we move toward file-less computing. A new era of information-based applications is beginning, but its success requires a world where information isn't fragmented across different files.

Time for a new view of data

Let's use your summer vacation as an example: All the digital information relating to your vacation is scattered across hundreds of files, emails and transactions, often locked into different applications, services and formats.

No matter how many fancy applications you have for "seamlessly syncing" of all these files, any talk of interoperability is meaningless until you have a basic fabric for viewing and interacting with your data at a higher level.

If not files, then what? The answer is surprisingly simple.

What is the one thing all your data has in common?

Time.

Almost all data can be thought of as a stream, changing over time:

The streams of my digital life

Already we generate vast streams of data as we go about our lives: credit card purchases, web history, photographs, file edits. We never get to see them on screen like that though. Combining these streams into a single timeline — a personal life stream — brings everything together in a way that makes sense:

A personal life stream


Asking the computer "Show me everything I was doing at 3 p.m. yesterday." or "Where
are Thursday's figures?" is something we can't easily do today. Products such as AllOfMe are beginning to experiment in this space.

We can go further — time itself can be used to help associate things. For example: Since I can only be in one place at one time, everything that happens there and then must be related:

All data at the same time is related

The computer can easily help me access the most relevant information — it just needs to track back along the streams to the last time I was at a certain place or with a specific person:

Related data can be found by finding previous occurrences on each stream

The world — our lives — is interconnected, and data needs to be the same.

This timeline-based view of data is useful, but it becomes even more powerful when combined with the annotations and semantic metadata gathered earlier. With this much cross-linking between data, our information can now be associated with everything it relates to, automatically.

Finally, we can do away with files because we have a system that works like the brain does – giving us another new power — to traverse effortlessly from one related concept or entity to another until we reach the desired information:

Associative data navigation

In a system like this we navigate based on what the data means to us – not which file it is located in.

There will be technical challenges in maintaining data that resides on different devices and is held by different service providers, but cloud computing industry giants like Amazon and Google have already solved much more difficult problems.

A world without files

In the world of linked data and semantically indexed information, saving or losing data is not something we'll have to worry about. The stream is saved. Think about it: You'd never have to organize your emails or project plans because everything would be there, as connected as the thoughts in your head. Collaborating and sharing would simply mean giving other people access to read from or contribute to part of your stream.

We already see a glimpse of this world when we look at Facebook. It's no wonder that it's so successful; it lets us deal with people, events, messages and photos — the real fabric of our everyday lives — not artificial constructs like files, folders and programs

Files are a relic of a bygone age. Often, we hang onto ideas long past their due date because it's what we've always done. But if we're willing to let go of the past, a fascinating world of true human-computer interaction and easy-to-find information awaits.

Moving beyond files to associative and stream-based models will have profound implications. Data will be traceable, creators will be able to retain control of their works, and copies will know they are copies. Piracy and copyright debates will be turned on their heads, as the focus shifts from copying to the real question of who can access what. Data traceability could also help counter the spread of viral rumors and inaccurate news reports.

Issues like anonymity, data security and personal privacy will require a radical rethink. But wouldn't it be empowering to control your own information and who can access it? There's no reason why big corporations should have control of our data. With the right general-purpose operating system that makes hosting a piece of data, recording its metadata and managing access to it as easy as sharing a photo on Facebook, we will all be empowered to embrace our digital futures like never before.

Photo: Filing Cabinet by Robin Kearney, on Flickr



Related:


January 05 2011

Developer Year in Review: Operating Systems

Our year in review concludes (slightly on the wrong side of the new year) with a look at what was up in operating systems. Rather than print a laundry list of who released what new version, let's take a look at some of the news that broke in 2010.

Linux: We're saved ... maybe?

As someone who has 10 shares of SCO framed and displayed in his bathroom, 2010 looked to be a very good year. The Beast from Utah finally exhausted all of its legal options, and cratered into a messy bankruptcy, leaving Novell with clear ownership of the Unix intellectual property that Linux may or may not incorporate. We all rejoiced, assuming that Linux would enjoy a happy existence in the future, unworried by fears of corporate protection rackets trying to intimidate people into paying for the free OS.

Then this fall, Novell announced that it was selling more than 800 of their patents to a consortium that includes Microsoft as a major player. Suddenly, all of the angst about IP attacks against Linux were back on the table, but now with known Linux-hater Microsoft appearing to hold the reins. Will further legal hijinks ensue? Only time will tell.

For Windows, second time's a charm

After the impressive (in the Hindenburg sense) launch of Windows Vista, Microsoft went back to the drawing board. They must have put a better grade of Kool-Aid in the water coolers the second time around, because Windows 7 has experienced a much warmer reception.

Vista never managed to crack the 20 percent adoption mark, even after four years on the market (it peaked just shy of 19 percent). By contrast, Windows 7 is already past the 20 percent mark, after only a little more than a year. XP, however, is still holding on to more than 50 percent of the Windows market. Not bad for a nine-year-old OS that isn't supported by Microsoft anymore.

MacOS gets a new distribution model

2010 brought point releases for Snow Leopard, but Mac-heads will have to wait until 2011 for the next major release, which we now know is called Lion.

Instead, the big news in 2010 was that Apple wants to do for desktop software what the iPhone App Store did for mobile. It remains to be seen if the new Mac App Store will be embraced by major software publishers. On one side of the equation, Apple is going to get a significant cut of the revenue from App Store sales, but on the other side, there's no need to create physical products to sell in retail stores. Add to that the fact that if a company won't sell their software in the App Store, a competitor might, and the App Store model has proven to be an effective way to sell software. Companies will ignore the App Store at the peril of their market share.

The other guys

Solaris: Reports suggest Oracle wants to bring Solaris back into a proprietary model, negating some or all of the work Sun did open sourcing it.

BSD: BSD adoption on desktops continues to be practically nonexistent, and even in the server market, it only accounts for 2.4 percent of servers. By contrast, Linux owns more than 60 percent of the server market, and even Windows has 15 times the installation base. It may be time for BSD to take a long hard look at itself, if it wants to avoid becoming irrelevant.

This concludes our year in review. Please return your seatbacks and tray tables to their full upright and locked positions. Next week, we'll get back to serving up the best of the week's news. Suggestions are always welcome, so please send tips or news here.



Related:




Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl