Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

August 10 2011

FCC contest stimulates development of apps to help keep ISPs honest

Last Friday, the Federal Communications Commission (FCC) announced the winners of its open Internet Challenge.

"The winners of this contest will help ensure continued certainty, innovation and investment" in the broadband sector," said FCC chairman Julius Genachowski at the awards ceremony. "Shining a light on network management practices will ensure that incentives for entrepreneurs and innovators remain strong. They will help deter improper conduct helping ensure that consumers and the marketplace pick winners and losers online, and that websites or applications aren't improperly blocked or slowed."

The contest received twenty four submissions in total, with three winners. MobiPerf, a mobile network measurement tool that runs on Android, iOS, and Windows Mobile devices, won both the People's Choice Award and best overall Open Internet App. MobiPerf collects anonymous network measurement information directly from mobile phones. MobiPerf was designed by a University of Michigan and Microsoft Research team.


Two apps and teams shared the Open Internet Research Award. ShaperProbe, which was originally called, "DiffProbe," is designed to detect service discrimination by Internet service providers (ISPs). ShaperProbe uses the Measurement Lab (M-Lab) research platform. All of the data collected through ShaperProbe will be publicly accessible, according to Georgia Institute of Technology, which developed the app.

Netalyzer is a Web-based Java app that measures and debugs a network. Notably, the Netalyzer Internet traffic analysis tool has a "Mom Mode," which may make it more accessible to people like, well, my own mother. Netalyzer was built by the International Computer Science Institute (ISCI) at the University of California at Berkeley.

More details about the winners and the teams that built them is available at

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science -- from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

20% on registration with the code STN11RAD

Open Internet questions

It almost goes without saying that this contest carried some baggage at the outset. Last December, the launch of a new contest by the Federal Communications Commission was overshadowed by concerns about what the new FCC open Internet rules could mean for net neutrality, particularly with respect to the mobile space that is of critical interest to many developers. Nonetheless, the FCC open Internet challenge went forward, focused on stimulating the development of apps for network quality of service testing.

Amidst legitmate concerns about the sustainability of apps contests, the outcomes of this Open Internet challenge offers a couple of important data points.

First, the challenge does seem to have stimulated the creation of a new resource for the online community: unlike the other two winners, the MobiPerf app was created for the contest, according to FCC press secretary Neil Grace.

Second, when this challenge launched, collecting more data for better net neutrality was a goal that organizations like the Electronic Frontier Foundation and M-Lab supported. The best answers to questions about filtering or shaping rely "on the public having real knowledge about how our Internet connections are functioning and whether or not ISPs are providing the open Internet that users want," wrote Richard Esguerra.

Now the public has better tools to gather and share that knowledge. Will these apps "shed light" on broadband providers' tactics? As with so many apps, that will depend on whether people *use* them or not. The two winning apps that existed before the contest, Netalyzer and ShaperProbe, have already been used thousands of times, so there's reason to expect more usage. For instance, Netalyzer can be (and was) applied in analyzing widespread search hijacking in the United States. In that context, empowered consumers that can detect and share data about the behavior of their Internet service providers could play a more important role in the broadband services market.

Finally, the FCC has established new ties to the research and development communities at Berkeley, Georgia Tech and other institutions. It connected with the community. Integrating more technical expertise from academia with the regulator's institutional knowledge is an important outcome from the challenge, and not one that is as easily measured as "a new app for that." It's not clear yet whether the outcomes from the Apps for Communities challenge, set to conclude on August 31st, will be as positive.

The expertise and the data collected from these apps might come also in handy if the time ever comes when the regulator has to make a controversial decision about whether a given ISP's service to its users goes beyond "reasonable network management."

Reposted bykrekk krekk

December 29 2010

What lies ahead: Net Neutrality

Tim O'Reilly recently offered his thoughts and predictions for a variety of topics we cover regularly on Radar. I'll be posting highlights from our conversation throughout the week. -- Mac

Is mobile creating a new digital divide?

Tim O'ReillyTim O'Reilly: Many people are fretting that limited access to smartphones is creating a new digital divide. I think that’s a misplaced worry because all phones will be smartphones before long. That problem will take care of itself.

If we assume that all phones are smartphones, what happens at that point? First off, we’ll have major capacity problems because a lot more data will go over the airwaves. That's why some of the FCC's efforts to free up spectrum are so critical. We can’t keep using spectrum inefficiently and hope to have enough.

There will be spectrum congestion and various problems related to that in the near term, but eventually it will get sorted out. The telecoms will need to make investments, and application developers will have to get smarter about how their apps use data. Apps that are bad network citizens are going to stand out.

What will happen with mobile and net neutrality?

Tim O'Reilly: I used to be in the religious net-neutrality camp, but the realities of capacity mean quality-of-service prioritization has to happen. To be clear, I'm still strongly against discrimination that targets a particular company or application.

I see the idea of "absolute" net neutrality going away at some point. Eric Schmidt made an important point at Web 2.0 Summit: There are two concepts of net neutrality. One is you can’t discriminate against any particular company or any particular application. But on the other hand you can discriminate against classes of applications. You could prioritize video lower than voice, or a bulk download of data lower than something that requires real-time communication. Prioritization will be contentious, but capacity limitations will make it clear why it's necessary.

Note: Video of Eric Schmidt at Web 2.0 Summit is posted below:

Next in this series: What lies ahead in DIY and Make
(Coming Dec. 30)


December 21 2010

Steve Wozniak on the FCC and Internet freedom

Earlier today, Apple co-founder Steve Wozniak published a passionate open letter to the FCC that described his personal history with the telecommunications industry.

Wozniak followed that up with a surprise appearance at the Federal Communication Commission's public hearing on new open Internet rules and net neutrality. Steven Levy of Wired Magazine tweeted about the unexpected arrival: "Woz is at FCC hearing to speak against the plan--sez that with these rules, he couldn't have done Apple."

Interviewed by various media outlets after the hearing (see video below), Wozniak explained his presence at the hearing:

I wanted to be here because this day was so significant to my life. I had a ham radio license when I was 10 years old. I had the FCC spectrums on my wall in my room. I grew up admiring the FCC ... The FCC has always sort of had a white hat. This is a case where that hat could go black.

Wozniak was not happy with much of what he heard from the commission at today's hearing:

I don't think the rules went far enough in protecting individuals, but I tend to be very much on the side of the small guy being taken advantage of by the big guy. I feel sorry about that. I feel emotional about that.

Specifically, Wozniak expressed concerns over blocking issues:

... no blockages doesn't mean there will be no inhibitions. It wasn't clear what was presented here today to me if that means you can't really favor one source over another. You know, an innovator comes along, they shouldn't have any blocks on the Internet. To me, the Internet, the ISPs, should just be providing things like the copper to your house and the gear that puts it onto the Internet. Step back, get out of the way, don't try to make it go your way.

Wozniak also noted he had been personally affected by nearly every issue FCC commissioner Michael Copps raised in his statement at the hearing.

December 13 2010

Four short links: 13 December 2010

  1. European mobile operators say big sites need to pay for users' data demands (Guardian) -- it's like the postal service demanding that envelope makers pay them because they're not making enough money just selling stamps. What idiocy.
  2. Grace Programming Language -- language designers working on a new teaching language.
  3. Gawker Media's Entire Database Hacked -- 1.5M usernames and passwords, plus content from their databases, in a torrent. What's your plan to minimize the harm of an event like this, and to recover? (via Andy Baio)
  4. Macmillan Do Interesting Stuff (Cameron Neylon) -- have acquired some companies that provide software tools to support scientists, and are starting a new line of business around it. I like it because it's a much closer alignment of scientists' interests with profit motive than, say, journals. Timo Hannay, who heads it, runs Science Foo Camp with Google and O'Reilly.

December 06 2010

Four short links: 6 December 2010

  1. Apple I Basic as Mac OS X Scripting Language -- great hack. The “apple1basic” executable is a statically recompiled version of the original binary. All code is running natively. It plugs right into UNIX stdin and stdout. You can pass it the filename of a BASIC program to run. You can run BASIC programs like shell scripts. (via Hacker News)
  2. How to Discredit Net Neutrality -- the Level3-Comcast dispute isn't as straightforward as you might think (or as I implied). Increasingly, advocates of net neutrality have pegged their case to a larger and more powerful role for FCC regulation in the internet industry. And thus the net neutrality debate, instead of focusing on developing new institutional arrangements to preserve internet freedom on BOTH the demand and supply side, descends into a replay of the early 1980s, Reagan-era punch and judy show between democrats and republicans, with one arguing for "more government" and the other for "less government." Neither talking much sense about what the government should actually do. There's a missing discussion here about competition preventing carrier abuses, competition that the US lacks.
  3. The Dark Side of Open Source Conferences (Val Aurora) -- A good first step is for conferences and communities to adopt and enforce explicit policies or codes of conduct that spell out what kind of behavior won't be tolerated and what response it will get. Much in the way that people don't stop speeding unless they get speeding tickets, or that murder is totally unacceptable to most people but laws against it still exist, harassment at conferences may seem obviously wrong, but stopping it will require written rules and enforceable penalties.
  4. iDev Blog-a-Day -- love the layout and the content's good too.

November 30 2010

Four short links: 30 November 2010

  1. libgit2 -- a linkable git library. Ruby and Python bindings.
  2. Open Data: How Not to Cock It Up -- Tom Steinberg lays it out.
  3. Algorithm and Crowd are Not Enough -- My point isn’t that Google, Netflix, Amazon, Yelp or any of the others are doomed. But I do think there’s an opportunity brewing for entrepreneurs, websites and companies to add editorial components to the algo-crowd paradigm. O'Reilly's business is built on editorial value, whether in book selection or conference creation. We obviously see a continued role for editorial presence. (via John Battelle on Twitter)
  4. Level 3 vs Comcast (Denver Post) -- first shakedown from the carriers. Without mandated neutral carriers, the Internet will dissolve into a fiefdom of consolidated big players willing to pay the shakedowns of the telco goons.

September 13 2010

The distinctions and controversies of net neutrality

Network neutrality draws the kind of zealous attention that activists
like me have been hoping for years we'd get for Internet policy. I
don't think any cyber-issue has ever excited as much interest among
the general public. Nor, unfortunately, has any been described in so
many confusing and inconsistent ways.

My latest policy foray has been a small attempt to fix some of that.
I'm inviting everyone with an interest in network neutrality to view
and edit a wiki at the O'Reilly Commons site: Network Neutrality: Distinctions and Controversies.

Check the table of contents to get a sense of why polemics over
"network neutrality" get so tangled up. The term can mean at least
half a dozen different things, and there is just as much variety in
the practices that it's supposed to stop. So far as I know, this wiki
is the first disciplined attempt to distinguish the various threads.

I started the wiki because I think we need it. Just over the past few
weeks we've been treated to news coverage of href="">a
joint proposal from Google and Verizon, which I found href="">muddled
in ways that show why we need a finer understanding of the many topics
involved. The FCC has released a href="">request
for comments that shows they're trying to hone in on the
distinctions. And a recent href="">article
where I made an initial stab at dissecting the arguments was well
received and summarized in href="">Forbes

The next step is up to you. The wiki just digs down one or two levels,
and can benefit from lots of alternative viewpoints. It should also
have more references. The href="">license
on the wiki is pretty permissive about copying and reuse, and I'll
be happy to donate my contributions to any other worthy project. I ask
you to help me moderate it to cut down on the all-too-common
rhetorical tricks that muddy the issues instead of delineating them. I
know that many philosophical positions and intriguing proposals are
hard to fit within the existing format of the wiki, and I think those
positions and proposals need to be outlined elsewhere. Let's try to
make the wiki rich but brief.


September 02 2010 poised for an overdue overhaul

Gov 2.0 Summit, 2010For an agency that is charged with regulating communications, the Federal Communication Commission (FCC) has been a bit behind the curve in evolving its online presence. was launched in June 1995, redesigned in 1999, and relaunched again in September 2001. Since then, it has remained a largely static repository for public notices and information about the agency's action.

According to FCC officials, that's going to change, and soon. There was already some insight offered into redesigning the FCC website back in January on the agency blog, informed at least in part by discussions with Sunlight Labs on redesigning the government.

Yesterday, I interviewed FCC managing director ‪Steven VanRoekel at FCC headquarters about what rebooting‬ will mean for the agency, businesses and the American people. "The new site will embrace open government principles around communication and participation," said VanRoekel. "Consider, where over 30,000 ideas were generated, or Comments there go into the official record and are uploaded to the Library of Congress. You will see that in a much more pervasive way in the new"

Our short video interview is below. An extended interview follows.

Redesigning FCC websites for public comment

In January, the FCC launched and asked for public input on improving citizen interaction. The site, which was touted as the first website to solicit citizen interaction with the FCC, followed the launch of and last year. All three websites are notable for their clean design and integration of new media components (blogs, Twitter, etc). Chairman Julius Genachowski's introduction to the site is embedded below:

Improving public access to the FCC's operation is part of a new mentality, according to VanRoekel: "Last year, the chairman talked about entrepreneurs taking a rotation through government. We think a lot about bringing in great leadership and managing people around leadership. We were third from the bottom last year in rankings for the best places to work in federal government. We hired a new team to bring in a new culture, which means looking at citizens as shareholders."

One of the stated aims of was to gather feedback on how itself can be redesigned, a project that, as noted above, is long overdue. The announcement of the new site, for instance, showed up in email but was not posted in plain text on Like other releases, it showed up as a Word doc and PDF on the site. That said, the FCC has picked up the pace of its communications over the past year, as anyone who has followed the @FCC on Twitter knows.

Aside from the cleaner design of the new microsites and an embrace of social media, open government geeks and advocates took note of, which is meant to be "an online clearinghouse for the Commission's public data." The FCC has posted XML feeds and search tools for its documents that allow users to sort data by type and bureau.

Under the Media Bureau, for instance, visitors can explore DTV Station Coverage Maps, a key issue to many given the transition to digital TV earlier this year. But the maps are on the old For those who don't enjoy good public DTV reception, they'd have to find the tiny icon for below the fold and click through to get more information.

FCC's clunky clickstream

Navigation on is still a work in progress. In this example, a user who clicks a link for "DTV Station Coverage Maps" is taken to the old FCC site. From there, they need to find and click on a DTV icon to receive deeper information.

That kind of reciprocal citizen-to government interaction is precisely where the potential for these sites can be best realized, and where good design matters. So-called Web 1.0 tools like static websites, email and SMS used to share information about the quality of services. Web 2.0 services like blog comments and social media have, in turn, been deployed to gather feedback from citizens about the delivery of said services. The FCC began to pursue that potential in earnest in March, when the FCC went mobile and launched iPhone and Android apps for crowdsourced broadband speed testing.

The potential for empowering citizens and developers with open data s where VanRoekel focused first when we talked.

"We'll be announcing a couple of things next week at the Gov 2.0 Summit," he said. "Since we launched the speed test, we've gathered over a million data points. That continues to grow each day. We're going to launch a web services API where people can write apps against the speed test data. You'll be able to send us a GPS coordinate and we'll give you data against it."

FCC Chairman Julius Genachowski and Managing Director Steven VanRoekel will discuss their experiences turning into a 21st-century consumer resource at the Gov 2.0 Summit in Washington, D.C. (Sept. 7-8). Request an invitation.

If incorporated into or the thousands of online real estate brokerages, that kind of interaction has the potential to give people what they need to make more informed rental or buying decisions. "When I click on a house on a real estate site, why don't I see what broadband capabilities are there?" asked VanRoekel. "We're approaching .gov like .com. We're not only setting up data services and wrapping the API, but we're building apps as well, and utilizing the same APIs we expect developers to use."

A consistent challenge across government for releasing open data has been validation and accuracy. The FCC may employ crowdsourcing to address the quality issue. "Think about a map of broadband speeds," VanRoekel explained. "I would love the ability for users to show us what's valid."

Balancing transparency and open government

As a regulatory body, the FCC has both great power and great responsibility, to put it in terms that Stan Lee might appreciate. Despite the arcane nature of telecommunications law, the agency's decisions have the potential to affect every citizen in the nation. As VanRoekel pointed out, the FCC must follow administrative procedures and publish drafts of rulemaking for public comment, followed by a vote by the commissioners. In the age of the Internet and the open government directive, that process is due for the same reboot the will receive.

"Once approved, language in the APA [Administrative Procedure Act] says government will open up the notice of draft rules to enlighten public decision-making," said VanRoekel. "In the past, what that's meant is us putting it up on a website, in PDFs. Law firms would send clerks, who would photocopy folders and come back with comments at the draft rule. There was no way for an educator or an affected family to get involved. It's our vision that every rule that's up for decision in this agency will be opened for public input."

The first draft of that effort has been on display at "We made it so that an idea entered into our engines was entered into the public record," VanRoekel said. "An interesting fact there is that you, as a citizen or industry body, can see the comments and hold us legally liable."

The FCC is faced with difficulties that derive from handling the explosion of online feedback that contentious issues like net neutrality generate. "The volume of comments becomes our problem," he said. "When you have 30,000 ideas coming in and comments on top of them on the record, and we have a limited number of people that oversee the effort, that's our biggest challenge."

While the FCC has touted new tools for openness and transparency, it's also taken a beating about a lack of transparency in close- door meetings on Internet rules.

"There's a role to play on certain meetings where ex parte comes into play," said VanRoekel. "We tend to use ex parte as a mechanism for understanding. We ask vendors specific questions. Many times there are questions that involve their intellectual property."

The agency has since ended closed-door meetings, but the episode highlights the complexity of enacting new regulations in the current media climate.

Yesterday, in fact, as the New York Times reported, the FCC agency released a public notice seeking more input on open Internet rules, which the agency duly tweeted out as a PDF. The document is embedded below.

Ars Technica and others criticized the agency for asking more questions instead of taking action.

Will the FCC get net neutrality right?. Hard to say. The Center for Democracy and Technology, by way of contrast, endorsed the FCC focusing in on key issues in the net neutrality debate "as a good sign that the FCC is rolling up its sleeves to grapple with the most contentious issues." As my colleague Andy Oram pointed out this week, the net neutrality debate depends upon what you fear. The only safe bet here is that is likely to get a fresh batch of public comment to take into the record.

Addressing the digital divide

Online debates over net neutrality or the proposed broadband plan leave out a key constituency: the citizens who do not have access to the Internet. The information needs of communities in a democracy were the focus of the recent report by the Knight Commission.

To that point, VanRoekel spoke with the Sunlight Foundation's executive director, Ellen Miller, earlier this year about how everyone is changing everything. Their conversation is embedded below:

In our interview, VanRoekel focused on how mechanisms of community activation can be used to include disconnected people.

VanRoekel pointed to the growth of mobile access and social media uptake in communities that have been traditionally less connected. That's a focus that is substantiated by Pew Research that shows citizens turning to the Internet for government data, policy and services, particularly minority communities.

Traditional outreach is still viable as well. Community organizers can reach people on the ground and involve key constituencies. "We also can go back to 800 numbers," he said. "Using voice to offer access and adding the ability to enter into the public record."


August 30 2010

The network neutrality debate: It all depends on what you fear

The debate over network neutrality, which saw a spike in news reporting two weeks ago over the joint Google/Verizon proposal, confuses a lot of laypeople -- and not just laypeople -- because of all the different levels on which it's being argued and the opposing ways language is used by different participants. I discuss the use of loaded language such as "censorship" and "innovation" in an article on my website, which is also being published today in the American Reporter.

In that article, I suggest three levels into which one can roughly
divide the arguments made on each side. Competition is the
basis for the controversy, but both sides like to extend it to issues
of censorship. Most subtle, and most important, are questions
of creativity, usually aired as concerns about "innovation"
or a recently invented term, "generativity."

Each of these concepts appears different to those debating them
because they also apply at multiple levels of networking. For
instance, some people are most concerned with competition, investment,
and innovation at the application level (Google, Yahoo!, and so on)
while others worry more about the lower levels of networking
infrastructure: who will supply fiber and routers to make the Internet
work? I end the article with a bit about motivations and sources for content on the Internet.

You can find the full piece here.


August 12 2010

Four short links: 12 August 2010

  1. A Review of Verizon and Google's Net Neutrality Proposal (EFF) -- a mixture of good and bad, is the verdict. I am ready to give Google credit for getting Network Neutrality back on the regulatory agenda, whether or not this proposal was a strawman.
  2. Ten Principles for Opening Up Government Information (Sunlight Foundation) -- We have updated and expanded upon the Sebastopol list and identified ten principles that provide a lens to evaluate the extent to which government data is open and accessible to the public. The list is not exhaustive, and each principle exists along a continuum of openness. The principles are completeness, primacy, timeliness, ease of physical and electronic access, machine readability, non-discrimination, use of commonly owned standards, licensing, permanence and usage costs.
  3. What If the Web Really Worked for Science? Reimagining Data Policy and Intellectual Property (video) -- a talk by James Boyle on IP and science policy.
  4. Winners of the Apps for Army Challenge -- more Android apps than iPhone in the winners. (via Alex)

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
Get rid of the ads (sfw)

Don't be the product, buy the product!