Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

December 21 2010

Can standardization and innovation coexist in education?

"How could you personalize your music if your Pandora data was lost every time you logged in?"

The question came up during a great conversation with Matthew Rascoff of Wireless Generation about personalizing the educational experience. He was making the point that information about students is lost every time they change schools, making it impossible to use data for personalizing learning the way we personalize our music, video, and reading experiences using Pandora, TiVo, and Kindle.

Personalization is the holy grail of education technology, but it can't be achieved without mechanisms for rich data about each student's learning. And that data must be persistently stored and appropriately accessible. Matthew neatly turns the traditional metaphor of a "digital locker" on its head by replacing it with the "data backpack" -- a container that goes everywhere the student goes.

Matthew's insights led me to Wireless Generation's white paper on "An American Examination System." The paper outlines a platform for using technology to collect and store data on individual student performance. With this rich data following students, teachers will have the data available to know what students need to work on or learn next. The data can become the basis for "adaptive mass personalization."

What I particularly like about this vision is that it is designed with nicely partitioned layers: beginning with the technology and protocols for data collection and sharing, moving to a format for encoding a hypothesis about an optimal set of learning trajectories, moving to an automated and rich layer of high cognitive demand tests, and supported by grading mechanisms that are automated where possible, and draw on networks of graders where appropriate.

The idea is that by using technology, standardized tests can be graded by any teacher anywhere, not just a given student's classroom teacher. With this sort of expert crowd-sourcing, the community of teachers can become much more consistent in their expectations, and minimize the variation in grades that comes from the subjective nature of evaluating creative work.

Because the layers of functionality are segregated, this vision represents a meaningful platform approach to what I consider to be the absolute core issues for reinventing education: testing and assessment.

Standardization is what makes a platform powerful. Railroads are a platform for the transportation of goods and people because all tracks are the same size and all trains can ride on any track. This means there's an the opportunity for innovation regarding different kinds and models of trains, speed and efficiency improvements, business models for transporting goods, etc. Over time the train tracks and systems have come to support another layer of standardization -- containers -- that allow goods to be efficiently transported across railways, trucks, and ships. If that second layer of standardization had happened at the same time as the first, the resulting standards would have been far less efficient for the mass movement of goods than those that exist today. If business model competition didn't happen, the economic environment that made container standardization so powerful might never have developed.

When technology products seek to become platforms, they often look to lock up the value chain from top to bottom by standardizing every layer according to their own product's protocols. They "open" the top API layer to entice others to choose their platform. I've often heard this informally referred to as the "build it and they will come" strategy by those who are skeptical of it, and a "vertical integration" strategy by those who are fans.

In industry this plays out in the debate of "open" vs. "closed." Is the iPhone closed because of the many layers of the platform Apple controls, or open because of the breadth of innovators who can try their products on top of Apple's API's? Android embraces the idea that an architecture standardized at a lower layer will breed more innovation and eventually lead to global domination over the siloed Apple stack. The question of which layer to standardize in a cell phone platform is an open one and the economic implications of choosing the right layer are vast.

In the area of education technology, this question of where to standardize is even more critical. Whereas in industry, competing platforms will eventually answer the question of which approach generates the most innovation and success, in education, early decisions about platforms will become institutionalized and almost impossible to change for decades. This is a critical point to consider as the US Department of Education, governors, and chief state school officers all are investing heavily in a vision of the transformation of schooling through technology.

If standardization happens too far down the stack (say, below the level of student data flow protocols), the market will continue to be fragmented and there will be significant barriers for education innovations to reach students at scale. Innovators will essentially be selling school to school and district by district. If standardization happens too far up the stack, areas that are still immature in the field will become cast in concrete and innovation in those areas will be unlikely.

With a platform structure for education, the key question becomes what to standardize in order to enable wide adoption of innovations, and what things must be deliberately and specifically not standardized in order to allow new innovations to flourish.

Using assessment platforms as an example, the Wireless Generation White Paper highlights a specific theory for students' learning trajectories in math, yet at the same time provides a structure they refer to as a "honeycomb" for specifying that theory. If the field were mature in its understanding of learning trajectories, the suggested trajectories could be a useful place to standardize. But given that competing learning trajectories are still emerging, the honeycomb structure instead provides a well-defined place for critical innovation.

Wireless Generation honeycomb
An example of the honeycomb structure discussed in "An American Examination System" (PDF).

Platforms are valuable precisely because of what they do and do not standardize. Widespread standardization is difficult to accomplish, but critical to realizing the potential of technology in reinventing education. Wireless Generation has an intriguing framework for such a platform (which is probably why it was just acquired by News Corp. for $360 million). With the resources of News Corp. joined with the innovative platform approaches of Wireless Generation, standardization at the right layers is a real and compelling possibility. It will be fascinating to watch this marriage unfold.


September 28 2010

Education as a platform

Any and every education reform design is going to fail for two reasons. The first is that the problem is not one that is solvable by “design” in the traditional engineering sense -- the education system, including all its human elements, is too complex for that. The second is that the system as currently built contains feedback loops that damp out change.

At the Gov 2.0 Summit, Deputy Director for Policy for the White House Office of Science and Technology Policy,Thomas Kalil, referred to the challenge of educational software that improves the more students use it.

What would it mean to talk about a whole school system that improves the more students use it? I’ve heard the Department of Education's Steve Midgely refer to school as a service and education as a platform -- why not apply this kind of systems thinking to the Gordian knot of our education system, using the Internet as a lens and a platform model?

Applying the Web 2.0 model to education

The Web 2.0 model can be applied to education on two levels. The first is at the level of software that provides or supports educational experiences. The second is at the level of the human systems and workflows that are entailed by the day-to-day work of educators and students and citizens.

As a veteran software organization leader and management geek, I see deep parallels between the architecture of software platforms and that of human organizations. For example, in both cases, the question of quality has moved from “does it work as designed” (which is now considered table-stakes) to “is it designed to do the right thing?”

A software program that works exactly as intended but isn’t useful to its end users is a waste, as is a team that is executing to the letter of a Statement of Work at the cost of the real needs of its multiple and diverse stakeholders. So is an educational system that mass-produces graduates who are prepared for an agrarian/industrial economy when they will have to live and work in an economy powered by knowledge, collaboration and creativity.

One possible criterion for education as a platform is whether the underlying services can be mashed up to serve different sets of goals. As Clayton Christensen, Curtis Johnson, and Michael Horn outline in "Disrupting Class," our educational system was originally created to prepare the elite to rule, then to prepare all people (or at least white men) for informed citizenship, then to provide all students with an equal opportunity to prepare for college or careers, and now to prepare every child for higher education. These changing goals have strained the current school architecture to the point where any number of responsibilities that schools have traditionally considered important, such as vocational training and the arts, are now being dropped in favor of “the basics”. There are many who argue credibly that the goals of school ought to go beyond the basics to include modern work-force preparation (learning how to learn, collaborate, invent, communicate, etc.) and global citizenship.

By this criterion, the inability of our education system to accommodate these multiple and shifting goals suggests that it does not have the architecture of a platform (at least not a well-designed one). "Disrupting Class" suggests that, much as the web disrupted many traditional business models, technology in the form of online learning will disrupt school. The argument is that computer-aided education and online learning will be welcomed into schools as a low-cost way to provide services they otherwise couldn’t afford. These new offerings may be paid for by parents or schools and may give students school credit. Although the technologies will initially be very poor substitutes for classroom teaching, they will be much better than nothing and the schools will welcome them as they lose the resources to provide these services for students in traditional ways. Over time the technologies will continue to improve until they revolutionize what teaching and learning look like.

This disruptive mechanism has intriguing implications for education as a platform. First, critically, given the assumption that technologies are going to improve, there is an implicit feedback loop driven by criteria for what “better” looks like and that allows publishers and providers of educational services to continually adapt and improve -- more on that later.

Second, there is an implicit design approach that eschews traditional top-down, requirements-based engineering and assumes that the complex solutions that will someday serve education begin with very simple systems that then evolve in the market, much in the way the Internet evolved from the simple TCP/IP stack rather than the complex, traditionally designed OSI stack.

Third, I infer that innovation demands multiple, varied providers of computer-aided and online learning, which in turn calls for a layer of standardized interfaces, API’s, and delivery mechanisms that make it easy for great education innovations to spread virally in the way popular Internet innovations do.

Defining "better"

teacher's eyeThese implications for the content and software infrastructure of Education 2.0 translate in interesting ways to the parallel pedagogical and human infrastructures. First, and critically, in order to establish positive feedback loops for the work of students and educators there needs to be a set of criteria for what “better” looks like. Metrics in this area are the absolute crux of education reform and poor metrics are a major contributor to how innovations are damped out in the education system -- again, more on that later.

Second, my own experience in leading large software organizations suggests that traditional hierarchical, silo’d human systems are inefficient and resistant to change, but that they improve dramatically when restructured to support self-organizing teams. Self-organizing teams are the organizational equivalent of “simple systems” -- they evolve to support complex work with extraordinary capacity and adaptability. One key to self-organizing teams is to set goals (or in the language of emergence, "simple rules") that drive positive feedback loops -- such as “be the school everyone wants to transfer to”. Another is to remove constraints from the systems and allow for greater autonomy and innovation on the part of each member of the team. In industry, a classic example is to reward people on the basis of outcomes rather than seat time (great code doesn’t care if it was written in a suit from 9-5 or in pajamas at midnight.) What if students could advance based on outcomes rather than seat time in the classroom, and teachers were rewarded for many dimensions of student achievement and professional contribution?

Third, I am of the opinion that the distinction between formal learning (school) and informal learning (museums, Internet, community classes, affinity groups, etc.) is one that is both artificial and obsolete. In Education 2.0 there should be multiple providers of educational experiences, and standard discovery mechanisms that allow great experiences to spread virally as well as standard ways to give students credit for what they know and can do rather than for what classes they’ve sat through.

Student achievement beyond a test

There is broad agreement that testing in our educational system is fundamentally broken. Today, our educational feedback loops are driven by “the test” -- state-wide standardized tests in the basics that are used to assess whether every student is able to perform to a certain lowest common denominator. Collectively, the results of these tests are used to grade schools and determine whether they will continue to receive funding. Indirectly, attempts to reward teachers for performance are often based, in part or whole, on student performance on these tests.

The unintended consequence of using student achievement on “the test” to define what “better” looks like is that the feedback loops in the educational system will damp out work and innovations that aren’t measured at the lowest common denominator. In some cases, teachers will spend up to half the school year in “test preparation”, which doesn't leave time for any learning above and beyond the basics. There are horror stories of mathematically rich curricula developed at great expense and with deep thoughtfulness under National Science Foundation funding that aren’t adopted because much of the mathematical depth that students gain from it is not reflected by “the test”. There is a disservice done to teachers who are faced with extraordinary obstacles, such as teaching students who come to school hungry or fearing for their safety, or teachers who create extraordinary learning environments where students learn to create and collaborate in ways that are not reflected by “the test.”

Education as a platform must support vibrant innovation in the area of metrics. States, assessment publishers, web start-ups, researchers, parents and teachers must be able to experiment with different ways to measure student achievement, and, indeed, with what things are important to measure. In a world of assessment innovation, a student portfolio might contain a combination of completed projects in addition to state test results, richer third-party assessment results, and innovative assessments of non-traditional skills such as collaboration and creativity. Colleges and employers might value this multi-dimensional view of a student more than just grades and standardized test results when evaluating applicants. Parents and students might take ownership of enriching their portfolio of assessments according to their own values. Publishers of curriculum and educational experiences might be able to improve their offerings based on a broad set of assessments of student outcomes -- driving innovation in educational content. Administrators and states might be able to reward teachers for many different kinds of critical achievements.

What kinds of services would comprise an education platform that encourages innovation around metrics? Certainly data science is key -- instrumenting how students and teachers interact with technology and digital content and capturing that data according to open standards; storing and cleaning that data with appropriate privacy considerations; performing data mining, analytics, and analysis of the data; and creating meaningful visualizations of the data are all areas that can be both services in support of education as a platform as well as areas for innovation in their own right. The data itself will be a sort of national treasure -- the key to understanding what works and what doesn’t and the fuel for innovative services and applications.

Also key are mechanisms for discovering quality assessments -- Web 2.0 mechanisms of crowd-sourcing, peer review, and rating are commonly invoked for sorting out the best of the best when there is an explosion of innovative products available. But to date, much education-related crowd-sourcing has assumed that teachers will voluntarily rate and recommend educational products. With the burden that teachers are already carrying, that additional volunteerism is not a realistic expectation. Fortunately, there are two communities that might well provide data on products either because they are motivated or as a side effect of using them: education researchers and families. How will education-as-a-platform leverage these communities? Are these the same mechanisms required for discovering exceptional educational experiences?

These kinds of services from the technology infrastructure also serve the human infrastructure. With educators, administrators, and schools using a much richer metric of “better” than state test results, the same feedback loops that currently damp out change can instead drive improvement. Schools (and informal learning channels) can differentiate themselves by offering educational experiences that support the learning goals of each student -- as reflected by the unique portfolio that student is building. Schools themselves can actually “get better the more people use them.”

But how can these hypothetical innovative approaches to metrics (or for that matter curricula, pedagogy, digital textbooks and games or other innovations) spring up in a cumbersome, over-constrained system like our schools? Probably not through top-down design or national mandate. Perhaps, like online learning, an assessment ecosystem will have to evolve outside those constraints -- where there is currently no or little competition. Perhaps parents and students looking to have a more complete picture of a child’s skills and strengths and gaps will be the initial market for innovative assessments. What parent doesn’t want to understand their kids’ achievements in a deeper way than letters on a report card? If a wealth of affordable multi-dimensional assessments were only a click away, how many families would use them and in so doing help make them better? Perhaps, like online learning, innovation in assessments will, in the best way possible, disrupt school and class?

Photo credit: teacher's eye by HoQ-10, on Flickr


Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!