Adam Bosworth’s Weblog

I gave a talk yesterday at the ICSOC04. It was essentially a reminder to a group of very smart people that their intelligence should be used to accomodate really simple user and programmer models, not to build really complex ones. Since I was preceded by Don Ferguson of IBM and followed the next day by Tim Berners-Lee, it seemed especially wise to stick to simple and basic ideas. Here is the talk

---

I’m sandwiched by smarter and more august speakers. Don Ferguson of IBM builds edifices of such sophistication and elaboration as to daunt the designers of the extraordinary archways of the Alhambra. Tim Berners Lee created the web as we know it today and preaches a sort of religion about the semantic web from his eerie at MIT that is totally over my head. These are very smart gentlemen. One would be foolish to try to appear smart when speaking between them. Accordingly, I’m going to take the opposite tack. I’m going to talk about the virtues of KISS which I’ll conveniently describe as keeping it simple and sloppy and its effect on computing on the internet.

There has been, of course, an eternal tension between that part of humanity which celebrates our diversity, imperfectability, and faults, as part of the rich tapestry of the human condition and that part which seeks to perfect itself, to control, to build complex codes and rules for conduct which if zealously adhered to, guarantee an orderly process.

This talk is about this conflict as it relates to computing on the Internet. This talk is also a polemic in support of KISS. As such it is unfair, opinionated, and perhaps even unconscionable. Indeed, at times it will verge on a jeremiad.

It is an ironic truth that those who seek to create systems which most assume the perfectibility of humans end up building the systems which are most soul destroying and most rigid, systems that rot from within until like great creaking rotten oak trees they collapse on top of themselves leaving a sour smell and decay. We saw it happen in 1989 with the astonishing fall of the USSR. Conversely, those systems which best take into account the complex, frail, brilliance of human nature and build in flexibility, checks and balances, and tolerance tend to survive beyond all hopes.

So it goes with software. That software which is flexible, simple, sloppy, tolerant, and altogether forgiving of human foibles and weaknesses turns out to be actually the most steel cored, able to survive and grow while that software which is demanding, abstract, rich but systematized, turns out to collapse in on itself in a slow and grim implosion.

Consider the spreadsheet. It is a protean, sloppy, plastic, flexible medium that is, ironically, the despair of all accountants and auditors because it is virtually impossible to reliably understand a truly complex and rich spreadsheet. Lotus corporation (now IBM), filled with Harvard MBA’s and PhD’s in CS from MIT, built Improv. Improv set out “to fix all this”. It was an auditors dream. It provided rarified heights of abstraction, formalisms for rows and columns, and in short was truly comprehensible. It failed utterly, not because it failed in its ambitions but because it succeeded.

Consider search. I remember the first clunky demos that Microsoft presented when Bill Gates first started to talk about Information at your fingertips with their complex screens for entering search criteria and their ability to handle Boolean logic. One of my own products, Access had the seemingly easier Query by Example. Yet, today half a billion people search every day and what do they use? Not Query by Example. Not Boolean logic. They use a solution of staggering simplicity and ambiguity, namely free text search. The engineering is hard, but the user model is simple and sloppy.

Consider user interface. When HTML first came out it was unbelievably sloppy and forgiving, permissive and ambiguous. I remember listening many years ago to the head, then and now, of Microsoft Office, saying contemptuously in 1995 that HTML would never succeed because it was so primitive and that Word would win because Word documents were so rich and controlled in their layout. Of course, HTML is today the basic building block for huge swathes of human information. What is more, in one of the unintended ironies of software history, HTML was intended to be used as a way to provide a truly malleable plastic layout language which never would be bound by 2 dimensional limitations, ironic because hordes of CSS fanatics have been trying to bind it with straight jackets ever since, bad mouthing tables and generations of tools have been layering pixel precise 2 dimensional layout on top of it. And yet, ask any gifted web author, like Jon Udell, and they will tell you that they often use it in the lazy sloppy intuitive human way that it was designed to work. They just pour in content. In 1996 I was at some of the initial XML meetings. The participants’ anger at HTML for “corrupting” content with layout was intense. Some of the initial backers of XML were frustrated SGML folks who wanted a better cleaner world in which data was pristinely separated from presentation. In short, they disliked one of the great success stories of software history, one that succeeded because of its limitations, not despite them. I very much doubt that an HTML that had initially shipped as a clean layered set of content (XML, Layout rules – XSLT, and Formatting- CSS) would have had anything like the explosive uptake.

Now as it turns out I backed XML back in 1996, but as it turns out, I backed it for exactly the opposite reason. I wanted a flexible relaxed sloppy human way to share data between programs and compared to the RPC’s and DCOM’s and IIOP’s of that day, XML was an incredibly flexible plastic easy going medium. It still is. And because it is, not despite it, it has rapidly become the most widely used way to exchange data between programs in the world. And slowly, but surely, we have seen the other older systems, collapse, crumple, and descend towards irrelevance.

Consider programming itself. There is an unacknowledged war that goes on every day in the world of programming. It is a war between the humans and the computer scientists. It is a war between those who want simple, sloppy, flexible, human ways to write code and those who want clean, crisp, clear, correct ways to write code. It is the war between PHP and C++/Java. It used to be the war between C and dBase. Programmers at the level of those who attend Columbia University, programmers at the level of those who have made it through the gauntlet that is Google recruiting, programmers at the level of this audience are all people who love precise tools, abstraction, serried ranks of orderly propositions, and deduction. But most people writing code are more like my son. Code is just a hammer they use to do the job. PHP is an ideal language for them. It is easy. It is productive. It is flexible. Associative arrays are the backbone of this language and, like XML, is therefore flexible and self describing. They can easily write code which dynamically adapts to the information passed in and easily produces XML or HTML. For them, the important issue is the content and the community, not the technology. How do they find the right RSS feeds? How do they enable a community to collaborate, appoint moderators, and dynamically decide whose posts can go through and whose should be reviewed? How do they filter information by reputation? These are the issues that they worry about, not the language.

In the same way, I see two diametrically opposed tendencies in the model for exchanging information between programs today:

On the one hand we have RSS 2.0 or Atom. The documents that are based on these formats are growing like a bay weed. Nobody really cares which one is used because they are largely interoperable. Both are essentially lists of links to content with interesting associated metadata. Both enable a model for capturing reputation, filtering, stand-off annotation, and so on. There was an abortive attempt to impose a rich abstract analytic formality on this community under the aegis of RDF and RSS 1.0. It failed. It failed because it was really too abstract, too formal, and altogether too hard to be useful to the shock troops just trying to get the job done. Instead RSS 2.0 and Atom have prevailed and are used these days to put together talk shows and play lists (podcasting) photo albums (Flickr), schedules for events, lists of interesting content, news, shopping specials, and so on. There is a killer app for it, Blogreaders/RSS Viewers. Anyone can play. It is becoming the easy sloppy lingua franca by which information flows over the web. As it flows, it is filtered, aggregated, extended, and even converted, like water flowing from streams to rivers down to great estuaries. It is something one can get directly using a URL over HTTP. It takes one line of code in most languages to fetch it. It is a world that Google and Yahoo are happily adjusting to, as media centric, as malleable, as flexible and chaotic, and as simple and consumer-focused as they are.

On the other hand we have the world of SOAP and WSDL and XML SCHEMA and WS_ROUTING and WS_POLICY and WS_SECURITY and WS_EVENTING and WS_ADDRESSING and WS_RELIABLEMESSAGING and attempts to formalize rich conversation models. Each spec is thicker and far more complex than the initial XML one. It is a world with which the IT departments of the corporations are profoundly comfortable. It appears to represent ironclad control. It appears to be auditable. It appears to be controllable. If the world of RSS is streams and rivers and estuaries, laden with silt picked up along the way, this is a world of Locks, Concrete Channels, Dams and Pure Water Filters. It is a world for experts, arcane, complex, and esoteric. The code written to process these messages is so early bound that it is precompiled from the WSDL’s and, as many have found, when it doesn’t work, no human can figure out why. The difference between HTTP, with its small number of simple verbs, and this world with its innumerable layers which must be composed together in Byzantine complexity cannot be overstated. It is, in short, a world only IBM and MSFT could love. And they do.

On the one hand we have Blogs and Photo Albums and Event Schedules and Favorites and Ratings and News Feeds. On the other we have CRM and ERP and BPO and all sorts of enterprise oriented 3 letter acronyms.

As I said earlier, I remember listening many years ago to someone saying contemptuously that HTML would never succeed because it was so primitive. It succeeded, of course, precisely because it was so primitive. Today, I listen to the same people at the same companies say that XML over HTTP can never succeed because it is so primitive. Only with SOAP and SCHEMA and so on can it succeed. But the real magic in XML is that it is self-describing. The RDF guys never got this because they were looking for something that has never been delivered, namely universal truth. Saying that XML couldn’t succeed because the semantics weren’t known is like saying that Relational Databases couldn’t succeed because the semantics weren’t known or Text Search cannot succeed for the same reason. But there is a germ of truth in this assertion. It was and is hard to tell anything about the XML in a universal way. It is why Infopath has had to jump through so many contorted hoops to enable easy editing. By contrast, the RSS model is easy with an almost arbitrary set of known properties for an item in a list such as the name, the description, the link, and mime type and size if it is an enclosure. As with HTML, there is just enough information to be useful. Like HTML, it can be extended when necessary, but most people do it judiciously. Thus Blogreaders and aggregators can effortlessly show the content and understanding that the value is in the information. Oh yes, there is one other difference between Blogreaders and Infopath. They are free. They understand that the value is in the content, not the device.

RSS embodies a very simple proposition that Tim Berners Lee has always held to be one of the most important and central tenets of his revolution, namely that every piece of content can be addressed by a URL. In the language of RSS we call these “PermaLinks”. This idea has profound value. Dave Sifry of Technorati pointed out to me recently that one of the most remarkable things about RSS and Web Logs (Blogs) is the manner in which they have started to solve one of the most tragic things about the web, namely the incivility of the discourse. The web, in many ways, today represents a textbook example of the tragedy of the commons. Because sending email is virtually free, we have spam. Because posting messages is virtually free and anonymous, we have groups where a small number of people can overwhelm the discussion with loud and senseless chatter. But one of the values of being able to reference every element is that now comments about elements can be distributed over the web. The web becomes something like a giant room in which people comment on other people’s thought via posts in their own Web Logs. In so doing they put their reputation on the line. These are hardly cheap and anonymous posts. They take up real estate in a place that is associated with your own point of view and reputation. And thus the comments tend to be measured, thoughtful, and judicious. Furthermore if they are not, either you can decide that it is OK or you can opt out. It is like dueling editorials in a pair of news papers.

By contrast, the rigid abstract layers of web service plumbing are all anonymous, endless messages flowing through under the rubric of the same URL. Unless they are logged, there is no accountability. Because they are all different and since the spec that defines their grammar, XML Schema, is the marriage of a camel to an elephant to a giraffe, only an African naturalist could love these messages. They are far better, mind you, than the MOM messages that preceded them. Since they are self describing, it is possible to put dynamic filters in to reroute or reshape them using XPATH and XSLT and XML Query and even other languages all of which can easily detect whether the messages are relevant and if so, where the interesting parts are. This is goodness. It is 21st century. But the origination and termination points, wrapped in the Byzantine complexity of JAX RPC or .NET are still frozen in the early bound rigidity of the 20th.

I would like to say that we are at a crossroads, but the truth is never so simple. The truth is that people use the tools that work for them. Just as for some programmers the right tool is PHP and for others the right tool is Java so it is true that for some programmers the right tool is RSS and for others it is WS-*. There is no clear “winner” here. What I am sure about is the volumes and the values. The value is in the information and its ability to be effortlessly aggregated, evaluated, filtered, and enhanced.

What does this mean to you? Think of the radio. When it was a novelty, the real value was in the radio itself. There was relatively little content, but lots of people wanted the radio. At a certain point, however, radios got good enough and transmission got good enough and the value ineluctably swung to the content. This is why the DRM fights are so bitter, why PodCasting is so revolutionary, why Howard Stern was paid so much to play on a private radio model. That’s where the value is. We have arrived at the same point for computing. The value is neither in the computers nor in the software that runs on them. It is in the content and the software’s ability to find and filter content and in the software’s ability to enable people to collaborate and communicate about content (and each other). Who here really cares if Excel adds a new menu item unless it is one that lets you more easily discover information on the web, possibly update and interact with it or with others about it.

What about mobile phones. What do they mean? Is it really interesting to have a spreadsheet or a power point on your mobile phone? Or is it more interesting to know where the nearest ATM is, where is the nearest Indian restaurant that your friends like, which are the CS books in the store for a given course, which course has the best section person and what its schedule is, or what the reviews of the books say. Is it really interesting to have an address book that is synced to your PC or is it more interesting to see the presence of the people who are involved in your class, your project, your party plans, and be able to coordinate and plan an event with them? And if it is the latter, then isn’t the value really coming from the knowledge of with whom you are working, socializing, and studying; what they think about things you care about such as movies, classes, restaurants and news articles rather than the software on the device itself? Isn’t the device really just a sort of n-way radio/classified? Soon as you deliver context and content and community and collaboration over the web, 2 billion people will be able to see and interact with your solutions.

There is a lot of talk about Web 2.0. Many seem to assume that the “second” web will be about rich intelligent clients who share information across the web and deal with richer media (photos, sound, video). There is no doubt that this is happening. Whether it is Skype or our product Hello, or iTunes, people are increasingly plugging into the web as a way to collaborate and share media. But I posit that this isn’t the important change. It is glitzy, fun, entertaining, useful, but at the end of the day, not profoundly new.

What has been new is information overload. Email long ago became a curse. Blogreaders only exacerbate the problem. I can’t even imagine the video or audio equivalent because it will be so much harder to filter through. What will be new is people coming together to rate, to review, to discuss, to analyze, and to provide 100,000 Zagat’s, models of trust for information, for goods, and for services. Who gives the best buzz cut in Flushing? We see it already in eBay. We see it in the importance of the number of deals and the ratings for people selling used books on Amazon. As I said in my blog,
“My mother never complains that she needs a better client for Amazon. Instead, her interest is in better community tools, better book lists, easier ways to see the book lists, more trust in the reviewers, librarian discussions since she is a librarian, and so on”.
This is what will be new. In fact it already is. You want to see the future. Don’t look at Longhorn. Look at Slashdot. 500,000 nerds coming together everyday just to manage information overload. Look at BlogLines. What will be the big enabler? Will it be Attention.XML as Steve Gillmor and Dave Sifry hope? Or something else less formal and more organic? It doesn’t matter. The currency of reputation and judgment is the answer to the tragedy of the commons and it will find a way. This is where the action will be. Learning Avalon or Swing isn’t going to matter. Machine learning and inference and data mining will. For the first time since computers came along, AI is the mainstream.

I find this deeply satisfying. It says that in the end the value is in our humanity, our diversity, our complexity, and our ability to learn to collaborate. It says that it is the human side, the flexible side, the organic side of the Web that is going to be important and not the dry and analytic and taxonomical side, not the systematized and rigid and stratified side that will matter.

In the end, I am profoundly encouraged and hopeful that the growth on the web is one which is slowly improving the civility and tenor of discourse. Just as Porn seems to be an unpleasant leading user of technology, so does crude and vituperative communication seem to be a pattern for early adopters and it is a relief to see that forms of governance, trust and deliberation are now emerging.

There are those who will say that all this is utopian. If Utopian means not being afraid to dream, then indeed it is. So was Tim’s initial vision of universal access to information. So is Google’s mission. T.E Lawrence wrote in the Seven Pillars of Wisdom,

“All men dream: but not equally. Those who dream by night in the dusty recesses of their minds wake in the day to find that it was vanity: but the dreamers of the day are dangerous men, for they may act their dream with open eyes, to make it possible”

I encourage all of you to act your dreams with open eyes. I encourage all of you to dream of an internet that enables people to work together, to communicate, to collaborate, and to discover. I encourage all of you to remember, that in the long run, we are all human and, as you add value, add it in ways that are simple, flexible, sloppy, and, in the end, everything that the Platonists in you abhor.

This entry was posted on Thursday, November 18th, 2004 at 1:44 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.