A Timeless Way of Building Software

Most of my readers know that I'm a software architect by trade.  I've been creating software large and small for over twenty years.  And I've experienced movement after movement in software design from object-orientation in the 1980s and early 90s to component-based design, distributed objects, Web-based software, service-oriented architecture and too many others to even mention.  I'm pretty jaded at this point because I've learned, in general, the problems that you couldn't solve in the previous generation of technique are often only marginally more solveable in the next generation (which is invariably invented to "fix" the previous problems.)

Alas, a genuinely better mousetrap is really hard to find.

So in the end, if you couldn't do whatever it is you wanted to do with the previous generation of technique, it's actually not that likely you'll succeed in the next.  Certain software problems remain hard, and in general, it mysteriously happens to involve the juncture between technology and people in some way.  To paraphrase this, I could say that the software and techniques get better fairly constantly, but people remain the same.

And please, bear with me because I'm going to try out a real zinger on you in a minute. 

Because every once in a long while, something new and big actually does come along.  Or at least something that looks new and big.  One of the new and big things that came along about ten years ago was the concept of design patterns .  It was pretty neat stuff.  It said that despite the current technology we have, the processes that continue to evolve, there are certain timeless solutions to certain software design problems.  It was a revelation at the time.  And the writers of the book that explained this got both famous and very successful.  Why? Because these design patterns really worked is why.  And anyone who has read the books and has ever really built software recognizes these patterns.  And what was strange was that no one really expected it. One day, we just had them.  And the kicker was, they were always there, but now they were in our conscious thought and we had real names for them.  My point: They were in our face all the time but most of us couldn't see them.

We are in a similar place with the Web right now.  We've done this Web stuff enough now that we are just beginning to see the design patterns.  What works, and why, in a specific situations, bounded by forces.  Some folks have had the hubris to give this next generation a name and to tease out these patterns.  Some people are just now going aha, and some people haven't got it yet, and most of the rest of us either aren't ready for it or just haven't heard of it.  But, I will tell you this.  It's quite real.  The best practices and design patterns of Web software are just starting to become understood.  The strange part is, we're discovering the same things over again.  What's old is new again.

Now, before you get all worked up or worse, I bore you and you stop reading, I will give you a nice list of the the forces behind these patterns.  If you recall, design patterns are a solution to a problem in context.  We are starting to get the context and even the outlines of the patterns of this "new" generation of software.  But we have a long way to go still.  The Web is a monstrously big space with big problems, and it's not getting better.  There are one billion of us out here now.  Clearly understanding what it takes to create great software on the Web that is successful, useful, and vibrant will be an ongoing challenge for a long time.  But it will get easier because we are codifying our knowledge of this exciting and huge place where we now find ourselves.

Figure 1:  The driving forces in modern software.
With a rough comparison between SOA
and The Timeless Way (Web 2.0 by any other name).

Now is where I'm going to hit you with a flight of fancy.  I'm going to use Christopher Alexander's opening chapter of a Timeless Way of Building and tailor it to describe this old-but-new way of building the Web and software for it.  We are lacking for a little inspiration and this book in particular continues to sell upwards of 10,000 copies a year, 25 years after it was frst published.  And Christopher Alexander, for those of you who may not know, was the person that originally discovered the design pattern.  But it wasn't for software.  It was for creating great, timeless buildings.  He was one of the first that realized that his field of endeavor has certain elemental, timeless cores, no matter the technique, building material, or the people.  It was an amazing discovery that poured over into the software world with considerable success. 

My assertion is that nothing has really changed in software, we might understand the forces better but they are almost always the same.  People want software that does what they want, is available when they need it.  They want software that grows with them, helps them, teaches them, and lets them do the same with others.  They want software that gets out of their way, disappears, and is more convenient by far than inconvenient.  A nd they want to pay as little as possible for it, but enough so that it's worth it.  They are willing to have software get right into the middle of their lives.  If it's the right software.  And as long as we've had software, they've always wanted this. But now they might actually start getting it.

In any case, I don't literally believe every phrase in this take-off, but I do believe the overall concept deeply and profoundly as a software professional.  And I will continue to update the diagram above (clearly marked beta 1) until we have more of the forces in it. And some are definitely missing.  Please, as always, leave any comments and suggestions for improvement below.

And now, without further ado, here is the The Timeless Way of Building Software, with sincere apologies to Christopher Alexander:

The Timeless Way of Building Software

Inspiration For The Next Generation of Web Software

There is one timeless way of building software.  It is decades old and is the same today as it's always been.  And because it is timeless, it will always remain this way.

The great software of our time has always been created by people who were close to this way.  It isn't possible to create great software - software that is satisfying, and useful, and makes itself a natural extension of life - except by following this way.  And as you will see, this way will lead anyone who looks for it to elegant, vibrant software which is itself timeless in its form.

It is the process by which the function of a piece of software grows directly from the inner nature of people and naturally out of the raw bits, the otherwise meaningless digital medium, of which it is made.

It is a process which allows the life inside a person, or a group of people, or a community to flourish, openly, in freedom, so vividly that it gives rise, of its own accord, to the natural order which is needed to be contained within it.

It is so powerful and fundamental that with its help you can create software that is as beautiful and enriching as anything else you have ever seen.

Once you understand this way, you yourself will be able to create software that is alive, that is intertwined comfortably with your life and the lives of others. You will design worlds where you and others will want to work, play, and co-exist together; beautiful places where you can sit and dream comfortably.

This way is so powerful, that with its help hundreds or thousands, or even hundreds of thousands of people, can come together together to create software and community which is as alive and vibrant, peaceful and relaxed, as any living experience has ever been.

Without the central control of authorities and experts, if you are working in this timeless way, a genuine place will grow right from underneath your fingertips, as steady as the grass in the fields or the trees in your backyard.

And there is no other way in which a software which is fundamentally good can possibly be made.

That doesn't mean that all ways of making software are identical.  Quite the contrary. It means that at the core of all successful software and at the core of all successful processes of creation and evolution, there is one fundamental invariant feature which is responsible for their success.  Although this way has taken on a thousand different forms at different times, in different places, still, there is an unavoidable, invariant core to all of them.

Take a look at the some of the great Web software like Google's search page, Flickr or del.icio.us.  They all have that unique, yet unhurried, grace which comes from perfect ease and natural balance.  But what is it they have in common exactly?  They are beautiful, ordered, harmonious - yes, all of these things.  But especially, and what strikes to the heart, they live.

Each one of us yearns to be able to bring something to life like this. Or just be a part of it somehow.

It is a fundamental human instinct, as much a part of our desire as the desire to be part of something greater than ourselves.  It is, quite simply, the desire to make a part of nature, to complete a world which is already made of mountains, streams, stones, buildings, ourselves, our living systems, and our increasing connectedness together.

Each one of us has, somewhere in our heart, the dream to make a living world, a universe, and place of our own for us to share with others.

Those of us who have trained as software designers have this desire perhaps at the very center of our lives; that one day, somewhere, somehow, we shall build a software experience which is wonderful, beautiful, and breathtaking; a place where people can go and live their dreams.

In some form, every person has some version of this dream; whoever you are, you may have the dream of one day creating a most beautiful place, virtual or otherwise, where you can come together with others and freely share your knowledge, learn, participate in your community or government, and otherwise conduct your daily interaction with the rest of the world.

In some less clear fashion, anyone who is concerned with communities and other large group efforts has this same dream, perhaps for the entire world.

And there is a way that software can actually be brought to life like this.

There is a definable sequence of activities which are the heart of all acts of software design, and it is possible to specify, precisely, under way conditions these activities will generate software which is alive.  All this can be made so explicit that anyone can do it.

And just so, the process by which a group of independent people can make software become alive and create a place as real as any other can equally be made precise.  Again, there is a definable sequence of activities, more complex in this case, which are the heart of all collective processes of software creation.  And it is also possible to specify exactly when these processes will bring things to life.  And once again, these processes can be made so explicit, and so clear, that any group of people can make use of them.

This process is behind the design of community built software like Linux, Apache, Wikipedia, and many others.  It was behind the design of the great virtual places for people to live and work: the Internet, Usenet, and the World Wide Web.  It was behind the creation of simple, satisfying software of the kind that powers the iPod, the Blackberry, and Firefox; of SourceForge, Wikipedia, and BitTorrent.  In an unconscious form, this way has been behind almost all ways of creating software since the beginning.

But it has become possible to identify it, only now, by going to a level of analysis which is deep enough to show what is invariant in all of the different versions of this way.

This hinges on a form of representation which reveals all possible design processes, as versions of one most fundamental set of patterns.

First, we have a way of looking at the ultimate constituents of the environment: the ultimate "things" which a piece of software is made of.  As we shall see, every piece of software is made of certain fundamental entities known as design patterns; and once we understand software in terms of its patterns, we have a way of looking at them, which makes all software, all of their parts and function, all members of the same class of thing.

Second, we have a way of understanding the generative processes which give rise to these patterns: in short, the source from which the ultimate constituents of software come.  These patterns tend to come from certain combinatory processes, which are different in the specific patterns that they generate, but always similar in their overall structure, and in the way they work.  They are essentially like languages.  And again, in terms of these pattern languages, all the different way of building software, although different in detail, become similar in general outline.

At this level of analysis, we can compare many different software creation processes.

Then, once we see their differences clearly, it becomes possible to define the difference between those processes which make software vibrant, alive, and useful, and those which make them the opposite.

And it turns out that, invariant, behind all processes which allow us to make great software, there is a single common process.

This single idea is operational and precise.  It is not merely a vague idea, or a class of processes which we can understand: it is conrete enough and specific enough, so that it functions practically.  It gives us the power to make software and virtual communities live, as concrete as a match gives us the power to make flame.  It is a method of a discipline, which teaches us precisely what we have to do make our software what we want it to be.

But though this method is precise, it cannot be used mechanically.

The fact is, that even when we have seen deep into the processes by which it is possible to make software alive, in the end, it turns out this knowledge only brings us back to that part of ourselves which is forgotten.  Although the process is precise, and can be defined in exact scientific terms, finally it becomes valuable, not so much because it shows us things which we don't know (though it may do that as well), but instead, because it shows us what we know already.

Of course, this way of building software has never be named.  It's not service-oriented architecture, or the personal software process, or agile methodology, or the unified process, or CMM, or any of the others.  It's the actual things that are conceived and done and worried about when software is created and used.  For now, because all software is quickly becoming connected to all other software, and because the Web is becoming the place where more and more of the relevant software is, and finally because it is a more complete reconception of what we thought we knew, we'll give it a name temporarily.  An unsatisfying name, but one that we can remember for now.

We will call it Web 2.0.

What do you think?  Are we at a place where we can really identify the design patterns in Web 2.0?

Traceback:  http://web2.wsj2.com/a_timeless_way_of_building_software.htm