From the Mist

(article was never fleshed out, included here for completeness)

Domain Driven Design has become, to my great joy, a mainstream practice. Various excellent books have appeared on the subject, but for some reason I have constantly felt I missed something essential – or at least essential for me. So I decided to write on the subject myself.
This article attempts to bring something entirely new into the subject of software development, or at least something that is, in my view, severely lacking from almost every book on the subject. We are living in a time when software is becoming increasingly essential in our society. The essence of software however is not lying in its technical applications, or the expertise needed to write software in programming languages designed for technical purposes. In fact, to remain focussed on that aspect of software will prove to be detrimental to further evolution. It will, even more than it is already doing, slow progress in a way that to me is unacceptable. I do not claim to have invented new thoughts or to have the copyright on the ideas you will encounter in this book. On the contrary: I am convinced most if not everything you will read here has been said in some form or another by other people. I will even endeavour to give honour where honour is due. My ambition will be to mix these various thoughts into a new amalgam, and by doing this in a way that while exposing them they will immediately be applied in practice, so they will win in power and impact.
Enough on the introduction. Let’s commence.

Typographical conventions

Footnotes are numbered with a closing bracket: 1). These can be found at the bottom of the page and relate more to the content of the story.

Smalltalk source code is in syntax coloured sans-serif font (not fixed-sized as in most other programming languages examples in other books – this has been the custom with Smalltalk code since the publication of the Smalltalk books by Adele Goldberg[1]). I have added the custom as introduced by Pugh and Wilff to print the names of methods in bold.

Introduction of the MijnGeld case study

For the practical aspect of this article, one case study will be used: that of a bookkeeping program. While read this program will grow to a fully functional (and of course freely downloadable from my website 1) piece of software.

So what will this bookkeeping program do?
At first I want to keep things as simple as possible, but not too simple. It would be too simple to make a program that would record my bank transactions, and offer reports on my income and spending. Why would this be too simple? Because it would lead to a piece of software that is dead from the start: it does nothing, it only records and playbacks.
It would not be software as I define it – it would be hardware, a video recorder, a phone. There is nothing more pathetic than software trying to be hardware.
So the bookkeeping program will not be a recorder. What then? An essential characteristic of living software is that it moves by itself. It will have a goal in life, it will take decisions and change itself by doing that. In a way, but this will be elaborated upon much more later in the book, software writes itself. It should not be the programmer writing it.
The goal of the bookkeeping software will be simple in the beginning: it will do its utmost best to keep the balance of the account zero. Not positive (I will not yet want to save), not negative (I will not yet want to have debits). In the course of writing the book this functionality will grow, and we will pay special attention to the way in which this kind of growth is managed.
Because the software is a living entity, or will be, the name “bookkeeping” is no longer appropriate. We will use another name for this software, one I think is very appropriate indeed: My Money, or, in Dutch: MijnGeld. This name is more appropriate because it designates the piece of evolutionary software we are going to work on as an entity in its own right that mirrors another entity in the outside world: my financial assets. And I guess many of you will agree that money is in itself already an almost living entity with very interesting biological characteristics.
The theory of mirroring is introduced in the main text in the first chapter, The essence of software.

The essence of software

I cannot really take off without making a best effort at conveying my thoughts on what comprises the essence of software. This chapter will contain a fair amount of philosophy, and I have done my best to write it in such a way that, if you so choose, you can skip this chapter without being unable to appreciate the following one, and later return to it in order to appreciate the approach even better.

The biological metaphor

We’ll say it again: software is not a technical thing. The closest metaphor could be the one Kevin Kelly uses 2 in defining technology as a new life form. Maybe it is true as he says that the biological metaphor for technology has become a cliché, but in my experience the consequences of thinking in this metaphor are seldom applied. Software is a complex, moving thing, never at rest, constantly adapting, morphing itself to work better or to offer more functionality. It is the spirit in the machine, itself it cannot be grasped, and it resists manipulation. We may devise as much controlling mechanisms as we want, in our software factories, assembly lines, software design methodologies or project management, but all will be in vain: software cannot be controlled. The only thing we will accomplish is a piece of software that consists mainly of controlling routines, and only an almost negligible portion of end-user functionality.
The impossibility of control in complex systems is systematically denied in modern culture. We live in an age where we prefer to live convinced of our power over life and nature. We are in control, and if and when we are not, it is only a matter of time until we are.
An example of the consequences of this mindset is the recent economic events leading to the worldwide collapse of financial systems in 2008. For years a economic belief system has ruled which declared economic processes to be completely controllable, and in fact even dared call itself “the final economic theory”. Economical systems are interesting examples of complex systems. And as such cannot be controlled or steered.
Another example is the weather, for which we spend millions to set up better prediction models. It will be in vain, as meteorological scientist should be the first to admit. Were it not they have a vested interest in those funds…
Gregory Bateson in his book (or rather a compilation of a series of papers) called “Steps to an Ecology of Mind” applied the biological metaphor to the area of thinking and mind[2]. I often refer to the writings of Gregory Bateson, because his research is of special interest to people attempting to make computers to be actually useful. The whole area of Cybernetics was built on the groundbreaking work of Bateson and Norbert Wiener, and has been a first incarnation of the field of computers and software[3]. To see the mind as a complex system avoids the vague and emotion-loaded notion of consciousness, but nevertheless succeeds in creating a wonderful and rich image of the uniqueness of our mind.
The extent to which the ecological system is a complex system is one I want to give a little extra attention to.

Mirroring

The biological metaphor is one common thread in the approach in this book. Another important one is mirroring. Some recent scientific theories claim than humans started to make use of language to communicate about a million years ago[4], and that the very process of employing language in a specific way that differs in a fundamental way from how animals communicate, has defined our physical as well our mental layout. Another research has shown that our brains, not only our brains but brains in general in all animals, contain specific neurones called “mirror neurones”[5]. One can see these neurones as specialised devices to aid the organism in connecting with the surrounding world.
In fact a whole scientific discipline has evolved around this theory, linking several scientific disciplines in an effort to understand more about what it is to be human, with interesting results in areas like autism therapy.
A mirror is a device that reflects. Our mind is in essence a mirroring device: we are able to constantly reflect the outside world in our inner, mental world.
This inner world is not a “perfect” reflection, that is it is not merely a replica of the outside world. It is in a sense more than that: for example our brains seem to be constantly at work “predicting” the future [6].
One of the most intriguing writers that moulded my thoughts on the subject, is a Dutch psychiatrist, professor J.H. van den Berg. He became rather famous (in fact more famous internationally than in The Netherlands) with his first book, Metabletics, translated in English as The Changing Nature of Man [7]. Jan Hendrik van den Berg introduced me to one of the purest phenomenological approaches to philosophical issues similar to the early Husserl. His writings showed me how intricately our observations are connected to our thoughts, our image of the world. This image is nothing else but the mirror of the world we create in ourselves.
This book is a mirror in several senses:

1. it mirrors my thoughts on the subject of computers and software
2. it mirrors in itself the theoretical aspects of those thoughts into the practical application of MijnGeld
3. it mirrors, as you read it, your own view of the world, and hopefully introduces some few gateways to enrich that view or that world

An important characteristic of a mirror is that it is not an exact replica of the real world and should not attempt to be. I will not go deeply into this aspect here, except from mentioning that a mirror reflects the real world, and in reflecting will change a few things. It is easy to overlook, but a real mirror for example will exchange left and right: the face that looks at you in the mirror is not the face other people see! In a similar vein the software mirror of the thing is different in small but important ways. This will be referred to and further elaborated upon later in the two main patterns:

1. The Time Reversal Pattern
2. The Active-Passive Pattern

Software turns things around!

Quickening

We have talked about objects that are reflections of outside world concepts or things. We have tried to start describing these objects by defining them to be living entities, doing things on their own, and collaborating with a great deal of other objects to do their thing in an appropriate way. I will repeatedly come back to this image, this way of looking at software. Whenever you think of your problem domain, and how to morph it into a software program, you should try to think of this swarm of objects, this ant-hill of ant-like software components, each on its own so simple, but al together able to move mountains.
But especially for those of you (which unfortunately will probably also be most of you…) already polluted by the machine-metaphor which is prevalent in computer science, this metaphor may sound interesting and even alluring, but impossible to understand in a way that is concrete enough to apply in practice.
So let’s start play.
What does a “living” object look like?
One way of exploring this is by comparing “dead” objects (which, unfortunately, are the objects most of us software programmers are accustomed to) with “living” objects.
An example of a dead object is a bank account like this:

Entries into the account are made by Account::addEntry(newEntry).
Why is this a “dead” domain? You can easily detect this property by noticing that things are done to the objects, not by the objects. They are passive themselves, they do not act on their own. They are, so to say, passive victims of whoever fancies to send them certain messages like addEntry().

An example of a domain which is a bit more alive is the next one:

In this model a bit more behaviour is introduced. The Transaction has two entries (a so-called two-legged transaction[8]). The behaviour is in this class Transaction. It “executes itself”, that is:

• it is told some things (passive: amount, from account and to account)
• it does some things (active: do some things with the accounts)

This can be further explained by the code of method fromTo (in Smalltalk, which has keyword syntax for multi-argument methods, this becomes from:to:)3:

from: firstAccount to: secondAccount withAmount: anAmount
“Create the entries in both accounts for both legs of the transaction.”

| firstEntry secondEntry |
firstEntry := Entry amount: anAmount negated.
secondEntry := Entry amount: anAmount.
firstAccount addEntry: firstEntry.
secondAccount addEntry: secondEntry

You may have noticed that the complete passivity of objects in our first attempt has improved. Not much, I agree, but some.
This may not seem much but there is actually quite some improvement. An important aspect of object-orientation is encapsulation. A transaction is told some basic things, but the manner in which it will subsequently perform its responsibilities is invisible from the outside. This can for example be illustrated by the necessity to deduct some transaction fees for each transaction. This could be implemented by adding one line to the previous method: 4

from: firstAccount to: secondAccount withAmount: anAmount
“Create the entries in both accounts for both legs of the transaction.”

| firstEntry secondEntry |
firstEntry := Entry amount: anAmount negated.
secondEntry := Entry amount: anAmount.
firstAccount addEntry: firstEntry.
secondAccount addEntry: secondEntry.
self deductTransactionFee

You may now appreciate this one level of added indirection since we can now change behaviour in one place, and all originally involved parties will not need to change in any way.
But is this one step enough? Can we take our metaphor of living objects further? Because you may observe that although the second solution is a little better, it is still far removed from being “alive”, whatever that may mean.
We will take a bold step here, of which I will hope that you are willing to follow me here.
I will introduce time. And memory. Something living entities possess.

Living in time

Living objects have memory, an essential attribute to learn.
Introducing time into the story can be very simple:
timeLine <SortedCollection of: Event>

Smalltalk Syntax

This is a short chapter, as it should be. Programming is difficult enough even with a minimally complex language. With thanks to Travis Griggs.
There is actually two parts to Smalltalk code:

• The language syntax
• The structure of the code base (classes, packages, etc.)

Language Syntax

As many have said before, the Smalltalk language syntax is almost ascetic. Everything you encounter (well, almost everything ;-)) is messages sent to receivers.

Code structure

Different Smalltalk dialects have decided on different approaches to structure the Smalltalk class library. Depending on your dialect of choice you may encounter packages like in Java, Namespaces like in .NET, bundles, parcels, categories and what not.
For this book we can discard these differences for the most part. The important thing is classes containing:

• Methods
• Attributes, called Instance Variables in Smalltalk

Bibliography

Goldberg, A. and D. Robson, Smalltalk-80 : the language and its implementation. Addison-Wesley series in computer science. 1983, Reading, Mass.: Addison-Wesley. xx, 714 p.

Bateson, G., Steps to an ecology of mind; collected essays in anthropology, psychiatry, evolution, and epistemology. Chandler publications for health sciences. 1972, San Francisco,: Chandler Pub. Co. xxviii, 545 p.

Waldrop, M.M., Complexity: the emerging science at the edge of order and chaos. 1992, New York: Simon & Schuster. 380 p.

Deacon, T.W., The symbolic species : the co-evolution of language and the brain. 1st ed. 1997, New York: W.W. Norton. 527 p.

Nehaniv, C.L. and K. Dautenhahn, Imitation in animals and artifacts. Complex adaptive systems. 2002, Cambridge, Mass.: MIT Press. xv, 607 p.

Hawkins, J. and S. Blakeslee, On intelligence. 1st ed. 2004, New York: Times Books. 261 p.

Berg, J.H.v.d., The changing nature of man; introduction to a historical psychology, Metabletica. 1st ed. 1961, New York,: Norton. 252 p.

Fowler, M., Analysis patterns : reusable object models. The Addison-Wesley series in object-oriented software engineering. 1997, Menlo Park, Calif.: Addison Wesley. xxi, 357 p.

Footnotes

• 1 The download link is on my website: http://www.robvens.nl but note there are two versions of the software: one that is and has been used in production for many years, and the other which is from this book and very much evolutionary. The first one can be found on another site: http://www.sepher.nl
• 2 http://www.ted.com/index.php/talks/kevin_kelly_on_how_technology_evolves.html
• 3 For those unfamiliar with Smalltalk syntax, take a short trip to 5: Smalltalk Syntax
• 4 I will briefly mention an important pattern here which I explicitly did not apply, namely the Add, Don’t Modify pattern. That is, if this code would have been in production I would refrain from modifying the code in way I did it here, but create a wrapper solution that would leave the original code untouched but added the desired behaviour elsewhere. More about this pattern later.