Meditation #6 agile manifesto principle 11, insert “do not”

“The best architectures, requirements, and designs emerge from self-organizing teams”

is the eleventh principle out of twelve in the agile manifesto. 

Taken at face-value it is somewhat difficult to understand what exactly it means. In order to unwrap the import of this we need to investigate the meaning of some key concepts.

The core idea seems to be that order in the form of architecture and design emerges by itself from a self-organising team. 

Self-organisation occurs at many levels in nature. From basic physical processes and chemical reactions leading to crystallisation over biology to macro phenomena in society and the economy. It is a property of multiple interactions within a system. 

The idea that dynamics within a system generates order by itself can be traced to ancient atomists. According to Democritus the world consisted of small invisible atoms in a void. The motion of the cosmos separates the atoms according to their properties. Heavy ones go together like pebbles on the beach. This is also how life appeared: living things emerged out of slime. 

This view was influential in philosophy until the 18th century, where the 2nd law of thermo dynamics was discovered. According to the 2nd law of thermo dynamics, order will decrease with time in any isolated system. This means that a system cannot by itself increase order without influence from outside the system. 

The agile manifesto thus seems to expound a pre-socratic natural philosophy that has been abandoned due to better knowledge in modern science and understanding of natural laws that were gained during the past couple of centuries. 

Even if order would arise from the self-organisation of the team one would expect this only to result in emergent order of the team, not its products. This is similar to how physical and chemical processes result in emergent order of the material itself as is the case with crystals. Note also that crystals do not emerge as order by itself but due to external processes in the form of heat and pressure. Powdered carbon does not spring into crystal form spontaneously. 

Consequently a consistent emergentist view could hold that the order of a team would predictably find an ordered form characterised by some properties like group size, structure or composition. But there is no reason to assume anything about the products of the team.

We can therefore conclude that order does not arise by itself. The best architectures and designs do not just emerge de novo from team interactions when these teams are self-organising. This would be against the 2nd law of thermo dynamics. Order has to be come from the outside. 

Unfortunately, this erroneous view has imbued agile development with an unwillingness to design and architect to the point where many modern developers are vocally antagonistic to any form of upfront design. The effect, of course, is an increase in disorder. This can take many forms. Some are visible as technical debt, some are invisible as bad design leading to instability of systems. 

One would think that eventually they would come to the realisation that this was the case, but another part of the agile mindset precludes this realisation, namely that gradual changes and refactoring are normal and laws of nature. In a sense they are correct. Because nothing is done to reduce entropy due to the unwillingness to do upfront design systems constantly need to be refactored taking away the possibility to work on something more worthwhile. 

Another consequence is instability because system interactions become more complex when there is no design to willingly make them simpler. 

I have worked with different generations of developers and systems, from mainframes to apps. Pre-agile systems can be really ugly too, no doubt about that, but they were made in an era where design and architecture was often done as a natural part of development. I have seen such systems run without error in a stable fashion for 40 years plus. If a modern system developed by an agile team runs more than a few years at all and without incidents it would be a rare occurrence. This is not because modern developers are worse, quite the contrary. Today developers typically have longer more dedicated study programmes behind them. The reason is not either that technologies change faster today. The reason is only that it is viewed as bad to do upfront design and considered okay to rewrite everything in the name of refactoring every once in a while. 

This eleventh principle has potentially undermined most of the gains that the other eleven principles have brought. Fortunately, many companies have not implemented agile in full as envisioned by the agile manifesto. Today for example SAFe has a more realistic view of the need for design and combines this with the insights of the agile manifesto. 

We should therefore either delete the eleventh principle or amend it with a “do not” to read: “The best architectures, requirements, and designs DO NOT emerge from self-organizing teams”. This has to be supplied by dedicated architecture and design work from outside the development team.