Iterative or agile development in one flavor or other has become the standard for IT development today. It is in many contexts an improvement on plan based or waterfall development, but it inherits some of the same basic weaknesses. Like plan-based development it is based on decomposing work into atomic units of tasks with the purpose of optimizing throughput and thereby delivering more solutions faster. In most formulations from SAFe over kanban to DevOps the basic analogy is the production line and often the actual source of inspiration is from the manufacturing world with titles such as Don Reinertsen’s “Managing the Design Factory”. Similarly, the plot of Gene Kim’s 2013 DevOps novel “The Phoenix Project” revolves around learning from factory operations to save a troubled company’s IT development process. While iterative approaches spring from advances in manufacturing processes like Lean, Six Sigma, TQM and others, they are stuck in the same mental prison that waterfall was: a linear mode of thought where the world is a production line through which atomic units move and become assembled. To understand why let’s dig a bit deeper.
The origins of iterative development’s linear mode of thought
Like most things development practices don’t spring from a vacuum. They have roots in the culture in which they emerge. The anthropologist Bradd Shore has argued that the most pervasive cultural model underpinning everything from sports over education to fast food in American culture is modularization. A cultural model is a way of structuring our experiences and how we think about problems and solutions. According to the modular model things are broken down to isolated component parts with a specific function. Through the outsized influence of American culture on modernity globally this model has been disseminated in various forms to the whole world. It can however be traced back to the production line.
As early as 1948 the British anthropologist Geoffrey Gorer noted in a study of the American national character how the pervasive atomism of American institutions could be traced to the great success of the production line in American industry. According to Gorer the industrial metaphors became the basis of a distinctively American view of human activity. It is indeed so pervasive that it persists today also in the foundations of how we view the activity of IT development, an activity where no physical goods move through any physical space but nevertheless, we have chosen this as the model to conceptualize it while others could have been chosen. The basic unit are tasks that are worked on one at a time by a specialist. The deliverable passes through different specialists as it passes through the production line. From design, through development, test, to deployment.
It is perhaps not surprising that the state of the art in development globally is based on the model of the production line since it has been immensely successful and, in many ways, transformed our world to what we see today, but the question is whether that continues to be helpful. Is the production line really the best model of conceptualizing IT development?
Another, more circular, approach
In the new millennium a new design philosophy emerged that challenged these assumptions that had been so pervasive in modern culture. It focused on circularity rather than linearity. It was developed in a number of different books such as McDonough and Braungarts Cradle to Cradle: Remaking the Way We Make Things that converged on a model that valued circularity rather than linearity. This is why it is commonly known as circular economy. One of the main ideas about circular economy is that we should think differently about waste, not as something to throw away but rather as a potential resource. Another important idea is to think in systems related to each other. The systems perspective requires us to think about feedback loops and dynamics of the whole system that a solution is part of. It is not enough to think about the production line because is embedded in bigger systems, like the labor force, politics, the ecosystem and the energy system. What is an improvement in a linear view may not be when we consider the wider system-effects. The circular economy takes inspiration in biology where metabolism is a key concept. This leads to a focus on flows of materials, energy and water in order to understand the metabolism of cities or countries. Superficially it could seem like agile development is similar in so far as here we also find a focus on flows, but that is deceiving. In agile the flow is only one of throughput, where something comes in, goes through the process and something else comes out, which marks the end of the scope of interest. The agile version is a linear focus on flows without any interest in systemic effects.
A circular form of development
The circular economy has made an impact in physical product design, city planning and management and production of physical goods. This shift taking place in the wider culture also has the potential to help us break out of the mental models that are a consequence of the production line of the industrial revolution and impact the way we develop tech products too. By moving from the linear mode to a circular mode of thinking we may harness many of the same beneficial effects that the circular economy does. Let us look at some examples of how that would change how we develop tech products.
| Agile development | Circular development |
| Basic metaphor is the production line, where throughput and production are in focus | The basic metaphor is one of metabolism where life and complex systems are in focus |
| Responsibility of development ends with deliverables deployed | Working solution is a shared responsibility |
| Promotes centralized view of production due to low cost and concentration of expertise | Promotes decentralized models where development takes place in the natural context of where it creates value |
| Standardization of end product, process and technologies | Standardization of components, protocols and interfaces |
| Optimizes for throughput | Optimizes service utility |
| Operates on service level agreement | Operates on fitness functions |
| Rewards hours spent | Rewards value produced |
| Build from new | Reuse reappropriate |
| Focus on transactions and interactions | Focus on system dynamics |
| Development based on business requirements and user acceptance test | Cocreation and dialogue based on vision and goals |
A circular mode of development will do many of the same things as is done in agile development and share some if not most processes and techniques, but the basic approach and mind set is radically different. Let us look at some of the more important possibilities.
Development as a complex system
In a traditional setting development consists in multiple modular activities that performance of which have little or no effect on each other. Business development and design is done in isolation from development, which is done in isolation from operations. Between them are handoffs that are always fraught with conflict and miscommunication.
In a circular perspective development is no longer just a sequence of atomic tasks that need to be completed by specialists but a fabric of interlocking areas that affect each other. By purposefully considering the entire fabric and its feedback loops development efforts optimize not just throughput but the utility of services produced and the interplay with the environment. For example, the programming languages used to develop a solution affects the employees working on it. It also affects recruitment of the right talent. If the language chosen is for example Scala because it is deemed superior for a given problem, this also affects Human Resources. Scala developers are among the highest paid globally, on average $77.159 per year and are difficult to find because they are fairly scarce. If HR was involved in this decision and the focus was on affordability and availability of talent, C++ might make sense with an average salary of $55.363 globally. From a financial perspective alone the $22.000 difference per developer per year going forward could also be important. Furthermore, the demography of C++ developers and Scala developers may be different, which affects work culture. Work culture can be an important factor to attract and retain talent. If the drive is towards a younger profile, this may go in the other direction with Scala more popular among the young.
What might look like an isolated technology choice in a linear mode of thought actually has wider impacts for the whole organization. Viewing decisions from the point of view as a complex system will help bringing these dynamics to light. In the example the decision could be optimal locally but not globally and may introduce unforeseen systemic effects.
Shared responsibility
Different functional areas such as sales, development and operations are commonly separate areas each with their own leadership and responsibility. Units of work pass between these different modules and changes responsibility along the way. But the responsibility stops at the boundary, which is the source of many political border wars.
Rather we should look for how to build a shared responsibility for not only the entire organization (business and IT) but also external entities like suppliers and collaborators. All should be responsible for the whole and not just their part. Today even in forward looking agile organizations the responsibility of development, infrastructure, business operations, sales and HR all belong in separate departments even if ideas such as DevOps is trying to break down the boundary between the two first.
Better ways to implement a shared responsibility must be developed. There are alternatives: colocation of the different functions for example. This is what makes it easy for start-ups smaller than a couple of 100 employees to move faster than the competition and often deliver vastly more value to the customer. It would make more sense if teams were organized around an objective rather than a type of work. If developers, support, marketing and sales were all part of the same team equally responsible for the same objective, work would more seamlessly align toward that.
The challenge of course is to find meaningful objectives that at the same time does not produce team sizes that are too big. One way is to architect the structure top down to make sure all the necessary objectives are represented. Another way could be to allow teams split up once it grows and divide the objective into sub-objectives. There is no easy answer, but the first step is to break down the default linear thinking that organizes responsibility around types of work rather than common objectives.
Decentralization
While agile development often works from the premise of decentral and empowered teams and work relatively independently, they still invariably belong to the technology function and are ultimately managed by the CIO or CTO. This brings with it some degree of centralization. If instead the creation of technology is just one aspect in a shared responsibility, it will allow decentralized multi-disciplinary teams to appear.
Agile methodologies try to make the connection to the business by inserting a part time representative from the business as the product owner, which often turn out to be an IT person anyway. Sometimes product managers encapsulate the business perspective but are mostly seen as something outside the development team. These are all ways around the implicit centralized mode of production.
Rather, having a truly cohesive multidisciplinary team means that there is no longer any need for a centralized mode of production. These teams would have all or most of the skills they need to fulfill their objectives. The skills may therefore vary greatly, but the decentral teams should be able to work on services in isolation using the tools and technologies that fit to maximize the success of their service. This will increase the agility of the entire organization.
The degree of decentralization can differ according to the context of the business. For some heavily regulated industries, decentralization makes less sense than other industries that are less regulated. Complex industries like nuclear power might similarly have strict requirements for centralizations on many parameters. In general, it can be said that complexity draws an ideal solution towards centralization but the general impetus should be towards decentralization. The focus should be on business services.
Standardization
To have a decentralized focus on services it becomes critical to focus on standards. This is nothing new of course. Standards exist and are deployed in multiple contexts. In this context there is a need for clear standards of components, protocols and how interfaces are built and maintained. A precondition for decentralization and local autonomy around a service is that its interfaces are well defined and standardized. This is frequently done through an interface agreement that specifies what consumers can expect from this service. When the interface agreement is in place, autonomous development of how to support it is possible. The standards however need to be more than just protocols and service level agreements, it also involves quality. As an example, let us imagine a bank providing a service for risk scoring of counterparties. It is not sufficient that we know the protocol, how fast the response time is and the logic of the service, we also want quality standards around accuracy, false positives and errors since these aspects are likely to affect other services like credit decisions and customer relationship management. The concept of standards and protocols are thus expanded compared to common services.
Some degree of standardization across the services are also necessary. In the case of web-services it would not make sense that some services used XML, others JSON and still others invented their own protocol. It would be similar to different organs in an organism using different blood types. You can choose only one blood type as an individual, but different blood types may work equally well. Similarly, there needs to be only one standard for service interface protocols.
Optimize utility
The focus on services delivered means that focus will be on the utility of that service to its consumers. It may seem self-evident but utility to consumers is not. To determine whether a service is useful you have to take the perspective of the consumers of the service. This is why user interviews, surveys, Net Promotor Score, focus groups etc. have been developed as techniques in product development. These are all ways to find out if a product or service is useful. This, however, is merely aimed at the end user, but a technology product usually rely on many other services. These should have a similar strong focus on whether they are useful. If we run a real estate company, the users will naturally be interested in the website and how it works. But an underlying service such as the price estimation engine will be only indirectly relevant to end users. It is not easily measured by existing methods mentioned above. The utility of the service to its consumers may be speed, accuracy, additional information or something else entirely. But whatever the utility is, this is what needs to be optimized.
Transition to fitness functions
The consequence is that we also need to rethink how to measure utility. The traditional service level agreement will not work for this way of working because it only relates to superficial features that may or may not be important like latency, uptime, service windows etc. Rather we need to focus on the fitness functions of the services as the relate to the utility of the service. The term fitness function is borrowed from evolutionary theory and designates a function that measures how close a potential solution is to solving a problem. In nature the problem is related to survival of a species. For example, the speed and agility of a spring buck is a part of a fitness function that determines its survival in the encounter with lions on the savannah.
In product development it is related to the utility of a service and thereby how it contributes to the overall success of the product or organization. An example could be how fast a website is ready to be used by the customer. If for example it is a social media service and it takes more than 2 seconds it is the equivalent of being caught by the lion. A service level agreement might specify a latency of 2 seconds but that is not the same as the functionality of the site being available. There could be many aspects involved, the browser type of the user, underlying services that take longer to load content etc. Thinking in terms of fitness functions makes it clear that it is a shared responsibility.
Rewarding value created
If teams are working on their services by optimizing the utility as measured by the fitness function, it seems strange that they should be rewarded by how much time they spend working. Another approach is to reward the value that their work produces or the fitness of the service. It is probably rarely possible to do this 100% since most people need some sort of predictability of remuneration for their work but then regarding value created can be worked into the reward structure in other ways such as bonuses calculated on performance based on the fitness function. There are already well-established ways to reward employees, the only change is that it should be based on the value being created.
Reuse and reappropriation
A key concept in circular economy is to reuse and reappropriate rather than buying new. When a new need arises, rather than starting to build it straight away, it should be investigated if other existing solutions could be used. Sometimes this requires a stretch of imagination, but it definitely requires knowledge of what exists. A solution for CRM could be used for case management, and a Service Management solution can work equally well for HR. By looking at what already exists, money and time can be saved. These solutions and the investments that have gone into them will be preserved. It is not trivial to develop something that is ready for production. Using something already in production thus also minimizes risk.
In biology this process is known as co-option, means a shift in the function of a specific trait and it was at the heart of Darwin’ theory of evolution. One example is feathers, which originally developed in order to regulate heat, but later was co-opted for flight. The same can be said of our human arms, which were originally legs but were coopted into holding and manipulating objects. These hands that are now typing are coopted from crude legs where the fingers originally were made only to keep balance now perform a magnificent array of functions. There is no reason why the co-option of existing systems could not provide the same effect.
The drive towards reuse also goes into how we design new solutions. Designing for reuse has already been a recurrent theme in development. This is the basis of service-oriented architecture, microservices, the use of libraries and object-oriented programming. It also works at higher levels. However, it requires a bit more reflection and abstract analysis. Developing a proper information architecture as a foundation is a precondition. Designing for reuse is not trivial and does take more time than just starting from one end and building what seems to be needed.
Focus on system dynamics
In traditional agile focus is on interactions as per the agile manifesto. Interactions are important and the agile manifesto’s focus on responding to change is important. Unfortunately, this is not sufficient in a complex system, where dynamics cannot be explained from individual dynamics. In a circular mode we want to focus on system dynamics particularly feedbacks and other system effects. Some of the most important effects of the dynamics of complex systems are delayed response, cascades, oscillations and instability. These will often appear puzzling and mysterious since the source is unknown and can’t be seen immediately from the interaction. The only way to try and tame a complex system is to focus on understanding the system. To do this one the most important is to identify and measure the feedback loops of the system.
Co-creation and dialogue
Another consequence is that teams will have to be interdisciplinary and work together based on co-creation and dialogue rather than through requirements gathering from business followed by development and finished with user acceptance testing. Teams will develop visions and set goals together that will guide development activities. It doesn’t mean, however, that everyone have to sit together always and only talk to each other. Different disciplines still need to work with similar people most of the time to hone their skills, but a significant portion of time must be dedicated to work together. Therefore startups are typically better and faster at adapting to new needs in the market, they naturally work in this mode, since they are so small that everyone works and talk together.
Perspectives
Most of these thoughts are not new and neither are the challenges. For example, Working in autonomous multidisciplinary teams is known from a matrix organization. The challenge is that specialized knowledge is not developed sufficiently. If five C++ developers work in five different autonomous teams, they will never learn from each other. This may lead to locally suboptimal solutions, but that is natural. We know this from biology too: humans have two kidneys but need only one, we do not need the appendicitis at all, but it is there. The spleen seems similarly superfluous. From a logical top-down design perspective there are some features that don’t make sense. However, the organism is designed by its ability to survive and thrive in an environment, which means that as long as theses locally suboptimal features do not interfere with that superior goal it is okay.
Similarly, there is no simple solution for magically fixing worker compensation. It is well known that the incentive structure is very hard and may have unintended consequences. This does not mean that we can’t be inspired to think about it in a more circular way in some cases. Some jobs will continue to be standard compensation per hours worked but we might try to make them more incentive driven and align the work being done with the value we want to create. Being physically at work sitting in front of the computer rarely produces value by itself. A circular perspective may help us to focus work on what does.
Towards a circular development
The agile revolution was a welcome improvement over the plan based or waterfall methodologies that were dominating It development at the time. Unfortunately it copied a mentality that was inherited from the industrialisation and modernity of the previous century. It is time to evaluate whether that way of thinking is still the best way to get work done. Experience in other areas of society has questioned whether the linear and modular thinking with a focus on throughput is optimal and increasingly a circular approach is adopted. This has not yet had any significant effect on IT development methodology. However, that should change in order to reach the next stage. If we can imagine a radically different way of working as is outlined here we can also change. Agile development has not made all the problems of development magically go away neither will a circular approach, but agile has reached the limit of where it continues to provide improvements regardless of the flavor. It is time to try another approach in order to make sure that we adapt work and technology to the 21st century rather than being stuck in a mindset that only made sense in the 20th.
English 
Danish