No modern technology has added technical debt faster than AI. Technical debt is the future cost of reworking created by a focus on expediency rather than long-term design robustness and the implementation of AI has been expedient. Developers have often been able to pick and choose what frameworks, languages and platforms to use if they could just produce some AI. This has resulted in a situation where a large technical debt has been created and organizations are faced with a significant future debt restructuring.
Under pressure
Experiments, prototypes, MVPs, POCs, and pilots have been built in a wave of innovation. The name of the game for a POC is speed and agility, so whatever works for the developers is good. That makes for an eclectic mix of technologies determined more by developers’ preference and familiarity than with accepted standards used in other solutions.
Contrast this with how enterprise development is usually done. Usually it has to proceed along predetermined guidelines, platforms and patterns. Only approved technologies are used and the approval itself takes time and will usually only be done using mature technologies. The reason for this is to ensure that the company can sustain and support the product in the long run.
That would be fine if the POCs were matured and transformed to compliant enterprise solutions as they were moved to production. To follow normal procedure would unfortunately often mean rewriting the code, changing the database and other middleware, which can quickly become a development effort that eclipses that of the POC. Due to the focus on AI and the pressure to show results quickly that rarely happens. Either the POC remains in beta or is rolled out prematurely more or less as is because there is an executive pressure to deploy the AI solution to production. The first course of action creates waste and the latter creates technical debt.
How AI exacerbates technical debt
Consequently, a significant amount of technical debt has been created. Technical debt is nothing new, but the dynamic is only exacerbated by the unique character of AI technologies:
The need for model updates. It frequently requires continuous retraining to keep it from model drift. Newer and more efficient models appear quickly and may require a rebuilding of the entire system or significant parts of it.
Heightened open-source dependency. AI solutions use open-source software extensively, which will add to the legacy dynamic because not all open source projects remain successful. Some stop being maintained and fall into disuse.
Developer churn. AI developers are hot property and may soon find better work elsewhere. Since documentation is rarely a priority for POCs the code will not be well understood and needs to be either documented properly or rewritten.
Increased regulatory attention. Since AI is a popular topic in society, politicians have seen a need to engage with it. That often results in new regulations. That may prompt a review of the solution to make sure that the organization is compliant. If not, the code needs to be reworked.
Malicious intent. AI is also a target for criminals. AI systems sometimes behave in surprising ways, and the frequent use of open source software, leaves many new vectors of attack open. To defend against this requires a lot of reworking of the solution.
Technical debt restructuring
Due to these factors AI is quickly becoming a major technical debt burden if not handled properly. If these things are not addressed, the AI marvels of today will be a parallel IT estate equivalent of the mainframe of tomorrow. The first step is to recognize the problem and catalog the nature and extent of the debt. After that organizations need to have adequate standards and procedures to run AI securely in full production capacity. Once this basis has been established plans to migrate technical debt can be made. Mature organizations already have similar in place for mainframe and similar legacy technologies. This tool box will work equally well with AI. But it’s time to start acting now before use becomes too embedded in the business as usual and integrated with other solutions. There is thus still a window to act before the debt restructuring becomes critically expensive.
Danish 
English