One of the big learning curves with AI is the generative nature of the technology. LLMs (Large Language Models) generate new artifacts that by design contain randomness. The same AI prompt does not always create the same output artifact. And this is at the core of the technology. You cannot make the uncertainty go away.
AI is different from traditional algorithms for information handling. An algorithm performs the same computation and returns the same result from the same input. That repeatability has been a foundation of software engineering and the repeatability has served us well.
We are so used to deterministic behaviour in both software and the creation of software that LLM-introduced variability is quite a whammy to take in.
In 2025 (when this is written) a lot of the variability is not benign variation. LLMs can spit out lies, hallucinations and half-truths. It is quite logical to consider LLMs unreliable.
Certainty was never there in product work - Roger L Martin's truth about customers
We used to be able to control what software we create and how it exactly worked. We used to be able to control budgets and processes.
But we were never able to control everything. The world around us was always changing. What we thought was right turned out to be not right. We were smart and became agile.
Yet, we cling to the want for control of the future. We craft plans and require compliance of ourselves in the future.
I learned from Roger L Martin my favourite way to prove that future is beyond our control. It goes…
"We can control budgets and we can control our plans. But we cannot control customers. We cannot force the customers to buy our products. They have their free will so they will do whatever they like with their money. Costs can be controlled. Revenues cannot be." (paraphrased; not direct quote)
Then you realise one of the sources of variation and complexity is utterly outside your control. It would simply be absurd to try strong-arm the customer (I know such stories exist though).
The company creating software was of course not fully controllable. Humans and companies are deeply complex and unpredictable. We just so wanted to be in control and thought that was hallmark of professionalism. To a degree yes. But we've never fully been in control.
What I like about Roger L Martin's way is that it makes you realise there is a lot of variation from a source that you simply cannot control. You don't need to argue whether or not there is complexity inside the company. All of us can agree that because customers are fickle then we will not be able to make the uncertainty go away!
LLMs bring variability and unpredictability front and centre in product creation
I predict we will turn the dial much more into explorative and adaptive development. LLMs are already able to create many more solution options and product ideas that we humans can think of.
Large sweeping architecture changes change from 12 months lead time to 1 month lead time. Agents handle complex slices of product development. I posit we end up revisiting many of the fundamentals of the value chain in larger companies.
But most importantly, there will be uncontrollable variability in the output of our work. Our LLMs will create things that we did not anticipate. We initially will not like it. But that very variability contains accidents for the good. There will be mistakes and hallucinations. There will be gems too.
Variability (as many of things in life) is a two-sided sword.
Predictions
So, we will hate variability but the upside is such huge that the industry moves along the adoption curve. But the quality problem remains. What is right and what is hallucinations?
I predict we will keep moving toward data-informed feature and product iteration. We measure everything. We keenly monitor the user journeys playing out. We monitor more the health of our systems. We do not try decide "inside the company" what is right and wrong. We deliberately ship options and see what thrives on the market.
The linear "big plan", "big product", and "big launch" model keeps on getting more and more risky. Again, we replace this gradual roll-out and extreme speed.
But most of all, smart companies let go of the requirement of knowing exactly a priori. It is not about compliance to plan. It is not about meeting the target. It is about shipping the right product. It is about taking the iterations and care dampen down the wrongs. It is about stumbling upon the things that make customers happy.
The new goal is creating even more options. The goal is to find the gems in the haystack.
Original article published on LinkedIn.