Software that knows: why now

Part 2 of the series “Software that knows”.

Recap

The hypothesis: capture an organisation’s knowledge, make it formal and explicit, and put it at the centre of its business applications.

Prior work

The idea isn’t new. Expert systems in the 1980s and model-driven development in the 2000s aimed at something similar.

Both stalled, in large part because formalising knowledge was slow and expensive.
It needed specialists, such as knowledge engineers and ontologists, to interview experts and model what they knew.
Expert-systems research had a name for it: the knowledge acquisition bottleneck.

What changed

LLMs and agents lower the cost of the first draft.

Turning documents and conversations into structured knowledge is now much faster and cheaper.
For example, a first draft of a small ontology can be extracted from an internal SOP in one working session, without a dedicated ontologist.
Even with review, the total effort is a fraction of what it used to be.

Two bars on a time axis. Before LLMs: one long bar, specialists interview experts and model by hand. With LLMs: a much shorter bar, split into a thin LLM draft segment covering about 80% of the content in about 20% of the time, and a wider human review segment covering the last 20% in about 80% of the time. Illustrative, not to scale.

What didn’t change

The draft isn’t the result. It follows a familiar 80/20 split: LLM extraction covers roughly 80% of the content in 20% of the time. Review and acceptance by the people who own the knowledge take the other 80% of the time, for the last 20%: exceptions, edge cases and unwritten rules.
Review is needed for adoption. People tend to trust and use rules they helped shape.
Review is needed for compliance. Formal knowledge that drives business applications has to be checked before it’s relied on.
Accountability stays with people. LLMs are tools, not owners: an LLM can’t answer for a decision, bear its consequences or be held liable. Regulation points the same way; the EU AI Act, for example, requires human oversight of high-risk AI systems.

How we’d know it’s wrong

Human review costs more than LLM extraction saves.
Extraction quality doesn’t hold beyond small, well-written documents.

The ideas and arguments in this article are my own. An AI assistant (Claude) helped with drafting, editing and fact-checking.