Stop fighting with your LLM - Why hallucinations are a good thing

AI
1

Figting your LLM
Figting your LLM1376×768 291 KB

Hallucinations are a good thing.

There, I said it.

For the past few years, the AI industry has treated hallucinations like a software defect that needs to be engineered out of existence. Every new model promises fewer hallucinations. Every benchmark celebrates lower error rates. Every vendor claims they’ve found a way to make AI more trustworthy. And yet, the very thing we’re trying so hard to eliminate might be the source of AI’s greatest value.

For centuries, we’ve embraced the idea that two heads are better than one. Not because the second person is always right, but because they often see something the first person missed. Two people approaching a problem together can produce a better outcome than either person working alone. Each person brings different experiences, perspectives, and ways of thinking to the table. Whoever said the other person must be, well an actual person?

I use conversational AI agents every day. I bring the initial idea and then let the AI refine it and challenge my assumptions. Hallucinations are often where creativity lives. The final result is something better than I could have created myself and way better than if I would have just asked AI to do it for me. The “creativity” is a product of a conversation, the two heads theory.

The same capability that occasionally produces a bad answer can also produce a breakthrough idea. If you remove every possibility of being wrong, you may also remove every possibility of being surprising. The trick is knowing when to be creative and when to be consistent. During design, creativity has enormous value. We are searching for possibilities, challenging assumptions, and imagining solutions that do not yet exist. In those moments, an LLM’s ability to make unexpected connections (i.e. hallucinations) is often its greatest strength.

Once an idea becomes real the objective changes. At that point, creativity takes a back seat to consistency. At this level, decisions need to be not only explainable, but repeatable. AI used at runtime needs to be treated differently in terms of guardrails and control. If ten people submit the same request, they should all receive the same outcome. This can be accomplished by using repeatable workflows that define how work progresses from start to finish. Then layer in Agent guardrails that define what the AI can and cannot do along with deterministic business rules that define when AI participates and when it doesn’t.

The question isn’t whether AI should be creative or deterministic. The answer is both. The real challenge is knowing when hallucinations are beneficial and when they create risk.

2

That’s a good perspective! The real challenge for me is that hallucinations are easiest to spot when working within my own expertise. At the edge of my knowledge of a subject, they’re much harder to distinguish from genuine insight. That’s where AI is most useful but also where we need to be really careful.

3

Great point Paul, and this is the foundation behind human-in-the-loop. You still need SMEs to validate the information is correct and a feedback loop.

4

Good framing, the design-time vs runtime split is the part that matters most. One nuance I’d add: creativity and hallucination aren’t quite the same thing, and I think it’s worth teasing them apart. Creativity is the model connecting ideas that actually exist. Hallucination is stating something that doesn’t exist with the same confidence. I’d call the first a feature and the second still a bug, just one whose cost changes with context: in design time you catch it, fix it, and move on; at runtime it’s a decision or a wrong record submitted.

Which ties back to your point about guardrails: at runtime what I keep running into is that the agent has no memory of what happened, doesn’t know the business rules, and leaves no auditable trail. So alongside the guardrails, I’d add a harness: a workflow that defines what happens next, deterministic rules for when AI participates, and a log trace to reconstruct the decision. Fully agree it’s not creativity vs determinism, it’s both, the model brings the spark, the platform brings the state.