Imagine getting into an Uber.
The driver says:
“Don’t worry about where we’re going. I can’t explain the route, I can’t show you the map, and I can’t tell you why I’m making any of these turns. Just trust me.”
Most of us would get out immediately.
Yet that’s exactly how many organizations are approaching AI today.
AI recommends:
- Which customer should get a loan
- Which transaction looks suspicious
- Which claim should be escalated
- Which case should be prioritized
But when someone asks “Why did the AI make that decision?” the answer is often:
“We’re not really sure.” and that my dear reader is The Black Box Problem.
As AI becomes more powerful, explainability becomes more important…not less.
Because the real risk isn’t that AI makes a bad decision.
The real risk is that nobody understands how the decision was made, who approved it, what data was used, or how to challenge it.
This is where I think many organizations are focusing on the wrong question.
They’re asking:
“How do we get AI into our processes?”
Instead, they should be asking: “How do we put governance around AI?”
One of the ideas Alan Trefler, Founder and CEO of Pega, has emphasized is that AI should operate within a framework of business rules, policies, and outcomes, not as an uncontrolled system making decisions on its own.
That’s why I find Pega’s approach compelling.
Rather than treating AI as a standalone tool, Pega embeds it into workflows where organizations can:
Understand the context of a decision
Apply business rules and guardrails
Keep humans in the loop when needed
Maintain a complete audit trail
Monitor outcomes over time
AI shouldn’t be a black box. It should be a glass box.
Powerful enough to transform work, but transparent enough to earn trust.
Because in the age of AI, trust may become the most valuable capability of all.
