This is a well-known LLM non-determinism challenge in agentic workflows.
You’re not doing anything wrong conceptually.
LLMs (even with the same prompt) don’t always call the same tools on every invocation. Tool selection depends on the model’s probabilistic output. Without strong prompt engineering or deterministic constraints, you’ll see inconsistency (1–2 out of 10).
Either use deterministic approach or enhance Prompt Engineering.
Improvements using Prompt Engineering:
-
Strengthen the tool description. The action tool description is what the LLM uses to decide whether to call it. Make it unambiguous.
-
Add explicit instruction in the agent prompt. In the agent’s system/instruction prompt, add an explicit step
Improvements using Determinism:
- Consider a deterministic post-agent step. For production patterns, rather than relying on the agent to self-trigger the automation, have the workflow read the agent’s decision output after the agent step completes and use a standard Pega decision shape/automation shape to trigger the case update. This makes the action 100% reliable regardless of LLM tool-calling behavior.
Best Practices and Accuracy/Reliability Requirements commonly point towards favoring a deterministic approach.