Agent Observability

AI
1

Pega GenAI Agent Auditing and Monitoring

Pega GenAI Agent executions are tracked OOTB to ensure full transparency, monitoring, and auditability of AI-driven interactions.

Architecture Highlights

  • Persistence: Every GenAI Agent execution generates an instance of the Pega-Autopilot-Conversation class (a work-derived class).

  • Case ID: These instances use the prefix PXCONV-.

  • Context Management: This case acts as the “memory” for the agent, storing the conversation history and providing the necessary context for multi-turn interactions.

  • Linking: * Assistant ID: Records the specific Agent rule instance used.

    • pyContextID: Maps the conversation back to the caller’s instance key (e.g., a specific Case ID), allowing you to see which case triggered the agent.
      image
      image1920×990 287 KB

Monitoring & Tracing

  • AI Tracer: This specialized tool captures the granular “Reasoning and Acting” (ReAct) steps. It allows developers to see how the agent interpreted a prompt, which tools it chose to call, and the raw data exchanged with the LLM.

  • Tracing Workflow Agents: While the AI Tracer is primarily launched from standalone agent sessions (like landing page widgets), the PXCONV- case remains the primary audit source when agents are executed via the Run Agent smart shape within a workflow.

  • Execution Visibility: The execution path and tool data context are stored within the conversation instance, providing a permanent record of how the AI arrived at a specific conclusion or action.

image
image1024×692 103 KB

2

Thanks for sharing.

Does the AI Tracer read directly from the PXCONV- conversation instance, or does it use a separate data source? Asking because if they share the same underlying store, then PXCONV- already contains the full ReAct trace (tool calls, raw LLM payloads) — which would make it sufficient for production audits without any additional instrumentation.

3

Hi @Kamil_Janeczek , yes the AI Tracer is indeed sourcing info directly from the Conversation case (stored as raw JSON data)

4

Hey @Kamil_Janeczek Yes as Cedric mentioned it’s all in the JSON string of the conversation case.