We need to capture the system and token usage details when connecting to the Pega AI Agent rule. Currently, there is a clipboard page that provides a snapshot of LLM token consumption and processing insights when using the Generative AI rule (Generative Insights). We are looking to identify a similar mechanism or page that can provide the same visibility and monitoring when executing the AI Agent rule.
Thanks
Dr Chandan Pan,PhD in AI
I hope the instances of this class (Pega-Autopilot-Conversation) will provide you more insights on the turnaround time for executing the tools during the conversation agent
Dear Ramesh,
Thank you for your quick insights and guidance.
In our use case, we are leveraging the AI Agent rule within the case flow to automate decision outcomes. As part of our governance, we would like to analyze certain execution KPIs like token usage, execution time etc.
We’ve observed that similar information is available through the AI Tracer feature in the Conversation Agent, as well as via the GenAI Connect rule through the out-of-the-box GenAIResponse page. While debugging with the standard Tracer Tool, we noticed that this page is also created during the AI Agent rule execution, but it appears to be removed shortly afterward.
Could you please advise if there is a recommended or persistent way to access these execution details/or this Page for AI Agent runs?
Thanks again for your support.
Thanks
Dr Chandan Pan,PhD in AI
Hi,
As Ramesh mentioned, every Pega agent (conversational or step agents) will trace activity as part of a Pega-Autopilot-Conversation case. The comprehensive trace including granular metrics for each signal/action (tool, duration, nbr of tokens, retry count, model used, …) is stored in pyStateJSON case property, as a JSON message.
The AI Tracer (think of it as the Constellation case view of the PXCONV- case) is consuming this raw data to surface it in a more end-user friendly way. It uses Data Page D_pxAgentMetrics to transform it into a Pega object structure.