How can we track the GenAI calls made by an application

AI
1

Basically consider we have application using Genai connectors while we would like to understand below details -

  1. Average time for each call over a period
  2. Differentiation of which call consumed the highest token
2

@Aditya_Sethia - Explore the properties available in Data-GenerativeAI-Provider class as a starting point to capture all the LLM call metrics to derive from.

pyGenAIServiceOverhead - Gen AI Connect time spent on service layer excluding model processing duration in seconds.

pyGenAIServiceTotalTime - Gen AI Connect time spent includes model processing duration in seconds.