In Gen AI Connect, Gen AI Coach, and AI Agent rules, we can configure the LLM model to be used. However, what is the recommended approach to ensure a seamless transition when an LLM version is deprecated and a newer version becomes available? Currently, updating to a new model requires manual changes across these rules. If we use the Pega default model, will Pega automatically handle model upgrades and deprecations behind the scenes, or would manual intervention still be required to switch to a newer LLM version?
I would avoid situation when model is automatically update. You want to have under control. Changes between the models, versions, can be very distinct. So, you want to test your agentic flow once the old model is deprecated. So, you can choose the best next one. And for that you need to have a set of evals, I mean tests that you will run across your processes with different prompts, with different models. And then you compare which one is the best. And on this basis you will choose the model.
@KamilJaneczek Thanks for the reply. Just to confirm my understanding, does this mean we need to manually update the Gen AI Connect, Gen AI Coach, and AI Agent rules whenever the configured model is deprecated and a replacement model is released?
If a GenAI capability is already deployed to production, would the process then be to evaluate and test the newer model, update the relevant rules, and redeploy the changes? If so, is this ongoing model lifecycle management considered an expected operational overhead when working with LLM-based applications?
This is not what I intended to say. My point was that you want to have a control check and once you need to update the model, you make sure you have your evals ready to run them and check with golden sets.
You don’t want your models being silently update without noticing.