I was trying to figure out how to put a new adaptive model in shadow mode. For example;
Let’s say we have an adaptive model in production (it could be an OOTB model or a custom one) now I create a new adaptive model maybe with different predictors or different settings. I want to put this new model to production as a shadow.
On the training, I saw an example to put one predictive model as a shadow but not an adaptive model. And when I try to select a new adaptive model, the list is not showing me any adaptive models (I’m doing this operation by clicking the “replace model” option below).
Maybe I’m missing something, is anyone has a similar experience on this topic?
You simply run both the “shadow” and the actual model in sequence. This ensures both are executed, so both learn, but the propensity from the shadow model is overruled by the actual one.
If you’re using Predictions, you accomplish this by customizing the prediction strategy
@recepanilaydemir , Currently, there is no provision to include a new Adaptive model in shadow mode. I hope you are using the “Replace model” option in Prediction.
You can still add an adaptive model manually in the Strategy.
Can we get more details on the use case to understand more?
@recepanilaydemir Just to be clear - to implement this more advanced pattern, even while you can not currently do this from the Predictions UI or API, like I said in my earlier answer, you can place your shadow models in the decision strategy that is behind that prediction. To monitor the shadow model you would look at the model in the models tab of prediction studio.
I was trying to explore the new capabilities with 8.6. The main requirement is to either use some old adaptive models from before the CDH upgrade or test some adaptive models’ settings. This might be a good addition for future releases because I believe the creation of the predictive models requires some technical knowledge like data science, but for the adaptive models with Pega, anyone can do it. Shadowing an adaptive model with an adaptive model option could be a good feature.
