External trained predictive model behavior

General
, ,
1

Hello,

We have created predictive model out of externally trained model(through pmml file), came across couple of questions on predictive model behavior

Configured the predictive model in strategy,

  1. Observed that propensity is getting varied even for the same set of predictor values(unit tested predictor propensity, preview propensity value from strategy. is this expected behavior? could you pls explain little here?

  2. there are no monitor data after data passes through this predictive model. Do we have to configure anything to display the data in **Monitor Tab,**or its expected that it will not display the monitor data for externally trained models. Pls advise.

  3. This question is slightly bit out of Pega, the external model is getting trained in data bricks side and data science team claims that they can provide pmml file per model. we have 80+ models that should get trained as part of our business process and that has to produce propensities in Pega

[if you could answer]Is there anyway that data bricks team can accommodate all models in single PMML file (they use pyspark to train model) , as we cant import 80+ models in strategy and conditionally execute ?

2

@PonnurangamN1990

  1. The model should return the same output with the same inputs for the predictors. You can also test from Prediction Studio, run the predictive model from there

  2. if you have outcomes to the predictive models you can capture these as pyOutcome and pass to pxAdaptiveAnalytics. I’ll try to look up some more detailed instructions for you, but it works in the same way that outcomes are captured for adaptive models.

  3. This would depend on the use case. If the aim is to replace the adaptive model propensities in the Next-Best-Action Designer framework, I would warn against that because adaptive models have great benefits in their granularity in the number of actions/treatments they handle, the ability to use real-time contextual data as predictors, the handling of new actions/treatments, maintenance etc. You can always contact your Pega account executive to get more guidance on this.

3

@Ivar_Siccama For more detailed documentation on 2. see Pegasystems Documentation

4

@Ivar_Siccama i did quick test for point 1 : ran the model for below set of data and captured the pypropensity which is still different than one generated from strategy.

Model unit test.png
Model unit test.png1152×817 30.5 KB

I have reviewed the same through strategy and write to temp table where i could see different propensity value for same set of data.

Any lead for this behavior would be highly appreciated. Thanks!!

5

Created ticket INC-B7617 to triage.

6

@PonnurangamN1990 @Ivar_Siccama

I checked INC-B7617 and can see that our GCS support team helped resolve the issue.

The closing description of the ticket reads as follows:

Issue primary reason description:

Client have property with same name causing the issue that the value of propensity was different with prediction studio and from data flow

Explanation description:

Changed the data flow that need to be used the same properties which we are using for the prediction as well helped in resolving the issue