We have an option to define(change) ADM Model update frequency. The challenge I see here is that this ADM rule will be used for different propositions which will have different ADM models.
Now lets say if we have 3 propositions and recorded responses for each prop are as below.
Prop 1 —> 5K (Per Hour)
Prop 2 —> 500 (Per Hour)
Prop 3 —> 50 (Per Hour)
Required, we want our models to be updated every 2 hours. If I define, ADM Model frequency to 10K, this will work for Prop 1.
If I define ADM Model frequency to 1000, it will work for Prop 2 but for Prop 1 it will update 10 times/2 Hour. And similar for Prop 3.
How do we define a value to fit our requirement? Our recorded responses are different for every proposition. We do not have time based learning concept so we want to leverage this learning frequency to build our requirements.
That is a very good question, and unfortunately there is no easy way to configure this setting and update each model instance at the same regular interval.
However, it is not extremely important that your models update at exactly the same interval. Models mature relatively quickly, and especially if there is only a very small batch to process (e.g. 100 records) it is not expected to make a big change to the resulting model learning. This is one reason why the threshold is configurable per response count with a default that is on the higher end (5000) compared to your example.
The leading guideline when configuring this value is that your model should generally update at least a couple of times per day, around 2 hours as in your example is a common target. With the important caveat that if you have a relatively low response rate, frequent updates are not that important. Which means your main focus is your high response rate models to get them in the right ball park, and especially make sure that they do not update too often (as in every few minutes). Then your low throughput models get less updates, but that is OK as they also have less to process.
Hope this helps, and do let me know if you have more questions.
@vandt Just have one question on the slow learning models. Lets say we have four propositions Prop 1, 2, 3, 4.
On a hourly basis we are getting 100 responses for Prop 1, 2,3 and only 10 responses for Prop 4. (these are just examples)
So now, if we have only 10 responses per hour. That means on a daily basis our recorded response count for prop 4 is around 240-250. If I give model frequency as 500, will the recorded responses of prop 4 will be added to next day(250+250 = To trigger model update)? or will it start from 0 again? If this is the case our low response models wont learn anytime.
@vandt Thanks for the response. That means if the response count is 240 in next 12 hours, it will be updated irrespective of the count? My model update frequency is set to 5000.