What are the guidelines on storing embedded page (or page list) data in case vs. storing this data as separate instances and using auto-populate?
Some of the main considerations here are as follows:
- Is the “related data” of interest outside of the context of the case*?
- Are the “related data” types themselves complex types (composed of many other classes of data)?
- Are the volumes of the “related data” for a single instance of the case* “high”?
Consider externalizing the embedded data into distinct concrete instances when any of these are true, and even more so when multiple or all are true.
“High volume” is of course subjective, is dependent on many factors, and is informed by your client’s Non-Functional Requirements. Where a higher volume of embedded data starts having a noticeable impacting the time taken to open or save the case*, consider reverting to this approach.
Unit Test Cases - built in PegaUnit - can include a “Response Time” assertion on a Data Page that uses the Lookup Data Source to open your case*. Create Unit Test Cases like this - at the start of your project - with a volume of “related data” that is realistic and representative of Production. If these PegaUnit tests start failing because the response time exceeds the required threshold, you have crossed the line and should transition to managing the related data as distinct concrete instances.
That you can anticipate part of your case* data model that may be susceptible to this issue may, in itself, be a good enough reason to take this approach from the start.
- Whilst the above examples refer to a “case” as a typical scenario for this; this approach is equally applicable to Data instances that have embedded data that meets some/all of the criteria listed above.
Discussion on this topic was sought from the LSA Data Excellence (Pega 8.4) webinar conducted in July 2020. The webinar and its full set of discussions that arose from it are available at LSA Data Excellence: Webinar, Questions & Answers.
@BraamCLSA - I have a question regarding the topic of using embedded case data vs storing this data outside case context and then referring to them through a data page. If we follow the second approach (i.e. storing the data outside the case context), wouldn’t it impact the way we do field level audit for cases? I mean FLA works with case types (it works with data instances as well, but it’s not the same as with a case type) and its embedded properties. So, if we store the data external and source through data pages, wouldn’t we need to build the FLA for those data attributes separately instead of leveraging the out of the box case type audit? What is the recommendation around this topic?
@SuditSNG - that is a great question. I have not assessed the impact of this pattern on Field Level Auditing, but the research I have just done* suggests that auto-populate properties (which we now call either “Data Reference” or “Query” properties) are not considered by FLA.
If you are on Pega 8.4 or newer, FLA now stores the audit items as records of a distinct class (look for classes observing your application’s class names, but whose name starts with FLAudit-). While FLA is nicely configurable in the Case Designer for case types, the underlying rules that get created* on the class of instance being audited are:
- Declare Trigger that runs the activity pxFieldLevelAuditing, and
- Data Transform named pyFieldLevelAuditing which holds the properties subject to audit
In principle, you could replicate FLA on the data class by following the same conventions to at least have the changes made to key fields on your ‘external-to-the-case’ data instances stored as FLA records specific to that data type.
You could then perhaps implement a Declare Trigger on your new FLAudit-…-Data-… class to store a clone of the current FLA record back into the FLA records for the original case, as the case ID would have to be one of the properties stored with your external-to-the-case Data record to allow the Data Page to find them. If this Declare Trigger could correctly emulate how FLA records for “embedded data” case properties were stored, the visualisations of FLA for that case in the portal could show the changes made to external-to-the-case data in the context of the case.
There is some cost of ownership here in the extra engineering to push back the Data FLAs as Case FLAs, checking at update-time that Pega continues to manage embedded data FLAs as it previously has, and that relevant Data FLAs are stored once for their data record and once again for their corresponding case.
- Research and rule references are as observed on Pega 8.7. Verify on your version of Pega if different.
@BraamCLSA - Thanks a lot for looking into it and for your detailed explanation. As you mentioned there will be some cost of ownership of such an approach and also we will have to see if the same setup in a data class is able to replicate the FLA structure of a case type.
One other option that I was thinking of was to use the old way of auditing data instances (using pyTrackSecurityChanges declare trigger), however, then the FLA structure and display would be completely different from the case type.
@BraamCLSA Hi Braam the lsa course in pega academy has a enrollment and todo design pattern ,where it is telling to consider locking as a parameter when considering if we need to have embedded data or externalize the data . Can you give some light on that.
Also if we externalize the data we need to implement security for that instance .
Would you please suggest on those, what approach should we take