We have a requirement where 500-1000 records are to be uploaded on a case(which also will have multiple child cases with split records out of 500-1000 from parent) and should be reportable. We tried to store the records in pyworkpage in index but the load time for the case was very high. I had 2 approach in mind as mentioned below
1 - Use Index table - Load the records outside of pyworkpage so that the records are not stored on the clipboard after the transaction is committed. And when you need to display/edit the records again load the records in pre processing on a temp page outside of pyworkpage and after editing commit the transaction.
Advantage - Reportable solution with PEGA’s OOTB index processing. No separate DB table required in this case
Challenge - In Constellation we could not find a way to achieve this because in view you cannot define pages or source as a page. Can we let the pyworkpage include the index page and then after the page is submitted in post processing using a page-remove remove the index page?
2 - Use separate data table - Create a new data table to store the bulk records. use the same approach as mentioned above.
Advantage - Flexiblity wrt implementation and reportable as well
Challenge - Try to mimic what Index processing does will be risky. Also in future if index creation is required then there will be 2 tables having duplicate data - 1 - this new data table and 2 - index table.
Appreciate if anyone can comment on this or suggest a better approach.
Hi Kamil, Thanks for your response. Please see the doc attached with issue - even when we have set the view as editable, on screen at run time it is not allowing us to edit entries. can you let us know if we are missing any configuration?
Index tables are not intended for user-managed CRUD operations. My recommendation would be to use a dedicated Data Object/Data Table rather than an Index table.
While CRUD operations on an Index table are technically possible, Index tables are primarily intended for reporting and search optimization, not as a transactional data store. They are derived data maintained by Pega through Declare Index processing, so using them as the system of record can introduce maintenance, synchronization, and consistency risks over time.
I don’t see the future indexing concern as a significant challenge with Option 2. If the uploaded records are persisted in a dedicated Data Table using a flat relational structure (similar to an Index table), that table can serve as both the system of record and the reporting source.
There are several configurations need to be done to make it working.
But even if you make it working, as Ramesh said, index table are not meant for manual CRUD actions. The whole purpose of index is that system is managing that.
So you are either fine with keeping it on a case and exposing for reporting with index or option two is better.
@VirendraSawant can we understand where the performance / UX concern lies?
Is it the physical upload that take too long? [when user clicks submit after uploading a file, the screen take a long time to confirm its all done]
Is it the physical opening of the embedded data table? or data reference table? (can you confirm which you are using). [when user clicks on the assignment, the form takes a long time to open because all the rows are being loaded]
I had a similar request recently, it was more around area 2, for embedded data. We solved this by splitting up the screen, so we didn’t try to load 1,000 records.
I can’t share that implementation, but we are talking about embedded data (multiple records) using an action to edit. Like:
Rather than showing all 1000, we showed this list broken up in a logical sequence, so in the above image if you edited “Arts and Culture” then it would load another embedded list inside that object, that had say 200 instances inside that (say Beneficiaries in this example).
The user would edit those 200, then move on to the next logical grouping.
Could you break it up like this somehow? From a UX perspective this is much better too, providing a long list of 1,000 items in an excel like capability is never great, lots of scrolling up and down trying to find and work on what I need. One wrong scroll and you lose your place…
Create a separate Data Table to store the 500 to 1000 records, instead of using the pyWorkPage or an Index table, since Index tables are meant only for search and reporting, not user editing. Keep the records on a temporary page outside the pyWorkPage so the case loads quickly. When a user opens the case, load the records from the Data Table onto this temp page for viewing or editing, then save the changes back to the table once the case is submitted. This same table can also be used directly as the reporting source, so no extra index table is needed.
