We have configured Data flow to read files in PEGA cloud repository and load the tables in application.
currently we incrementally move files to cloud repository, and in source Data-set we configured the file path as something like /*.csv
Everytime the Data flow is executed, the above setup reads all the file in the repo file path, we need to restrict the process to read only the new files loaded in the repository.
If you’re using a dataset to extract then there’s no restriction to read only new files. So you can try one thing on the data flow, extract the dataset then add a filter to filter out based on pxUpdateDateTime and then set the filtered file in the file path csv.
Or else you can extract via a report definition in the first place and then set in the csv file.