We have a table with few thousand records in it, we want read each record from that table and format the data and move the formatted data to a file. All the rows from the table must be added to the single file.
Approach tried:
Issues faced:
Please help how to club all record together in a single file using data flows.
I am also facing the similar kind of issue.. Multiple files are generating in multiple nodes..
Can you please send me the configuration on how you are fetching and transforming the data. Ideally the data flow and the data set should be in the same class.
When the data flow executes it will fetch single record and put it on the primary page and process that primary record. Basing on the mapping on the file data set 2nd tab it will create the file. So likewise it will process all the records at once and create single file only.
Hi @RuchaS16869320,
Use this article https://docs-previous.pega.com/decision-management/87/managing-data-flow-runs?
Regards,
Sai G
@RuchaS16869320 Updated the mapping tab, still its creating files on different nodes based on the thread. The requirement is to create a file on single node
@RuchaS16869320this is because the partitioning key on the source data set will be used to partition the data and distribute the load across the data flow nodes to provide concurrent execution.
If you want files to only be written to a single node, you need to configure the source data set with no or a static partitioning key so there will only be a single partition. However, the way File data set works is that data is still flushed intermediately, and this would result in multiple files on the system