Can we process multiple files by using file listener

General
, ,
1

Can we process multiple files at a time by by using file listener. multiple as in 50 files at a time.

thanks in advance

2

@VishantC

No in Pega a file listene processes files one at a time by design. File listeners are configured to monitor a specified location or repository and pick up files sequentially for processing. While it can detect multiple files in the folder, it processes them individually in the order they are found in the file path.

3

Hi @VishantC,

Yes, in Pega, a File Listener can process multiple files,

  • Handling 50 Files:
    • By default, a File Listener processes files sequentially as they’re picked up from the directory. To handle 50 files “at a time,” you have two options:
      • Batch Processing: Queue the files and process them in a controlled batch.
      • Parallel Processing: Leverage Pega’s multi-threading or queue mechanisms to process multiple files concurrently.
  • Scaling to 50 Files Concurrently:
    • Threads: In the File Listener rule, you can adjust the Number of Threads property (under the “Processing” tab). Set it to 50 to allow up to 50 files to be processed simultaneously, assuming your Pega nodes can handle the load.
    • Queueing: If 50 threads strain resources, use a Queue Processor or Job Scheduler to manage the workload. The File Listener can move files to a queue, and the Queue Processor can handle them in batches or parallel streams

Regards,

Mohd Qizer Uddin