The default number of partitions after 8.7 for a queue process is 6. We can increase it using pyalterstreampartitions activity.
The question we have is, is there a 1:1 relation between the maximum of threads which can be spawned across cluster to the number of partitions?
Example: If i increase the number of partitions to 40, can i increase the number of threads across cluster for a Queue processor to 40? or the maximum threads allowed is always 20 for a queue processor?
I believe it supports only up to 20 threads per cluster.
@sangr2 Can you confirm on it? I didn’t find any clear documentation on it. If you can provide any supporting documentation it will be of great help.
@KarthickRajasingh in the older versions it was 20 partitions. Now in recent versions pega supports 6 partitions in a cluster. Please check 6th question in the below url.
https://docs.pega.com/bundle/platform/page/platform/background-processing/queue-processor-faq.html
Could you please check the below articles? Very useful documents are related to Kafka and queue process documents
Note: My understanding is that from Pega Platform 8.7 onwards, it will create only 6 participations by default in 8.7 and before 8.7 versions it will create 20 participations by default.
If you want to increase or decrease the number of participants, you can run the below activity.
Activity Name: pxAlterStreamPartitions (This activity available in Pega 8.7 and above versions)
Parameters:
stream: Pass in Your QP name All in Capital Letters
partitions: Number of partitions, I have tested it, I gave the value 25 , It is allowed and Created the 25 partitions in Personal Edition 8.7
As above mentioned the Tool tip message, It is Same for the all versions, 8.6 and 8.7, The Value should be dynamic for every QP. Even The default value 6 partitions in the 8.7 Pega Platform, Still Tool tip message not updated and It is showing the 20.
Thanks,
Ashok
In my scanario, the maximum number of threads is 25, as I have increased the number of threads from 6 to 25. Still, if you want an increase, you can increase threads.
FYI, I have again increased 100 Threads,
PFA
Thanks,
Ashok
To test this try to execute this qp which would get execute all the requestors at once. I believe this might create the 40 threads (requestors). But the disadvantage of having the threads (requestors) it might eat up the resources.
To check the threads, open the dataflow associated with the QP and check how it behaves.
Maximum number of threads across all nodes in a cluster for a QP is 20. That’s the limit for a QP. Please check 8th point in the below url.
https://docs-previous.pega.com/system-administration/87/queue-p
And if you increase beyond 20, only 20 will process messages from a QP. And the threads should be increased on the CPU and heap. We need to ensure if Host server can handle the load before increasing the threads to 20. But maximum number of threads limit is 20 for a QP.
https://support.pega.com/question/increasing-thread-count-stream-node-and-custom-node
@Bhumireddy I completely understand and we did test from our end to make modifications on number of partitions but my question is related to number of threads. As explained by you, you can have 25 partitions but what is the maximum number of threads? is it 25 or 20 is the max?
