pyBatchIndexProcessor fails to run

General
, ,
1 
pyBatchIndexProcessor fails to run 

pyBatchIndexProcessor dataflow shows error 

Slow component [OutputStage] with id [Destination] on data flow [pyBatchIndexProcessor] belonging to class [System-Message-QueueProcessor] detected. 
Please check the logs of node [9e85a7e0b971aad3f64d4f60e19d8005] or the corresponding PEGA0131 alert for full details of the slow dataflow processing thread stacktrace. Stacktrace ID [f9ede8a5-a753-4eb4-bb88-a560e5671f90]

Anybody had this issue?

2

a @JohnPaulRaja,C I am al.so facing the same issue in Pega 8.8.4 . is this issue resolved.

com.pega.platform.executor.queueprocessor.QueueProcessorRuntimeException: Unexpected exception while executing QP: 
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.RequeueErrorHandler.handle(RequeueErrorHandler.java:44)
	at com.pega.dsm.dnode.api.dataflow.service.ResilienceConfig$CustomErrorHandlerWrapper.handle(ResilienceConfig.java:439)
	at com.pega.dsm.dnode.impl.dataflow.resilience.AtLeastOnceResilienceStrategy.handleErrors(AtLeastOnceResilienceStrategy.java:335)
	at com.pega.dsm.dnode.impl.dataflow.resilience.AtLeastOnceResilienceStrategy.onError(AtLeastOnceResilienceStrategy.java:202)
	at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor$4.process(DataFlowExecutor.java:371)
	at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor.runEventLoop(DataFlowExecutor.java:218)
	at com.pega.dsm.dnode.api.dataflow.DataFlow$2.emit(DataFlow.java:339)
	at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:59)
	at com.pega.dsm.dnode.impl.dataflow.task.strategy.ExecutionStrategy.executeDataFlow(ExecutionStrategy.java:122)
	at com.pega.dsm.dnode.impl.dataflow.task.strategy.MultiplePartitionExecution.executePartitions(MultiplePartitionExecution.java:87)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.executeDataFlowWithPartitions(ExecutePartitionsRunnable.java:86)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.access$900(ExecutePartitionsRunnable.java:34)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable$1.run(ExecutePartitionsRunnable.java:67)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable$1.run(ExecutePartitionsRunnable.java:62)
	at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:305)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.call(ExecutePartitionsRunnable.java:62)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.call(ExecutePartitionsRunnable.java:34)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:69)
	at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:66)
	at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77)
	at com.pega.dsm.dnode.impl.prpc.PrpcThreadFactory$PrpcThread.run(PrpcThreadFactory.java:164)
Caused by: com.pega.dsm.dnode.api.dataflow.StageException: Exception in stage: OutputStage
	at com.pega.dsm.dnode.api.dataflow.StageException.create(StageException.java:54)
	at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onNext(DataFlowStage.java:504)
	at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onNext(DataFlowStage.java:426)
	at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor$2.process(DataFlowExecutor.java:329)
	... 23 more
Caused by: com.pega.dsm.dnode.api.ExceptionWithInputRecord: com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorTransactionFailedException: PRRuntimeException
	at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onNext(DataFlowStage.java:503)
	... 25 more
Caused by: com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorTransactionFailedException: PRRuntimeException
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorOnlyOnceExecutionStrategy.saveOffsetAndCommit(QueueProcessorOnlyOnceExecutionStrategy.java:297)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorOnlyOnceExecutionStrategy.after(QueueProcessorOnlyOnceExecutionStrategy.java:225)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorUsageReportingExecutionStrategy.after(QueueProcessorUsageReportingExecutionStrategy.java:88)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorUsageReportingExecutionStrategy.lambda$execute$0(QueueProcessorUsageReportingExecutionStrategy.java:66)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1005)
	at com.pega.pegarules.session.internal.PRSessionProviderImplForModules.doWithRequestorLocked(PRSessionProviderImplForModules.java:83)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorUsageReportingExecutionStrategy.execute(QueueProcessorUsageReportingExecutionStrategy.java:52)
	at com.pega.platform.executor.internal.LogContextDecorator.runInDecoratedScope(LogContextDecorator.java:38)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorDataFlowFactory$1$1.onNext(QueueProcessorDataFlowFactory.java:96)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorDataFlowFactory$1$1.onNext(QueueProcessorDataFlowFactory.java:79)
	at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onNext(DataFlowStage.java:501)
	... 25 more
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
	at com.pega.pegarules.data.internal.access.DatabaseImpl.commit(DatabaseImpl.java:2210)
	at com.pega.pegarules.data.internal.access.DatabaseImpl.commit(DatabaseImpl.java:2137)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorOnlyOnceExecutionStrategy.saveOffsetAndCommit(QueueProcessorOnlyOnceExecutionStrategy.java:291)
	... 37 more
Caused by: java.lang.NullPointerException
	at com.pega.pegarules.priv.LogHelper.isDebugEnabled(LogHelper.java:1097)
	at com.pega.pegarules.generation.internal.cache.AssemblyCacheWrapper.noteChange(AssemblyCacheWrapper.java:817)
	at com.pega.pegarules.generation.internal.PRGenProviderImpl.noteChangeABAOnly(PRGenProviderImpl.java:523)
	at com.pega.pegarules.generation.internal.assembly.FUAManagerImpl.noteChange(FUAManagerImpl.java:3493)
	at com.pega.pegarules.generation.internal.PRGenProviderImpl.noteChange(PRGenProviderImpl.java:494)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.noteChange(UpdateProcessor.java:1249)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.notifyOtherSystemsAfterUpdates(UpdateProcessor.java:1010)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.notifyEverybodyOfChanges(UpdateProcessor.java:528)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.after(UpdateProcessor.java:340)
	at com.pega.pegarules.data.internal.store.DataStoreManager.doInTransaction(DataStoreManager.java:296)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.processUpdates(UpdateProcessor.java:233)
	at com.pega.pegarules.data.internal.access.DatabaseImpl.commit(DatabaseImpl.java:2175)
	... 39 more
3

@JohnPaulRaja,C @AnshumanM1959

Please can you go through the available documentation:

pyFTSIncrementalIndexer not running or turning off automatically

Stopping or pausing search reindexing

Managing queue processors

PEGA0131 alert: Detected slow Data Flow component

sy.png
sy.png1814×434 76.9 KB

From the FAQ:

Why is my new queue processor not working?

If your queue processor does not work, consider the following:

  • Queue processor rules rely on the DSM service. In Pega Cloud services and client-managed cloud environments, a stream node is automatically configured and ready to use. For on-premises environments, ensure that you define at least one node in a cluster to be a Stream node using the setting: -DNodeType=Stream. For more information, see Stream node configurations for a queue processor rule.
  • One problem may be that Pega Platform is not be able to properly resolve your queue processor rule. For more information on how queue processor rules work, see System Runtime Context and General system settings.
  • If further troubleshooting is necessary, see Troubleshooting background processing.

There is also a good Pega Academy course: Background process management