Real time processing data flow failure

General
, , ,
1

Hello, we have a real time process data flow that reads from a Kafka topic through data set rule and stores the data into a Cassandra table through another data set rule. It works fine in lower domains. When the flow is run in preprod domain, we are getting this error.

Data flow execution failed while the thread was initializing: NullPointerException: Details.

Clicking on the details, we get this:

java.lang.NullPointerException
	at com.pega.dsm.dnode.api.dataflow.DataFlow.getDataPartitions(DataFlow.java:805)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.PartitionCreator.createFor(PartitionCreator.java:80)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.PartitionCreator.access$000(PartitionCreator.java:45)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.PartitionCreator$1.run(PartitionCreator.java:62)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.PartitionCreator$1.run(PartitionCreator.java:59)
	at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:67)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:291)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1380)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1123)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:930)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:284)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:146)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.PartitionCreator.createFor(PartitionCreator.java:59)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.RunInitializationTask.preparePartitions(RunInitializationTask.java:204)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.RunInitializationTask.asyncExecution(RunInitializationTask.java:130)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.AsyncAssignedTask$1.run(AsyncAssignedTask.java:69)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.AsyncAssignedTask$1.run(AsyncAssignedTask.java:58)
	at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:67)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:301)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:146)
	at com.pega.dsm.dnode.impl.dataflow.service.servicepulse.AsyncAssignedTask.lambda$tryToExecuteInLock$0(AsyncAssignedTask.java:58)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:59)
	at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:56)
	at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:67)
	at com.pega.dsm.dnode.impl.prpc.PrpcThreadFactory$PrpcThread.run(PrpcThreadFactory.java:124)

Does anyone know what can we do resolve this issue? Would appreciate any input. Thanks in advance!!

2

@VenkatA2871 Did you get a resolution for this?

3

@Piyush - We did not get any response for the root cause. Anyway, we resolved the issue with these steps. We migrated the data flow run data instance (DATA-DECISION-DDF-RUNOPTIONS) in a product rule from dev. That worked fine in a lower domain. But did not, in pre-prod. Error as shown above. We deleted the data instance that was migrated and re-created new one and that worked fine in pre-prod. Don’t know whether this is the resolution or not. In production, we did not face any issue. The original data instance that was migrated from lower domain worked perfectly as such. Hope this helps.