Background node does not start on On-premise Pega '23

General
, , ,
1

Hi Everyone

After installed '23 successfully, we failed to start Pega as Stream node.I successfully addressed a Stream node issue by adding to prconfig.xml . Stream node is running.

However still are facing another issue on Search and BackgroundProcessing nodes. “Failed to initialize StreamAPI: attempt 1 from 6. Next retry in 20 seconds java.util.concurrent.TimeoutException: null”

There is no external service for Kafka and Elasticsearh. Also was no connection details for timeout issue ( source & target ip , port) in logs after changing log levels for cluster and stream class.

Troubleshooting Steps Already Taken:

Is there any configuration for internal stream conf ? What are the common causes for a TimeoutException during StreamAPI initialization on Search and BackgroundProcessing nodes?

************************* Logs ***********************************************

(.stream.StreamServiceValidator) ERROR - Failed to initialize StreamAPI: attempt 1 from 6. Next retry in 20 seconds
java.util.concurrent.TimeoutException: null
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]
at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]
at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]
at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]
at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]
at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]
at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
2023-12-28 15:25:55,508 [ALIZE_STREAM_SERVICE] [ STANDARD] (.stream.StreamServiceValidator) ERROR - Failed to initialize StreamAPI: attempt 2 from 6. Next retry in 20 seconds
java.util.concurrent.TimeoutException: null
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]
at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]
at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]
at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]
at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]
at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]
at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
2023-12-28 15:26:20,510 [ALIZE_STREAM_SERVICE] [ STANDARD] (.stream.StreamServiceValidator) ERROR - Failed to initialize StreamAPI: attempt 3 from 6. Next retry in 20 seconds
java.util.concurrent.TimeoutException: null
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]
at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]
at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]
at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]
at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]
at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]
at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
2023-12-28 15:26:30,246 [rver.Default-Pulse:2] [ STANDARD] (api.server.StreamServerService) DEBUG - Finished executing stream service pulse operation
2023-12-28 15:26:45,512 [ALIZE_STREAM_SERVICE] [ STANDARD] (.stream.StreamServiceValidator) ERROR - Failed to initialize StreamAPI: attempt 4 from 6. Next retry in 20 seconds
java.util.concurrent.TimeoutException: null
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]
at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]
at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]
at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]
at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]
at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]
at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
2023-12-28 15:27:10,514 [ALIZE_STREAM_SERVICE] [ STANDARD] (.stream.StreamServiceValidator) ERROR - Failed to initialize StreamAPI: attempt 5 from 6. Next retry in 20 seconds
java.util.concurrent.TimeoutException: null
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]
at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]
at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]
at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]
at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]
at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]
at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
2023-12-28 15:27:30,324 [rver.Default-Pulse:2] [ STANDARD] (api.server.StreamServerService) DEBUG - Finished executing stream service pulse operation
2023-12-28 15:27:35,515 [ALIZE_STREAM_SERVICE] [ STANDARD] (.stream.StreamServiceValidator) ERROR - Failed to initialize StreamAPI: attempt 6 from 6
java.util.concurrent.TimeoutException: null
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]
at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]
at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]
at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]
at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]
at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]
at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
2023-12-28 15:27:35,530 [ALIZE_STREAM_SERVICE] [ STANDARD] ( external.async.IStartupTask) ERROR - Error in the execution of Startup Task: [com.pega.dsm.dnode.api.server.StreamServiceInitializationTask]
com.google.inject.ProvisionException: Unable to provision, see the following errors:

  1. Error in custom provider, com.pega.platform.stream.StreamServiceException: Failed to initialize StreamAPI. Can not connect to the stream provider.
    at com.pega.platform.modules.internal.guice.StreamServiceModule.configure(StreamServiceModule.java:53)
    at com.pega.platform.modules.internal.guice.StreamServiceModule.configure(StreamServiceModule.java:53)
    while locating com.pega.fnx.stream.api.StreamAPI

1 error
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1025) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]
at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]
at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]
at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]
at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]
at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: com.pega.platform.stream.StreamServiceException: Failed to initialize StreamAPI. Can not connect to the stream provider.
at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:86) ~[stream.jar:?]
at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]
… 23 more
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]
at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]
at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]
at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]
at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]
… 23 more
2023-12-28 15:27:35,531 [ STANDARD] ( external.async.IStartupTask) INFO - All registered ‘StartupTasks’ (parallel cache loading) have finished.
2023-12-28 15:27:35,532 [ STANDARD] ( internal.mgmt.PREnvironment) ERROR - com.pega.pegarules.pub.PRRuntimeException: Startup Task: [INITIALIZE_STREAM_SERVICE] has failed after retries or with error

Thank you in advance for any suggestions, advice, or shared experiences related to this issue.

2

@kasime17008587

Hi, May I know your inputs on the below queries first?

  1. Was all the nodes working before prior to Pega Infinity 23 installation?

  2. Pega Infinity 23 is the first installation in On-Prem or was upgraded from any other version? if upgraded from different version? which one is it?

  3. Since it’s On-prem installation, May I know your environment details? and tools being used?

  4. Did you check urandom parameter is configured?

  5. did you make sure Pega-Engine • prconfig/services/stream/provider/default [ Value: ExternalKafka] exist by default when you install Pega Infinity’23.

3

@Kishore Sanagapalli Thanks for replying. Kindly check the info.

  1. It is the first start execution for all nodes.

  2. Pega Infinity 23 is the first installation in On-Prem

  3. Apache Tomcat 9 Server and Ms sql 15 DB

  4. Did you check urandom parameter is configured? NO Please more details

  5. Stream type is internal and applied in prconfig.xml ( On-premise '23 Pega platform does not start with Tomcat | Support Center applied to prconfig.xml)

4

@kasime17008587

Thanks for the details.

about urandom: -Djava.security.egd=file:///dev/urandom is one of the JVM argument plays major role in the genrating requestors for Pega. Since It’s on-prem, You need to check this parameter is configured or not in your middleware application at the JVM arguments definition section.

If it’s not there, please add it and restart the JVM’s and check once.

If you need more help whether urandom is configured ot not,

Issue the below command from your linux terminal:

ps -ef | grep java

Note: Let me know the outcome

5

@Kishore Sanagapalli

It is configured. Able to see to catalina.out logs.

org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.security.egd=file:///dev/urandom

Thanks

6

@kasime17008587

Hello,

Is it a fresh installation or an upgrade from a previous version? which version in case of.

Have you specified port for your stream service in prconfig?

Have you well defined node classification for Stream, Search & Backgroundprocessing?

What’s shown in Decisioning > infrastructure > Services > Stream?

Is it empty? or you’ve got old ref ready to be decomisionned ?

Do you see your node?

Have you tried a clean restart with pegatemp cleared and all stream tables emptied?

Thank you

Regards

Anthony

7

@kasime17008587 Please can you confirm that your open ticket is INC-B417 ( Background node does not start on On-premise Pega '23) that was logged 4th January?

Our GCS team have been waiting for a response from you since the 15th January regarding the below ticket update:

Current Status:

By looking at the stack trace, the background processing node is trying to start the StreamAPI module and perform validation of the stream service by verifying if a stream node is up in the same cluster, the stream validation is enabled in all nodes by default.

The exception is a straight timeout, which means the stream node is possibly not reachable from the background processing node perspective, as it’s not identifying the kafka server and a stream server in NORMAL status to perform the stream validation.

—> Can you please ensure that both servers can reach each other through the local network and also check if there is any firewall rule possibly blocking this connection?

—> can you please provide logs from the stream node which was already running when you tried to bring you the background processing node, so I can review the initialization for that node as well.

Next Steps:

Please get in touch with GCS to confirm on above items and provide logs from the stream node which should be running in the same Pega cluster.

8

Hi @Anthony_Gourtay

Thanks your supports in advance. still not fixed it. I raised SR regarding the issue. I share the solution here once fixed

Please find my answer.

Is it a fresh installation or an upgrade from a previous version? which version in case of. /// New Installation

Have you specified port for your stream service in prconfig? /// no override regarding ports

Have you well defined node classification for Stream, Search & Backgroundprocessing? /// yes

What’s shown in Decisioning > infrastructure > Services > Stream? /// Only stream nodes is running.

Is it empty? or you’ve got old ref ready to be decomisionned ?/// empty.

Do you see your node? /// yes . under decisioning landing and on Admin studio

Have you tried a clean restart with pegatemp cleared and all stream tables emptied? /// yes.

9

Hi @kasime17008587

Could you share with us the support case id so that we can track it and connect this thread with it for the Support Engineer?

Thanks!

10

@kasime17008587 please can you respond to the requests from GCS? INC-B417 can be access via MSP

Analysis was that this is a straight forward issue as you’re using an invalid PRCONFIG for external stream configuration. There was an issue with a previous version of our documentation which this incorrect PRCONFIG was provided. It was fixed few months ago but looks like you still have that version of instructions.

As the solution was provided which should address your configuration issue, we are planning to move this case to closure by tomorrow EOD If we don’t hear back from you, so you can try the provided solution and re-open the case in case you still need assistance with the same issue.

Jan 23, 2024 07:32 PM

Current Status:

(logs from 1/14). Your startup process is failing to initialize StreamAPI because it’s looking for external stream connection properties, as shown below:

2024-01-14 12:15:42,659 [ALIZE_STREAM_SERVICE] [ STANDARD] (prpc.service.ServiceDefinition) ERROR - Unable to create DSM service DATA-DECISION-SERVICE-STREAMSERVER DEFAULT

java.lang.reflect.InvocationTargetException: null
[…]
Caused by: com.pega.platform.stream.StreamServiceException: Failed to configure StreamAPI provider
[…]
Caused by: com.pega.platform.stream.StreamServiceException: ‘services/stream/broker/url’ isn’t provided.

solution:
Please try also providing the below PRCONFIG to the BackgroundProcessing node and see if that addresses the issue.

If not, please provide fresh startup logs from Stream and BackgroundProcessing nodes so I can review the startup history.

This PRCONFIG in theory would force the JVM to look for an internal stream node in the same cluster. Please confirm if you have tried this already or not.

If it doesn’t work, please try to change the node classification for the backgroundprocessing node and add Stream node type to it and see if it joins the same cluster as the other stream node. If it doesn’t identify the other node, it’s most likely a hazelcast/cluster management configuration issue or the nodes aren’t able to communicate through network.

Next Steps:

  • Client – to try the above suggestion and confirm back. If it doesn’t work, client to provide fresh startup logs for both stream and backgroundprocessing node so GCS can review.

***Edited by Moderator Marije to add update a week later ***

No response so we will assume the issue is now resolved with the above steps.