Data Flow java.lang.illegalstateexemption

General
, , , ,
1

Dataflow created to move the records from a table to another data table with transformation. I was able to move around 10k records but 3 records failed with java.lang.illegalstateexemption. This happens during dataset stage where the records will be written into the table. Need ways to enable logs to find more details. Is it possible to get the insert query triggered by the dataflow to store records with dataset in the logs which will help to identify the exact rootcause? Also, where can we find the full stack trace. The stack trace shown in the dataflow is not giving proper details.

com.pega.dsm.dnode.api.dataflow.StageException: Exception in stage: WorkData at com.pega.dsm.dnode.api.dataflow.StageException.create(StageException.java:54) at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onCompleted(DataFlowStage.java:521) at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor$5.process(DataFlowExecutor.java:384) at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor.runEventLoop(DataFlowExecutor.java:218) at com.pega.dsm.dnode.api.dataflow.DataFlow$2.emit(DataFlow.java:331) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:59) at com.pega.dsm.dnode.impl.dataflow.task.strategy.ExecutionStrategy.executeDataFlow(ExecutionStrategy.java:125) at com.pega.dsm.dnode.impl.dataflow.task.strategy.SinglePartitionExecution.executePartitions(SinglePartitionExecution.java:77) at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.executeDataFlowWithPartitions(ExecutePartitionsRunnable.java:86) at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.access$900(ExecutePartitionsRunnable.java:34) at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable$1.run(ExecutePartitionsRunnable.java:67) at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable$1.run(ExecutePartitionsRunnable.java:62) at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:67) at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:305) at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.call(ExecutePartitionsRunnable.java:62) at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.call(ExecutePartitionsRunnable.java:34) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:59) at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:56) at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:67) at com.pega.dsm.dnode.impl.prpc.PrpcThreadFactory$PrpcThread.run(PrpcThreadFactory.java:124) Caused by: java.lang.IllegalStateException: Should be handled by the source data observable at com.pega.dsm.dnode.impl.dataflow.resilience.retry.RetryingDataObservableDecorator.emit(RetryingDataObservableDecorator.java:119) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:353) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:55) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$3.emit(DataObservableImpl.java:176) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:353) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:55) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$3.emit(DataObservableImpl.java:176) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:353) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:55) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:117) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:106) at com.pega.dsm.dnode.impl.dataflow.SaveStageProcessor.onNext(SaveStageProcessor.java:140) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.commitBatchInternal(DataFlowStageBatchProcessor.java:147) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.commitBatch(DataFlowStageBatchProcessor.java:132) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.onCompleted(DataFlowStageBatchProcessor.java:118) at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onCompleted(DataFlowStage.java:517) … 24 more Caused by: com.pega.dsm.dnode.api.ExceptionWithInputRecord: com.ibm.db2.jcc.am.BatchUpdateException: [jcc][t4][102][10040][4.33.31] Batch failure. The batch was submitted, but at least one exception occurred on an individual member of the batch. Use getNextException() to retrieve the exceptions for specific batched elements. ERRORCODE=-4229, SQLSTATE=null at com.pega.dsm.dnode.impl.dataset.database.DatabaseSaveOperation$AbstractSaveOperation.execute(DatabaseSaveOperation.java:182) at com.pega.dsm.dnode.impl.dataset.database.DatabaseSaveOperation$1.executeExposedLogic(DatabaseSaveOperation.java:526) at com.pega.dsm.dnode.impl.dataset.database.DatabaseOperationsUtil.lambda$observableForDatabaseOperation$0(DatabaseOperationsUtil.java:73) at com.pega.dsm.dnode.impl.dataset.database.DatabaseOperationsUtil$$Lambda$358/0x000000002cf48930.emit(Unknown Source) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:353) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:55) at com.pega.dsm.dnode.impl.dataset.database.RetryableSave.lambda$save$0(RetryableSave.java:43) at com.pega.dsm.dnode.impl.dataset.database.RetryableSave$$Lambda$4609/0x00000000599bf510.emit(Unknown Source) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:353) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:55) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$3.emit(DataObservableImpl.java:176) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:353) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:55) at com.pega.dsm.dnode.impl.dataflow.resilience.retry.RetryingDataObservableDecorator.produceRecords(RetryingDataObservableDecorator.java:147) at com.pega.dsm.dnode.impl.dataflow.resilience.retry.RetryingDataObservableDecorator.emit(RetryingDataObservableDecorator.java:102) … 39 more Caused by: com.ibm.db2.jcc.am.BatchUpdateException: [jcc][t4][102][10040][4.33.31] Batch failure. The batch was submitted, but at least one exception occurred on an individual member of the batch. Use getNextException() to retrieve the exceptions for specific batched elements. ERRORCODE=-4229, SQLSTATE=null at com.ibm.db2.jcc.am.b4.a(b4.java:502) at com.ibm.db2.jcc.am.Agent.endBatchedReadChain(Agent.java:434) at com.ibm.db2.jcc.am.ko.a(ko.java:5664) at com.ibm.db2.jcc.am.ko.c(ko.java:5216) at com.ibm.db2.jcc.am.ko.executeBatch(ko.java:3206) at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecuteBatch(WSJdbcPreparedStatement.java:1146) at com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.executeBatch(WSJdbcStatement.java:856) at com.pega.pegarules.data.internal.store.DatabasePreparedStatementImpl.executeBatch(DatabasePreparedStatementImpl.java:491) at com.pega.dsm.dnode.impl.dataset.database.DatabaseSaveOperation$UpdateOperation.process(DatabaseSaveOperation.java:320) at com.pega.dsm.dnode.impl.dataset.database.DatabaseSaveOperation$AbstractSaveOperation.execute(DatabaseSaveOperation.java:176) … 53 more

2

@NivethaR3993

:warning: This is a GenAI-powered tool. All generated answers require validation against the provided references.

Let me address each of your questions with specific steps you can take to troubleshoot this issue.

Enabling Logs for More Details

To enable detailed logging that will help identify the root cause of the IllegalStateException, you can configure the following:

  1. Enable requestor lifecycle logging:
    • Set the logger com.pega.TRACE.requestor_lifecycle to DEBUG level
    • This can be configured in the Dynamic System Settings (DSS) or prconfig.xml
  2. For dataflow-specific logging, enable these loggers at DEBUG level:
    • For Data Flow service: com.pega.dsm.dnode.impl.dataflow
    • For dataset operations: PegaSearch.Indexer (if using Pega 7.3.1 or later)

Capturing Insert Queries in Logs

Yes, it is possible to get the insert queries triggered by the dataflow in the logs. To enable JDBC tracing that will capture SQL statements:

  1. Enable JDBC tracing through one of these methods:
    • Set the Java argument -Doracle.jdbc.Trace=true (for Oracle databases)
    • For Unix environments:
    • o ``export ANT_OPTS="-Doracle.jdbc.Trace=true -Djava.util.logging.config.file=myJDBCConfig.properties"
    • For Windows environments:
    • o ``set ANT_OPTS="-Doracle.jdbc.Trace=true -Djava.util.logging.config.file=myJDBCConfig.properties"
    • Alternatively, you can enable the Database Trace tool from the Performance landing page for a short period (note that this can impact performance)

Finding the Full Stack Trace

The stack trace shown in the dataflow UI is often abbreviated. To access the complete stack trace:

  1. Access detailed error reports from the Data Flow landing page:
    • Navigate to Configure > Decisioning > Decisions > Data Flows
    • Select the appropriate tab for your dataflow type (Batch processing, Real-time processing, or Single case processing)
    • Find and click on the name of the failed Data Flow run
    • In the error message for the failed run, click “Details” to view the full stack trace in a new window
  2. Check system logs:
    • The PegaRULES log files may contain more detailed stack traces
    • For dataflow-specific issues, check the logs from the node where the Data Flow service is running

Additional Troubleshooting Tips

Since the error occurs during the dataset stage when records are being written to the table, consider these additional steps:

  1. Check if there are any data validation issues or constraints in the target table that might be causing the IllegalStateException
  2. Examine the three failing records for any anomalies or differences compared to the successfully processed records
  3. If possible, try processing just the failing records separately to isolate the issue
  4. For dataflow runs that fail while writing to a database Data Set, check the documentation on “Data Flow runs return errors while writing records to a database Data Set” in the Troubleshooting Data Flows section of the Pega documentation

References:
Troubleshooting Data Flows
Investigating Data Flow run failures