Handle Invalid input to Date time attribute in Kafka JSON request

General
, , , ,
1

Issue Context

We have data flow which includes a Data set as source and activity as destination. System reads the value from kafka dataset and saves the record in pega database through activity. Whenever invalid value is passed in any one of the datetime attribute part of the Kafka JSON request, the attribute is dropped from the clipboard when it reaches the activity in data flow.

We are using activity in data flow because kafka JSON request has array of objects so we are looping through each one of the object and saving it to pega database.

Our requirement is to validate each property types and log an exception if invalid value is passed to a data type. But as the attribute itself is dropped we are not able to validate the property. Used Page Validate method in activity to ensure all the properties in the page gets validated but it didn’t work.

Is there a way we can ensure the attribute and its value retains during activity execution in dataflow when any of the attribute has invalid value.

2

@ArunP210 After engaging the Kafka SME for investigation, the support ticket INC-A22456 (Handle NULL/Empty value in JSON request) was closed with the following outcome:

Issue primary reason description:

The way of data processing in standalone activity and dataset is same.
In both the cases, when JSON is passed, it is mapped to clipboard and null values are not present , in order to make it work, use JSON Data transform to map the data properly before passing it to the activity.

Explanation description:

The way of data processing in standalone activity and dataset is same.

In both the cases, when JSON is passed, it is mapped to clipboard and null values are not present , in order to make it work, use JSON Data transform to map the data properly before passing it to the activity.