BIX real time data extraction questions on method, performance and security

General
, , , ,
1

Hi team, Pega 24.2 starts to support BIX real time data extraction. I have few questions about the architecture, performance and security.

  1. would like to the whether the real time data extraction can push real time data to database table? or this is only available to publish to Kafka?

  2. Is there any performance issue when the streaming is running (consider if there is a big load occurs at certain time).

  3. How BIX real time extraction handles retries if any failures happen?

  4. how the data is secured during data transit? considerations include: whether it uses TLS 1.3 .

2

@HaimingZ17189808 I see that you logged INC-C4584 and that our support team provided you with the answers:

1 It is only to Kafka. customer can implement DataFlow or other custom code to read from Kafka and push to database.
#2 We tried with 55 users concurrently creating\updating 100 cases.. haven’t observed any significant performance impact. it was at negligible numbers.
3 For now the error and exception handling isn’t great. we reuse the Queue Processor capability here.
We currently have a dead letter topic to which the messages are sent if failure is encountered while pushing data to Kafka. We are enhancing to retry the messages from the dead letter topic. It will be addressed in upcoming releases.
4 Data is written to Kafka using our DataSet rule. so, whatever is the security configuration for DataSet, all of that are applicable.

3

@MarijeSchillern : For failure while writing to Kafka would result in data loss? from dead letter topic Pega is not re-trying to send the message ?

This could result in data loss if no-retry is avaialble . Any recommendation on how to avoid /handle it ?

4

@Mayurk33 you can find documentation addressing the limitations here: Configuring BIX real-time data extraction

If you need further help, please ask this as a New Question.

5

@HaimingZ17189808: If you have implemented Real Time bix , if yes can you share some screen prints . I too have requirement to implement Real time BIX.