Pega 8 integration with AWS MSK kafka

General
, , ,
1

Hi,

Can we connect to external AWS MSK kafka from pega 8.6 with SASL authentication (JAAS configuration file).

have followed below link for SASL but no luck.
Pegasystems Documentation?

Thanks,
Vasu

2

@Vasu009 , are you able to connect to the AWS MSK ?

3

@satsihc6 Yes, i was able to establish connection to AWS MSK with SASL_Scram as authentication.

After included below lines in the .properties file (kafka instance)

sasl.mechanism=SCRAM-SHA-512
security.protocol=SASL_SSL
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \ username=“XXXXXXXXXXXX” \ password=“XXXXXXXXXXX”;

4

@Vasu009 Do you know how to use this SASL_SSL configuration in Pega 8.8 Containerized environments. I have doubts on creating ScrectStores.

Any one who has recently configured this with AWS MSK please assist me on that

Need to clarify following issues that I am facing with that.?

  1. What is STREAM_TRUSTSTORE_PASSWORD what are the steps to create it and can we store that truststore content in the same secret?
  2. STREAM_KEYSTORE_PASSWORD is there, but I cannot find KEY to store keystore content in the secret. Could you please clarify this with the steps?
  3. In STREAM_JAAS_CONFIG could you please let me know valid key value pair to create the secret. and can we use the same secret to store (1),(2) values along with 3rd.?

Its better someone can share the sample yaml files related for external kafka configuration with SASL_SSL. Because the doc is only for the PLAINTEXT mechanism.

5

@Vasu009 Hi, Greetings. I am trying to connect to MSK by enabling public access as well as using SASL/SCRAM, i have the below in properties file

security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \ username=“xxx” \ password=“xx-xxx”;

But i am getting

Could not connect to Kafka, reason: com.pega.dsm.kafka.api.KafkaConnectivityException: Error connecting to Kafka

Is there something i am missing? Can someone please help me with this?

Regards,

Bharat