Externalization of kafka - IBM Event Streams

General
1

Hi All,

As per this, it is mentioned that “you can buy a Kafka license from a provider, for example Confluent”. Does PEGA supports IBM event streams as external kafka for its stream service?

This information is not available in platform support guide too. Is there a list which we can refer to know the compactable kafka providers which PEGS supports.

Thanks.

2

Hi @ArulDevan I was also not able to find these details in the documentation.

Would you be able to submit a documentation feedback ticket against the External Kafka in your deployment article and suggest that they include these details there?

ddd.png
ddd.png2339×383 53.3 KB

Can you confirm that the Support Document Third-party externalized services Deployment Changes FAQs also does not clarify this point?

dfS.png
dfS.png1019×383 58.3 KB

As the article has been left open for response, feel free to hit ‘Reply’ on the main content in order to ask your question directly to the author who wrote the support document.

3

Hi @MarijeSchillern: Thanks for the guidance. Have raised a feedback in the documentation and also raised a query in other FAQ support doc.

Thanks.

4

@ArulDevan Please do share the update here as well.

To my knowledge, IBM Event Streams is not listed as official supported streaming service, however, it appears, it might work as long as IBM Event Streams suports API Key or User ID/Password Authentication.

5

Hi @MarijeSchillern and moderator team: Can you please help to get support from the product team on this?

Thanks.

6

@ArulDevan I am already tracking your requests and am locating the SME’s for you. We will respond asap.

7

@ArulDevan our SME posted a response via the Support Document: Third-party externalized services Deployment Changes FAQs

"This doc https://docs.pega.com/bundle/platform-88/page/platform/deployment/exter… provides details on setting up Elasticsearch.

Services like kibana/logstash/beats are all external to Pega and are up to the client to choose and deploy as needed."

Let me know which questions are still outstanding and I’ll ask them to post a response here.

8

Hi @MarijeSchillern: Thanks for tracking this. Noticed that the reply in the other thread is for elastic search. The query on Kafka is still open.

Thanks.

9

@ArulDevan I expect the answer is that if IBM Event Streams are fully compatible with Kafka, it should work.

@malaa1 would you be able to provide some input on this Kafka question?

" Is there a list which we can refer to know the compatible kafka providers which PEGS supports?"

10

@ArulDevan :Thanks for the question: Currently we don’t have list of compatible Kafka providers that you can use to connect with pega platform as external stream. But I can answer it in a little bit different format.

Essentially any kafka provider should support following 2 main points

  1. Supports/be compatible with Kafka protocol
  2. Supports any of the following authentication mechanims to connect to kafka
    • SASL using JAAS
    • SASL/PLAIN
    • SASL/SCRAM

Reference Doc: https://docs.pega.com/bundle/platform/page/platform/deployment/externalization-of-services/externalize-kafka-in-your-deployment.html

Now coming to IBM Event streams, it satisfies both requirements mentioned above as it

  • supports Kafka Protocol
  • supports connection via SASL PLAIN

https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-connecting
https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-kafka_using#kafka_api_client

Hope this helps.