Issue
Some organizations use Cribl Stream as an intermediary for parsing and transforming log data before forwarding it to a downstream system such as Splunk. A common question is whether Pega Cloud can stream logs directly to an on-premises Cribl instance.
Current Capability
Direct log streaming from Pega Cloud to an on-premises Cribl instance is not currently supported. Pega Cloud supports direct log streaming to Splunk, Amazon S3, Azure, and Google Cloud Platform (GCP) destinations.
Recommended Workaround: Use Amazon S3 as an Intermediary
You can achieve Cribl integration using the following pattern:
- Configure Pega Cloud to stream logs to an Amazon S3 bucket (Pega Cloud’s supported log streaming target).
- Configure your on-premises Cribl Stream instance to pull logs from the S3 bucket.
- Cribl performs the necessary parsing and transformation on the ingested logs.
- Cribl forwards the processed data to your on-premises Splunk instance (or other destination).
This pattern uses cloud object storage as a staging layer, enabling integration with on-premises data processing tools while maintaining compatibility with Pega Cloud’s supported log delivery mechanisms.
Benefits of This Approach
- No changes required to the Pega Cloud log streaming configuration.
- Cribl handles all custom parsing and transformation logic.
- The S3 bucket acts as a durable buffer, providing resilience if Cribl is temporarily unavailable.
Supporting Documentation
- Streaming Pega logs to an external Amazon S3 bucket: Pegasystems Documentation
- Streaming Pega logs to Splunk: Pegasystems Documentation
- Understanding and obtaining Pega log files: Pegasystems Documentation "