Encryption in File Dataset

General
1

Hi Team,

We are trying to use encryption option along with gz encryption provided in file dataset to create encrypted files. This is used in a dataflow. One gz file is getting generated and that is getting encrypted properly. However, the other generated file .gz.meta is not getting encrypted. Is there any way to encrypt the meta file as well? Has anyone tried this?

2

You have specified that When using the encryption option along with gzip compression in a file dataset within a dataflow:

  • The main data file (.gz) is properly encrypted

  • The metadata file (.gz.meta) remains unencrypted by design

Root Cause

This is a documented limitation in the Pega Platform. The system is designed to only encrypt the primary data file but not the accompanying metadata file when using File Dataset with data protection/PGP encryption enabled.

Workarounds

Since there’s no built-in solution to encrypt the metadata files, you have several options:

  1. Implement External Encryption Post-Processing

    • Create a custom post-processing step that runs after the dataflow completes

    • Use external PGP/GPG tools or scripts to encrypt the .gz.meta file separately

  2. Evaluate if Metadata is Required

    • If downstream systems don’t need the metadata file, consider deleting it as part of post-processing

  3. Use Secure File Transfer

    • Implement secure file transfer protocols (SFTP, FTPS) that provide transport-level encryption

    • This protects the unencrypted metadata file during transmission

  4. Official Status

This limitation has been documented in previous support ticket for Platform version 24.2.3, confirming that Pega does not currently support encryption of metadata files. An enhancement request (FDBK-140403) has been created to add this capability in future releases.

You’ll need to contact your Pega AE if you wish to discuss this enhancement request further.