Recommended Approach for Masking Prompts Over 20,000 Characters in Pega GenAI Connect

I’m implementing Pega GenAI Connect with Mask sensitive data enabled. According to the documentation, the Text Analyzer processes up to 20,000 characters by default. Content beyond that limit may not be analyzed, which could leave pyGenAIPIIdetector unmasked before the prompt is sent to the LLM.

Our prompts can exceed this limit, so we need a reliable way to mask sensitive data throughout the entire prompt. Increasing the limit may be an option, but we’re concerned about the impact on Text Analyzer performance, overall response time, and GenAI Connect timeouts.

What approach does Pega recommend for this scenario?

  1. Is there an out-of-the-box capability for masking prompts longer than the Text Analyzer’s character limit?
  2. If custom chunking is needed, what supported approach should we use to run entity extraction and masking on each chunk?
  3. How should we handle entities that cross chunk boundaries, such as a person’s full name?
  4. How can we maintain consistent placeholders across chunks and preserve any mapping needed to restore original values in the response?
  5. Should we reassemble the masked chunks into one prompt before invoking GenAI Connect, or send them in separate calls? What are the performance and timeout trade-offs?

Any recommended implementation patterns, supported extension points, or relevant documentation would be appreciated.

Thanks in advance for your guidance!:slightly_smiling_face:

Currently, there is no out-of-the-box chunking capability available in GenAI Connect. If your prompt exceeds the Text Analyzer processing limit (20,000 characters by default), you should carefully consider the overall token size and whether the entire content needs to be sent to the LLM.

As a general recommendation, keep the System Prompt focused on static instructions, guardrails, and behavior definitions, while using the User Prompt for dynamic business content and runtime context. To better understand the scenario, are you observing any specific issues when the prompt exceeds 20,000 characters? For example:

  • Does masking stop working for content beyond the 20,000-character limit?
  • Are entities after the limit not being detected by the Text Analyzer?
  • Are you seeing incomplete masking, performance degradation, timeouts, or any other unexpected behavior?

Understanding the observed behavior will help determine the most appropriate solution and whether the limitation is related to entity extraction coverage, masking execution, prompt size, or overall processing constraints.

For additional guidance on prompt design and the distinction between System and User Prompts, please refer to: System Prompt vs User Prompt: Building Secure Enterprise GenAI and Agentic AI Applications in Pega.