Possible improvements to Connect GenAI request/response configuration

AI
1

I wanted to bring two possible improvements related to the Connect GenAI rule, particularly around the request and response configuration, to the attention of the Pega product team.

1. Support for complex structured responses

Currently, for response mapping, we have options such as Structured – Single and Structured – List.

image
image1270×724 80.4 KB

But what if the expected response contains a combination of both?

For example, consider extracting invoice information:

  • Invoice date
  • Vendor details
  • Line items[]- a structured list containing multiple records

This is still a structured response, but the structure is more complex than either a single object or a simple list.

Using Unstructured doesn’t feel like the right fit here because the expected output has a clearly defined structure.

Would it be useful to support a Complex Structured response, perhaps with a configuration experience similar to a JSON Data Transform, where we could define nested objects and lists within the response structure?

I feel this could make Connect GenAI much more flexible for use cases where the LLM is expected to return richer structured data.

2. More flexibility for Document AI attachment processing

Document AI uses Connect GenAI, where we can specify an attachment field as the input. That works well when we simply want to process an attachment, but I think there could be value in providing more control over which attachments should be processed.

For example, could we support filtering based on attachment category or document type, allowing certain attachment types to be included or excluded?

Going one step further, I wonder whether Document AI could benefit from a more specialized GenAI configuration/rule type, where the attachment category and expected response structure are more closely associated.

For example:

Identity Document
→ Name, date of birth, document number, expiry date, etc.

Invoice
→ Invoice date, vendor, total amount, line items, etc.

The expected extraction structure is naturally different depending on the type/category of document being processed.

Having the ability to associate an attachment category with its own extraction/response structure could make Document AI configurations cleaner, more flexible, and easier to govern.

Interested to know whether these are already being considered on the product roadmap, or whether there are recommended patterns today for handling these scenarios.

I also feel it would be interesting to have a dedicated thread around new features, where Pega professionals working on real projects can share what may be missing or could be improved based on practical experience. This kind of real-time feedback from the field could help surface practical gaps early and give them the right attention from the product team.

Interested to hear what others think and whether you have come across similar requirements in your projects.

2

Hi Prem,

Regarding the unstructured mapping, the most common scenario is nested embedded structure and we use JSON data transform to map the JSON to clipboard. As part of the below article demo, I’m extracting the Policy Holder Details, Claimants (PageList), Claimant details (PageList) .

Policy Holder Details (Page)

   Claimants (PageList)

   Claim Details (PageList)
3

Thanks for the response, Ramesh. Yes, I acknowledge that using an Unstructured response + JSON Data Transform can handle this scenario.

My question is more around the product design behind the response definition in Connect GenAI.

My understanding is that when GenAI Connect was initially introduced, Unstructured was primarily useful for genuinely unstructured outputs. For example, generating a summary paragraph and mapping that response to a single-value property.

But the examples we are discussing here aren’t really unstructured responses. For example:

Policy Holder Details (Page)
→ Claimants (PageList)
→ Claim Details (PageList)

Similarly, an invoice could contain:

Invoice Details (Page)
→ Invoice number
→ Invoice date
→ Total amount
→ Line Items (PageList)

These are still structured responses, they simply have a more complex/nested structure.

There is also another advantage to defining the response as structured within Connect GenAI. When we use the structured response configuration, Pega takes responsibility for constructing more of the expected output structure in the request, including the response-format guidance/guardrails and example-based prompting.

I actually demonstrated this difference in a video I published today:
Connect GenAI video - structured response mapping

With the Unstructured approach, much more responsibility moves to the developer - we need to instruct the model to produce the correct JSON structure, ensure the necessary constraints are included in the prompt, and then parse/map that JSON back into the appropriate clipboard structure.

So I completely agree that Unstructured + JSON Data Transform is a workable solution today. My suggestion is more that a few additional OOTB response configuration options in Connect GenAI particularly support for nested/complex structured responses containing Pages and Page Lists together could make this scenario much cleaner and allow developers to benefit from more of Pega’s built-in structured-response handling rather than implementing that contract themselves.

That was the main point I was trying to highlight.

4

On the other hand, I must also mention, the unstructured response configuration in the Connect-GenAI do contribute for the Connect-GenAI reusability with prompt templates as input and unstructured response as output, so that the response mapping is handled outside of the Connect-GenAI making it more reusable!