Cutting GenAI Latency by 75% in Pega with Jev

[kaspk (Kajal Kaspate)]

In this article, I’ll show how to integrate Jev, TypeSafe’s newest type-safe model ( Introducing System One Models & Jev - TypeSafe AI Blog ) into a Pega process and make efficient use of its speed and structured output.

I called Jev through its REST API. Getting access is simple – just follow the quickstart instructions here:

​Call it: the API - Quick start - TypeSafe AI

In this example, I call the Pega GenAI connector and the Jev API in parallel to classify the same email, then compare the results. The goal is to assign the email to one of eight categories: Add Product, Additional Order, Cancel Order, Change Shipping Address, Line-Item Removal, Quality Change, RDD Change and Replace Product.

For my example, I used the following customer email, written in German:

Guten Tag,
 
Es ist ein direkte Nachfolgerartikel, aber von anderem Hersteller.
82721 - Hellmann's Dressing Balsamico 6x1L Flasche
 
Mit freundlichen Grüßen,

Using the Pega GenAI connector, I received the following response.

This request took 2 seconds and consumed 507 tokens in total.

2026-09-28 20:51:36,943 [itions=[0,1,2,3,4,5]] [  STANDARD] [                    ] [                    ] (Provider_Pega_PegaGenAI.Action) 
{choices=[ChatCompletionResponseChoice{finishReason='stop', message=com.pega.platform.genai.models.ChatCompletionMessage@4b572d1d}], usage=ChatCompletionResponseUsage{promptTokens=402, completionTokens=105, totalTokens=507}, message='null', httpStatusCode=200, created=0, model='bedrock/anthropic/Claude-Haiku-45/1.0', object='null'} 

The same request through the Jev REST API took just 469 ms.

2026-09-28 21:00:17,631 ] [  STANDARD] [                    ] [   Complain:01.01.01] (.internal.ComponentsHttpClient) INFO  lab-24348-eu-central-1|127.0.0.1|Rest|application|v2|assignments_var_actions_var062d43e11e85a35e2846a51b53f5d001|A1KCLT9HZYZ7A6SOXDC3QC434BDEQ9MW2A|Rule-Connect-REST.Classify-ClassifyEmailAPI.ClassifyEmail HXLGBFPNEKYHPA1NAA0YIRI8G5SK0MPJ6A - Finished connector client invocation in 469 ms 

It processed 414 tokens in total.

Sample output from Jev looks like below

{
    "model": "jev-1.13.0",
    "answers": {
        "bug_severity": {
            "type": "score",
            "score": 6.25,
            "confidence": 0.63,
            "legend": {
                "0": "Add Product",
                "1": "Additional Order",
                "2": "Cancel Order",
                "3": "Change Shipping Address",
                "4": "Line-Item Removal",
                "5": "Quality Change",
                "6": "RDD Change",
                "7": "Replace Product"
            },
            "probabilities": {
                "0": 0.07,
                "1": 0.02,
                "2": 0.0,
                "3": 0.0,
                "4": 0.0,
                "5": 0.06,
                "6": 0.01,
                "7": 0.84
            }
        }
    },
    "usage": {
        "input_tokens": 393,
        "output_tokens": 20
    }
}

This Json response can be used for various purposes. In my example I am showing it on the screen for the users to make a decision.

Performance-wise, the gain is significant: with Jev, processing time dropped by more than 75% (from 2 seconds to 469 ms) for a comparable number of tokens. Jev also returns structured responses, so the output can be consumed directly in the Pega process without extra parsing of free-form text.

What makes this especially interesting is that Pega isn’t tied to any single state-of-the-art (SOTA) model. You can plug any model into the architecture and realise productivity gains. Jev can complement the LLM for faster decision making, routing, classification or saftey checks. (Ref: What Is Jev? A Guide to TypeSafe AI’s System One Model )

[Tim_Straatsma]

Great read @kaspk !
I was wondering what makes JEV fundamentally different from other LLM models? Which types of things can it do in my process?

[kaspk (Kajal Kaspate)]

The fundamental difference is between how the JEV models create the output and how they are trained. JEV is non-generative model and trained using RLCD which is Reinforcement Learning for Calibrated Decisions rather than RLHF (RL with Human feedback). It is faster because it processes the whole input in a single pass rather than token-by-token which most LLMs do. It returns the structured, typed decisions with confidence score and probabilities.

[LEWIA (Andy Lewis)]

My thoughts…

First Thoughts on Jev (Part 1): From Vibes to Hunches

Jev is the hottest discussion in town right now - a new class of AI model that combines the large, unstructured input of a LLM with the short, sharp output of a if-then-else rule. It has a simple purpose: to make decisions based on a single question given a mass of textual data. Taking just hundreds of miliseconds and costing a fraction of a cent to execute, it moves away from the heavyweight and costly approach of using LLMs to decide on true or false type questions. My colleague explaimed whi…

[Peter_van_der_Putten]

To add, in AI terms JEV is a zero-shot classifier/regressor, which is quite different from an LLM (although LLMs can also be used for classification tasks). Let me explain these terms and why they matter.

Classification

Unlike an LLM, JEV does not generate text or other forms of content (summaries, answers, videos of corgis, etc.). Instead, it predicts labels, probabilities, or numerical values for a given input. The input can consist of a range of attributes and values, and attribute values may themselves contain free text.

To be more precise it can do the following tasks:

• Binary scoring/classification: Predict the likelihood of a binary outcome. For example, given inbound email text and customer tenure, what is the likelihood that this customer will churn? (Although customer churn is probably not a good use case for JEV, as I explain below.)
• Multiclass scoring/classification: Predict the likelihood of each class from a predefined set of labels. For example, given inbound email text, customer attributes, and a list of possible intents, what is the likelihood of each intent?
• Numeric regression: Predict a continuous numerical outcome given a set of inputs. For example, given a particular agent state and a proposed tool call, what is the likelihood that the tool call is safe?

The JEV documentation and communications use some terminology that may be confusing to AI practitioners. For example, they state that JEV produces a “decision”, whereas it is more accurate to say that it produces a prediction or classification. This distinction matters because an automated decision may also incorporate business rules, additional models, constraints, policies, and other logic. In JEV terminology, binary classification/scoring is called “noul”, multiclass classification/scoring is called “choice”, and regression is called “score” prediction.

Zero Shot

This is arguably the more interesting aspect. JEV is a foundation model, which means that, like LLMs, it can perform tasks without additional task-specific training or fine-tuning.

Note the word additional. JEV has of course been trained. The idea is that it has learned sufficiently general representations to generalize to new tasks, much like LLMs and other foundation models. It is also generally faster and cheaper than using an LLM for classification-style tasks.

The flip side is that when sufficient labelled training data is available, a task-specific model that is many orders of magnitude smaller than JEV (potentially containing only tens, hundreds, or thousands of parameters rather than billions) can still outperform it. There is also a requirement that the inputs be semantically meaningful. Attribute names and values should carry meaning that a general-purpose model can understand, rather than simply being arbitrary variables such as x, y, and z.

Use cases criteria

So where could ‘decision models’ like JEV, Laya, Kev, Nimble etc be useful? Some potential criteria are:

• Situations where there is little or no training data available, or where obtaining training data and expertise would be difficult (for example, agentic harness verification tasks, but probably not customer churn prediction).
• Problems that can be expressed as a single classification, probability estimate, or numerical prediction.
• Data that is semantically meaningful and understandable without extensive task-specific context.
• Use cases that require a single prediction within a few hundred milliseconds (whereas LLMs may take several seconds).
• Use cases that do not require making tens or hundreds of predictions per decision or per second, where traditional predictive models would likely remain more efficient.
• True decisioning use cases that go beyond just applying a single classification should not be built end-to-end with a decision model, but decision models could feed into a decision engine (if latency is less of an issue, decision models are still typically slower than regular predictive models and decision engines

Note many alternatives to JEV have been popping up within days after its release, and the field of tabular foundation models has been focussing on zero shot classification models already for a couple of years. JEV is not unique in that sense. Nor is it a silver bullet.

[kaspk (Kajal Kaspate)]

Thank you for adding this Peter! Very insightful!

[Murali.Krishnan.D1 (Murali Krishnan Divakaran)]

Very interesting comparison. Thanks for sharing @kaspk

[Tim_Straatsma]

That’s a great addition Peter, I really like your framing on when this type of model would add value, and when not!