running the prometheus java agent results in constant stream of errors from prometheus-http-1-3

Pega-as-a-Service
, , , , , , ,
1

We’ve enabled the built in prometheus agents on our pega containers built from docker.io/pega-ready:latest and we’ve notice that we get the following constant error messages every minute when enabled.

2022-07-05 10:20:23,128 prometheus-http-1-3 ERROR Invalid URL pegajdbc://1977189075:0/adm7.jar!//log4j2.xml java.net.MalformedURLException: unknown protocol: pegajdbc

The java agent was enabled with the following environment variable

CATALINA_OPTS: "-javaagent:/opt/pega/prometheus/jmx_prometheus_javaagent.jar=9090:/opt/pega/prometheus/tomcat.yml"

When enabled, prometheus metrics are able to be scaped with curl http://localhost:9090/metrics

see attachments for tomcat.yml and the output from curl http://localhost:9090/metrics and the full error message

Regards
Byron

tomcat.yml_.txt (1.18 KB)

metrics.out_.txt (29 KB)

full-error.txt (4.13 KB)

2

@ByronC46 Quick check is your Pega application up and running?

3

@SriharshaAnika Yes it is. It is a Dev instance which isn’t utilised much but I can login to the platform.

4

@SriharshaAnika Thanks for that information. It was helpful. As we don’t require JMX support for Log4j we have disabled it and these messages are no longer flooding our logs.

log4j2.disable.jmx=true

Regards
Byron

5

@ByronC46 Can someone advise me how to scrap Pega application metrics( non cloud V7.X,8.X) to Prometheus

i have posted question in community but no solution has been provided yet

How to expose application metrics to monioring frameworks like Prometheus,Grafana | Support Center (pega.com)

6

@SrinivasK6050 @SriharshaAnika

Appreciate an update on above question

7

@ByronC46

Hi it has nothing to do with your configuration or any thing. it is a problem from the JMX bean exposed by apache log4j2.

by default JMX exporter reads all the MBean’s and its associated attributes and keep only the attributes that you need for viewing. part of this operation when apache log4j2 mbean attributes are being read , error is happening at ConfigLocationUri attribute (this is an editable) attribute and after reading that attribute, bean is trying to validate the information and landing in error.

So you can ignore that error.

JMX Apache bean issue.png
JMX Apache bean issue.png1257×480 58.7 KB

Thanks

8

@ByronC46 Could you please accept the solution and close the thread