JMX to Prometheus exporter: a collector that can configurable scrape and expose mBeans of a JMX target.

This exporter is intended to be run as a Java Agent, exposing a HTTP server and serving metrics of the local JVM. It can be also run as a standalone HTTP server and scrape remote JMX targets, but this has various disadvantages, such as being harder to configure and being unable to expose process metrics (e.g., memory and CPU usage).

Running the exporter as a Java agent is strongly encouraged.

• jmx_prometheus_javaagent-0.19.0.jar

To run as a Java agent, download one of the JARs and run:

java -javaagent:./jmx_prometheus_javaagent-0.19.0.jar=12345:config.yaml -jar yourJar.jar

Metrics will now be accessible at http://localhost:12345/metrics. To bind the java agent to a specific IP change the port number to host:port.

A minimal config.yaml looks like this:

rules:
- pattern: ".*"

Example configurations can be found in the example_configs/ directory.

• jmx_prometheus_httpserver-0.19.0.jar

To run the standalone HTTP server, download one of the JARs and run:

java -jar jmx_prometheus_httpserver-0.19.0.jar 12345 config.yaml

Metrics will now be accessible at http://localhost:12345/metrics. To bind the java agent to a specific IP change the port number to host:port.

The standalone HTTP server will read JMX remotely over the network. Therefore, you need to specify either hostPort or jmxUrl in config.yaml to tell the HTTP server where the JMX beans can be accessed.

A minimal config.yaml looks like this:

hostPort: localhost:9999
rules:
- pattern: ".*"

As stated above, it is recommended to run JMX exporter as a Java agent and not as a standalone HTTP server.

./mvnw clean package to build.

The configuration is in YAML. An example with all possible options:

---
startDelaySeconds: 0
hostPort: 127.0.0.1:1234
username: 
password: 
jmxUrl: service:jmx:rmi:///jndi/rmi://127.0.0.1:1234/jmxrmi
ssl: false
lowercaseOutputName: false
lowercaseOutputLabelNames: false
includeObjectNames: ["org.apache.cassandra.metrics:*"]
excludeObjectNames: ["org.apache.cassandra.metrics:type=ColumnFamily,*"]
autoExcludeObjectNameAttributes: true
excludeObjectNameAttributes:
  "java.lang:type=OperatingSystem":
    - "ObjectName"
  "java.lang:type=Runtime":
    - "ClassPath"
    - "SystemProperties"
rules:
  - pattern: 'org.apache.cassandra.metrics<type=(\w ), name=(\w )><>Value: (\d )'
    name: cassandra_$1_$2
    value: $3
    valueFactor: 0.001
    labels: {}
    help: "Cassandra metric $1 $2"
    cache: false
    type: GAUGE
    attrNameSnakeCase: false

Metric names and label names are sanitized. All characters other than [a-zA-Z0-9:_] are replaced with underscores, and adjacent underscores are collapsed. There's no limitations on label values or the help text.

A minimal config is {}, which will connect to the local JVM and collect everything in the default format. Note that the scraper always processes all mBeans, even if they're not exported.

Example configurations for javaagents can be found at https://github.com/prometheus/jmx_exporter/tree/master/example_configs

NOTE

Both whitelistObjectNames and blacklistObjectNames are still supported for backward compatibility, but should be considered deprecated.

The format of the input matches against the pattern is

domain<beanpropertyName1=beanPropertyValue1, beanpropertyName2=beanPropertyValue2, ...><key1, key2, ...>attrName: value

No escaping or other changes are made to these values, with the exception of if attrNameSnakeCase is set. The default help includes this string, except for the value.

Examples

java.lang<type=Memory><HeapMemoryUsage>used: 914251736
java.lang<type=Memory><HeapMemoryUsage>max: 1610612736
java.lang<type=Threading><>thread_count: 140
java.lang<type=Threading><>peak_thread_count: 158
java.lang<name=G1 Old Generation, type=GarbageCollector><>CollectionTime: 0
java.lang<name=G1 Young Generation, type=GarbageCollector, key=CodeHeap 'profiled nmethods'><LastGcInfo, memoryUsageAfterGc>used: 26476672

To help us identity these, Numero added a patternMatchingString= to the HELP string of metrics. It contains the string that JMX Exporter matches our patterns againts.

The default format will transform beans in a way that should produce sane metrics in most cases. It is

domain_beanPropertyValue1_key1_key2_...keyN_attrName{beanpropertyName2="beanPropertyValue2", ...}: value

If a given part isn't set, it'll be excluded.

HTTP BASIC authentication supports using the following configuration algorithms:

• plaintext - plaintext password
• SHA-1 - SHA-1(<salt>:<password>)
• SHA-256 - SHA-256(<salt>:<password>)
• SHA-512 - SHA-512(<salt>:<password>)
• PBKDF2WithHmacSHA1
• PBKDF2WithHmacSHA256
• PBKDF2WithHmacSHA512

Plaintext example:

httpServer:
  authentication:
    basic:
      username: Prometheus
      password: secret

SHA-256 example using a salted password SHA-256(<salt>:<password>) with a password of secret

httpServer:
  authentication:
    basic:
      username: Prometheus
      passwordHash: 2bf7ed4906ac065bde39f7508d6102a6cdd7153a929ea883ff6cd04442772c99
      algorithm: SHA-256
      salt: U9i%=N m]#i9yvUV:bA/3n4X9JdPXf=n

PBKDF2WithHmacSHA256 example with a password of secret

httpServer:
  authentication:
    basic:
      username: Prometheus
      passwordHash: A1:0E:4E:62:F7:1E:0B:59:0A:32:EA:CC:7C:65:37:1F:6D:A6:F1:F1:ED:3F:73:ED:C9:65:19:37:21:5B:6D:4E:9D:C6:61:DF:B5:BF:BB:16:B8:9A:50:14:57:CE:3D:14:67:73:A3:71:1B:87:3B:C4:B1:0E:DC:2D:0B:10:65:D6:F5:B6:DA:07:DD:EE:DA:AC:9C:60:CD:B4:59:0C:C9:CB:A7:3D:7E:30:3E:43:83:E9:E4:13:34:A1:F1:87:5C:24:46:8E:13:90:A6:66:E1:A6:F3:0B:5A:E7:14:8A:98:6A:81:2B:B6:F8:EF:95:D4:82:7E:FB:5E:2D:D3:24:FE:96
      algorithm: PBKDF2WithHmacSHA256
      salt: U9i%=N m]#i9yvUV:bA/3n4X9JdPXf=n

• iterations = 600000 (default value for PBKDF2WithHmacSHA256 )
• keyLength = 128 bits (default value)

Notes

• PBKDF2WithHmacSHA1 default iterations = 1300000
• PBKDF2WithHmacSHA256 default iterations = 600000
• PBKDF2WithHmacSHA256 default iterations = 210000
• default keyLength = 128 (bits)

• sha1sum, sha256sum, and sha512sum can be used to generate the passwordHash
• openssl can be used to generate a PBKDF2WithHmac based algorithm passwordHash

HTTPS support can be configured using either a JKS or PKCS12 format keystore via two possible methods:

• Exporter YAML configuration

• System properties

Notes

• Keystore type is dependent on the Java version

• Exporter YAML configuration overrides System properties

1. Add configuration to your exporter YAML file

httpServer:
  ssl:
    keyStore:
      filename: localhost.jks
      password: changeit
    certificate:
      alias: localhost

2. Create a keystore and add your certificate

1. Add configuration to your exporter YAML file

httpServer:
  ssl:
    certificate:
      alias: localhost

2. Add your certificate to the application's Java keystore

The exporter YAML file alias should match the certificate alias of the certificate you want to use for the HTTPS server.

3. Define the application system properties for the Java keystore

-Djavax.net.ssl.keyStore=<keystore file> -Djavax.net.ssl.keyStorePassword=<keystore password>

The exporter thread pool can be configured via the exporter YAML file.

By default, a maximum of 10 threads is used.

httpServer:
  threads:
    minimum: 1
    maximum: 10
    keepAliveTime: 120 # seconds

• minimum - minimum number of threads
• maximum - maximum number of threads
• keepAliveTime - thread keep-alive time in seconds

Notes

• If the work queue is full, the request will be blocked until space is available in the work queue for the request.

The JMX exporter uses the AntuBLUE Test Engine and Testcontainers to run integration tests with different Java versions.

You need to have Docker installed to run the integration test suite.

Build and run the integration test suite:

./mvnw clean verify

Notes

• To run the integration tests in IntelliJ, you must build first the project from the parent (root) using Maven

  • The Maven build copies the core artifacts as resources to the integration_tests project

• Additional information can be found in the Integration Test Suite readme.

You can start the JMX scraper in standalone mode in order to debug what is called

git clone https://github.com/prometheus/jmx_exporter.git
cd jmx_exporter
./mvnw package
java -cp collector/target/collector*.jar  io.prometheus.jmx.JmxScraper  service:jmx:rmi:your_url

To get finer logs (including the duration of each jmx call), create a file called logging.properties with this content:

handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=ALL
io.prometheus.jmx.level=ALL
io.prometheus.jmx.shaded.io.prometheus.jmx.level=ALL

Add the following flag to your Java invocation:

-Djava.util.logging.config.file=/path/to/logging.properties

A Debian binary package is created as part of the build process and it can be used to install an executable into /usr/bin/jmx_exporter with configuration in /etc/jmx_exporter/jmx_exporter.yaml.