JMX Job

Overview

JMX (Java Management Extensions) is an industry-standard technology for monitoring and managing Java applications. Java applications instrumented with JMX expose a set of resources called MBeans (Management Beans) with attributes and methods that can be queried and invoked programmatically by the user.

The JMX job in Axibase Collector provides a way to query MBean attribute values from remote Java applications and transmit them to Axibase Time Series Database for alerting and long-term retention. MBean attribute values can be stored as properties or series in case of numeric attributes.

The JMX job can have one or multiple JMX configurations each describing connection parameters and MBean queries. Configurations in a given job commonly connect to the same Java application.

JMX Configuration

Connection Parameters

Field	Description
Name	Job configuration name.
Item List	A collection of elements to execute multiple requests to JMX service in a loop. The current element in the loop can be accessed with `${ITEM}` placeholder. When Item List is selected and `${ITEM}` is present in one of the fields, the job executes as many queries as there are elements in the list, substituting `${ITEM}` with element value for each request. The placeholder `${ITEM}` supports standard functions and column function.
Host	Hostname or IP address of the remote server running Java application with JMX service.
Port	JMX service port.
User Name	JMX username.
Password	JMX password.
Service Name	JMX service username. The default service name is `jmxrmi`.

Connection parameters must correspond to the com.sun.management.jmxremote settings specified by the target Java application.

ACTIVEMQ_SUNJMX_START="-Dcom.sun.management.jmxremote \
   -Dcom.sun.management.jmxremote.port=1090 \
   -Dcom.sun.management.jmxremote.rmi.port=1090 \
   -Dcom.sun.management.jmxremote.ssl=false \
   -Djava.rmi.server.hostname=NURSWGVML011 \
   -Dcom.sun.management.jmxremote.password.file=${ACTIVEMQ_BASE}/conf/jmx.password \
   -Dcom.sun.management.jmxremote.access.file=${ACTIVEMQ_BASE}/conf/jmx.access"

To verify connectivity with the remote host, click Test or Viewer.

Add hostname to the /etc/hosts file directory on the Collector machine in case of UnknownHostException.

Failed to retrieve RMIServer stub: javax.naming.ConfigurationException [Root exception is java.rmi.UnknownHostException: Unknown host: NURSWGVML011; nested exception is: java.net.UnknownHostException: NURSWGVML011]

Query Parameters

Field	Description
Entity	Entity name under which the database stores the data.
Predefined Tags	Predefined tags added to all commands, one `name=value` pair per line. This field supports the following placeholders: - `${domain}` = Domain of the MBean - `${name}` = Value of `name` attribute of the MBean
Command Type	Insert command type: `SERIES`, `PROPERTY` or `BOTH`.
Metric Prefix	Common prefix added to metric names, for example `jmx.activemq.` This field supports the following placeholders: - `${domain}` = Domain of the MBean - `${attribute_name}` = Value of attribute of the MBean The attributes are excluded from series tags, if the attributes are included using a placeholder.
Property Type Prefix	Prefix added to property type, for example `jmx.activemq.` Property type is set to MBean `type` attribute by default. This field supports the following placeholders: - `${domain}` = Domain of the MBean - `${attribute_name}` = Value of attribute of the MBean The attributes are excluded from property keys, if the attributes are included using a placeholder.
Excluded Property Attributes	List of attribute names excluded from property commands.
Excluded Series Attributes	List of attribute names excluded from series commands.

Excluded Property and Series Attributes support * wildcards, for example: DynamicDestinationProducers_*, QueueProducers_*.

Entity Name

If not specified by the user, the entity name is set to the value of the Host field by the database. The user can override this feature, for example, if the Host field contains a DNS hostname, but the user needs to collect data under a short hostname of the server where the Java application is running.

In addition, the entity name can be retrieved dynamically by specifying the MBean Object Name followed by > and the attribute name (mbean>attribute). For example:

java.lang:type=Runtime>SystemProperties.java.rmi.server.hostname.value

If the entity name query fails to provide a value or if the bean or attribute is not found, the entity name is set to the Host field. If the composite expression retrieves a value successfully, Axibase Collector database stores and reuses the value in case of subsequent connection errors.

Queries

Configuration includes a list of MBean queries consisting of two parts: object name pattern and attribute list.

The list of queries can be entered manually or by opening Viewer, expanding MBean hierarchy, and choosing attributes of interest.

Both parts of the query support wildcards:

Asterisk * replaces any number (including zero) of characters.
Question mark ? replaces any one character.

Object Name Pattern

The pattern pattern matches MBean instances of specified type and name.

org.apache.activemq:brokerName=localhost,type=Broker   -->     TotalProducerCount, TotalMessageCount

Negation is not supported by JMX natively and is therefore implemented on the client. To exclude MBean use != operator. For example, to exclude topics starting with underscore:

kafka.cluster:name=*,partition=*,topic!=_*,type=Partition

Example	Description
`*:type=Foo,name=Bar`	matches names in any domain whose exact set of keys is `type=Foo,name=Bar`.
`d:type=Foo,name=Bar,*`	matches names in the domain `d` that have the keys `type=Foo,name=Bar` plus zero or more other keys.
`:type=Foo,name=Bar,`	matches names in any domain that has the keys `type=Foo,name=Bar` plus zero or more other keys.
`d:type=F?o,name=Bar`	matches for example `d:type=Foo,name=Bar` and `d:type=Fro,name=Bar`.
`d:type=F*o,name=Bar`	matches for example `d:type=Fo,name=Bar` and `d:type=Frodo,name=Bar`.
`d:type=Foo,name="B*"`	matches for example `d:type=Foo,name="Bling"`. Wildcards are recognized inside quotes and can be escaped with `\`.
`d:type=Foo,name!=B*`	matches names that does not start with `B`, for example `d:type=Foo,name=Store`.

Attribute Name List

The list specifies attributes whose values are retrieved by the application and sent to the database.

The list can include specific names as well as name patterns.

You can specify the list of collected attributes by replacing specific attribute names with wildcards. For example, to collect all numeric attributes from MBean java.lang:*,type=GarbageCollector, specify * in the corresponding attribute field.

Object Name Pattern	Attribute Name List
`java.lang:*,type=GarbageCollector`	`CollectionCount, CollectionTime`
`java.lang:*,type=GarbageCollector`	`CollectionCount, LastGcInfo.*`
`java.lang:type=Memory`	`HeapMemoryUsage*`

Special processing for PROPERTY command:

Attribute Name list is ignored for property commands since property commands collect all attributes of the matched beans.
The default value for property type field is set to MBean type, for instance, to Broker in the example above. To override the default type, enter a custom value or use ${attribute_name} placeholders in the property_type column.

Metric Prefix Example

Metric Prefix:

jmx.${domain}.${name}

The attribute name is excluded from series tags, because the Metric Prefix field stores the name attribute using placeholder:

Item List Example

The kafka-cluster-jmx list contains items (host,port,entity):

172.17.0.1,1099,kafka1
172.17.0.2,1099,kafka2
172.17.0.3,1099,kafka3

Each row is split by FreeMarker function to extract values:

JMX Configuration Example

Viewer

Click Viewer to open a tree-based MBean navigator displaying available Management Beans and their attributes.

Select a checkbox next to an attribute name to add to the list of collected attributes. Modify the expression by replacing the specific Object Name fields with wildcards, if necessary.

JMX Viewer

Ignored MBean Attributes

The following MBean attributes are ignored from Viewer and the commands:

Attribute value cannot be obtained due to a processing error:
- UnsupportedOperationException
- UnmarshalException
- ReflectionException
- RuntimeOperationsException
- InstanceNotFoundException
Attribute value is NaN (Not a Number) for numeric attributes in case of a series command.
NaN can occur, for example, on division by zero.

To view attributes ignored due to processing errors, enable debugging for the MBeansInfoExtractor class:

<logger name="com.axibase.collector.model.jmx.MBeansInfoExtractor" level="DEBUG">
    <appender-ref ref="logRoller"/>
</logger>

Restart Axibase Collector process, execute the JMX job, or open the Viewer.

Review the log:

tail -f ./axibase-collector/logs/axibase-collector.log

# JMX Job

# Overview

# JMX Configuration

# Connection Parameters

# Query Parameters

# Entity Name

# Queries

# Object Name Pattern

# Attribute Name List

# Metric Prefix Example

# Item List Example

# JMX Configuration Example

# Viewer

# Ignored MBean Attributes