JMX and Java Flight Recorder#

Monitor internal server metrics using JMX to inspect connection pools, thread pools, memory usage, and more. Use Java Flight Recorder for low-overhead performance profiling and diagnostics.

JMX#

Java Management Extensions (JMX) is a commonly used interface for monitoring the internals of a Java-based application like the Curity Identity Server. This ability to peer inside the application, however, can be dangerous. It is for this reason that JMX is disabled by default. To enable it, the ENABLE_JMX can be set before starting the Curity Identity Server; the value is ignored and can can be any non-empty value (e.g., true, 1, etc.). This can be done on the command line like this, for instance:

Example of how to enable JMX from the command line by setting the ENABLE_JMX environment variable

$ ENABLE_JMX=1 idsvr

With JMX enabled, the following can be monitored and, in some cases, changed:

  • LDAP connection pools
  • JDBC (database) connection pools
  • Web server information (thread pools, object pools, ciphers, etc.)
  • JVM settings (e.g., memory, CPU usage, etc.)
  • Logging settings (including log levels per logger)

Note that serialization must be enabled for the javax.management.* classes in order for JMX to function properly. This should be handled automatically in typical scenarios.

Java Flight Recorder#

The Curity Identity Server ships with support for Java Flight Recorder and Java Mission Control. These are branded, fully-tested builds of the open-source JDK Flight Recorder and JDK Mission Control which Oracle released in 2018. Coupled with the instrumentation and data collection performed by Flight Recorder inside of Curity, Java Mission Control provides monitoring and management possibilities with very little impact on Curity’s performance.

Java Mission Control can be downloaded from Adoptium’s Web site. Once it is installed and started, you can use it to attach to an instance of Curity if JMX is enabled (as described above). This will allow you to monitor, in real-time, such things as CPU, memory, threads, garbage collections, and much, much more.

You can also record the performance for later analysis. This can be very helpful in difficult support cases, for instance. This can be done in Java Mission Control or by using the jcmd command that is shipped with Curity. Using either is a two-step process:

  1. Start the recording
  2. Stop the recording and save the results to a file

Starting a Recording Manually#

To start a recording or to connect to a remote instance of Curity using Java Mission Control requires additional parameters to be provided when starting that instance. Refer to the Monitoring and Management section of the Java SE documentation for details of what these parameters are. As an example, running Curity in a local Docker container (which effectively makes it remote), it is possible to connect to it from Java Mission Control if it is started with additional parameters that can be passed using the JAVA_OPTS environment variable like this

Starting Curity in a Docker container with remote monitoring enabled:

$ docker run -it \
    -e ENABLE_JMX=1 \
    -e JAVA_OPTS="-Dcom.sun.management.jmxremote.ssl=false
        -Dcom.sun.management.jmxremote.authenticate=false
        -Dcom.sun.management.jmxremote.port=7091
        -Dcom.sun.management.jmxremote.rmi.port=7091
        -Djava.rmi.server.hostname=localhost
        -Dcom.sun.management.jmxremote.local.only=false" \
    -e PASSWORD=$PASSWORD \
    -p 6749:6749 \
    -p 7091:7091 \
    -p 8443:8443 \
    curity/idsvr:latest

The example above is provided only for demonstration, and more secure options should be used in production.

With remote access enabled, a connection can be made using Java Mission Control by selecting the Connect… menu option in the File menu, or by selecting New Connection from the context menu of the JVM Browser, or by clicking the New Connect button from the toolbar shown in the figure below:

Creating a new connection to a remote instance of Curity
fig 1:Creating a new connection to a remote instance of Curity

Whichever method is used, the following dialogue page will be shown:

JMX connection modal in Java Mission Control
fig 2:JMX connection modal in Java Mission Control

Clicking the Test connection button should briefly flicker a status dialogue box and then the Status field should be changed to OK. If so, click Finish. Otherwise, tweak the settings used when starting the instance of Curity and refer to the Java Monitoring and Management documentation for additional guidance and troubleshooting tips. Baring any connectivity issues, a new JVM should be shown in JVM Browser treeview:

JVM Browser showing new connection to Curity instance
fig 3:JVM Browser showing new connection to Curity instance

If the MBean Server child node of this new connection is clicked, a dashboard is shown, like that shown below:

Dashboard showing memory, JVM CPU, remaining heap memory, and other aspects of the running instance of Curity
fig 4:Dashboard showing memory, JVM CPU, remaining heap memory, and other aspects of the running instance of Curity

To start a recording in Java Mission Control, right-click the Flight Recorder item in the JVM Browser treeview under the established connection, as shown in fig 3, and then select Start Flight Recording…. The following dialogue will be shown:

Starting a recording of the performance of an instance of Curity using Java Mission Control
Starting a recording of the performance of an instance of Curity using Java Mission Control

Select Finish to accept the defaults or make adjustments on the current or subsequent pages as required.

Starting a Recording from the Command Line#

When shell access to the machine running Curity is available, an alternative to start a recording is to use jcmd. With this tool, you need the process ID of the instance of the Curity Identity Server to be profiled or you can use the package name of its bootstrapper. Both alternatives work but only the latter is shown in the following listings:

$ jcmd se.curity.identityserver.app.Bootstrapper JFR.start

The jcmd command is included in various versions of the JDK, but is not directly provided by Curity. The various options and subcommands supported by this tool can be found in the jcmd documentation.

After the JFR.start subcommand is run, it will print the command necessary to dump a snapshot of analysis data. It will be something like this:

$ jcmd 59896 JFR.dump name=1 filename=FILEPATH # where 59896 is the process ID of Curity

The state of recording can be checked using the JFR.check subcommand with either the process ID or the bootstrapper package name like this:

$ jcmd se.curity.identityserver.app.Bootstrapper JFR.check

If recording is not currently underway, this command will provide the instructions on how to start one.

When the dump subcommand is run, the recording will be captured in the specified file. At that point, the file can be opened in Java Mission Control or other tools that support the JDK Flight Recorder format.

Dumping the recording to a file does not stop the recording. To do that, use the JFR.stop subcommand. This also accepts a filename and name parameter like JFR.dump does except that it stops the recording and the filename parameter is optional. If the filename is not provided, then a dump will not be simultaneously made. An example of stopping a recording named 1 is shown in the following listing:

$ jcmd se.curity.identityserver.app.Bootstrapper name=1 filename=/tmp/recording_1.jfr

Starting a Recording on Startup#

It is also possible to start a recording by providing certain command line options to the Curity Identity Server. This can be done in various ways. For example, this can be achieved by configuring the JVM options . A better way typically though is to set the JAVA_OPTS environment variable to include the parameters necessary to start the recording when the Curity Identity Server starts. Either way, the parameters will be something like these:

-XX:StartFlightRecording=filename=my-good-file.jfr,duration=10m

For information about available flags that can be passed when starting a recording, refer to the flight recorder command reference.

Using either Java Mission Control, command line options provided to the Curity Identity Server, or jcmd, the resulting file can be analyzed and potentially shared with support. This will give a lot of insight into the source of potential issues. For more information on Flight Controller and Mission Control, refer to the following sources:

Was this helpful?