Tag Archives: JMC

Java server application troubleshooting using JDK tools

by Mikhail Vorontsov

1. Introduction
2. Troubleshooting scenarios
    2.1. Getting a list of running JVMs
    2.2. Making a heap dump
    2.3. Analyzing a class histogram
    2.4. Making a thread dump
    2.5. Running Java Flight Recorder

1. Introduction

In the Java world most of us are used to GUI tools for all stages of Java application development: writing your code, debugging and profiling it. We often prefer to set up the server environment on our dev boxes and try to reproduce the problems locally using familiar tools. Unfortunately, it is often impossible to reproduce some issues locally for various reasons. For example, you may not be authorised to access the real client data which is processed by your server application.

In situations like this one you need to troubleshoot the application remotely on the server box. You should keep in mind that you can not properly troubleshoot an application with bare JRE in your hands: it contains all the troubleshooting functionality, but there is literally no way to access it. As a result, you need either a JDK or some 3rd party tools on the same box. This article will describe JDK tools, because you are likely to be allowed to use it on production boxes compared to any 3rd party tools which require security audit in many organizations.

In general case, it is sufficient just to unpack the JDK distribution to your box – you don’t need to install it properly for troubleshooting purposes (actually, proper installation may be undesirable in a lot of cases). For JMX based functionality you can install literally any Java 7/8 JDK, but some tools can not recognize the future JDK, so I advice you to install either the latest Java 7/8 JDK or the build exactly matching to server JRE – it allows you to dump the app heap for applications with no safepoints being accessed at the moment (some applications in the idle mode are the easy example of “no safepoints” applications).

2. Troubleshooting scenarios

2.1. Getting a list of running JVMs

In order to start working you nearly always need to get a list of running JVMs, their process IDs and command line arguments. Sometimes it may enough: you may find a second instance of the same application doing the same job concurrently (and damaging the output files / reopening sockets / doing some other stupid things).

Just run jcmd without any arguments. It will print you a list of running JVMs:

3824 org.jetbrains.idea.maven.server.RemoteMavenServer
2196
780 sun.tools.jcmd.JCmd

Now you can see what diagnostic commands are available for a given JVM by running jcmd <PID> help command. Here is a sample output for VisualVM:

>jcmd 3036 help

3036:
The following commands are available:
JFR.stop
JFR.start
JFR.dump
JFR.check
VM.native_memory
VM.check_commercial_features
VM.unlock_commercial_features
ManagementAgent.stop
ManagementAgent.start_local
ManagementAgent.start
GC.rotate_log
Thread.print
GC.class_stats
GC.class_histogram
GC.heap_dump
GC.run_finalization
GC.run
VM.uptime
VM.flags
VM.system_properties
VM.command_line
VM.version
help

Type jcmd <PID> <COMMAND_NAME> to either run a diagnostic command or get an error message asking for command arguments:

>jcmd 3036 GC.heap_dump

3036:
java.lang.IllegalArgumentException: The argument 'filename' is mandatory.

You can get more information about a diganostic command arguments by using the following command: jcmd <PID> help <COMMAND_NAME>. For example, here is the output for GC.heap_dump command:

>jcmd 3036 help GC.heap_dump
        
3036:
GC.heap_dump
Generate a HPROF format dump of the Java heap.

Impact: High: Depends on Java heap size and content. Request a full GC unless the '-all' option is specified.

Permission: java.lang.management.ManagementPermission(monitor)

Syntax : GC.heap_dump [options] <filename>

Arguments:
filename :  Name of the dump file (STRING, no default value)

Options: (options must be specified using the <key> or <key>=<value> syntax)
-all : [optional] Dump all objects, including unreachable objects (BOOLEAN, false)

Continue reading

Oracle Java Mission Control Overview

by Mikhail Vorontsov

Introduction

This article will describe the Java Mission Control – a JDK GUI tool (jmc / jmc.exe) available since Java 7u40. We will also discuss Java Flight Recorder – a surprisingly good JDK profiler with some features not available in any other project. Finally, we will look at JOverflow Analyzer – yet another semi-free tool (free for development, commercial for production), which allows you to analyze a lot of memory usage anti-patterns in your application based on a simple HPROF file.

Java Mission Control

Oracle Java Mission Control is a tool available in the Oracle JDK since Java 7u40. This tool originates from JRockit JVM where it was available for years. JRockit and its version of JMC were well described in a Oracle JRockit: The Definitive Guide book written by two JRockit senior developers (also visit the Marcus Hirt blog – the first place you should be looking for any JMC news).

Oracle JMC could be used for 2 main purposes:

  • Monitoring the state of multiple running Oracle JVMs
  • Java Flight Recorder dump file analysis

JMC license

Current JMC license (see “Supplemental license terms” here ) allows you to freely use JMC for development, but it requires the purchase of a commercial license if you want to use it in production (this is my personal opinion, I am not a lawyer 🙂 ). This means that you can avoid spending extra dollars if you have a proper QA process 🙂

JMC plug-ins

JMC offers a few plugins. You can install them via Help -> Install New Software menu (you may not know that plugins exist and never go there 🙁 ). Note that each plugin may have its own license, so be careful and read the licenses. I will give an overview of “JOverflow Analysis” plugin in this article – it looks for a list of inefficient memory usage patterns in your app heap.

Realtime process monitoring

You can attach to a JVM by right-clicking on it in the JVM Browser tab of the main window and choosing “Start JMX Console” menu option. You will see the following screen. There is nothing fancy here, just pay attention to the “+” buttons which allow you to add more counters to this screen.

Main monitoring screen

Main monitoring screen

Continue reading