Monitoring Weblogic using JMX in SiteScope

28 February, 2008 – 2:45 pm

Using JMX (Java Management Extensions) it is possible to monitor Managed Beans published by MBean servers. Weblogic (WL) provides MBeans for both admin and managed servers.

You can find out more info about WebLogic MBeans and accessing them with JMX here …

Following are some augmented instructions for setting up JMX monitors within SiteScope. I used this excellent article as a starting point.

Types
There are two types of MBeans which we’re interested in namely:

1. Configuration MBeans, which expose attributes and operations
for configuring WebLogic Server resources.
2. Runtime MBeans, which provide information about the runtime state of its resources.

In terms of performance monitoring we’re more interested in the latter, but from an overall system level monitoring point of view, sysadmins may also be interested in static config MBeans.

In reality there are 1,000’s of runtime MBeans available for WebLogic and navigating/finding the correct bean inside SiteScope can be difficult due to it’s clunky web interface …

As a starting point refer to the MBean reference previously mentioned to get an overview of runtime beans available. In this case I’m interested in the JMSRuntime bean …

Using the WL Scripting Tool
If you are working on the WL server you can use the WL Scripting Tool to browse the available Runtime beans:
D:\Data\bea\wlserver_10.0\server\bin>setWLSEnv.cmd
D:\Data\bea\wlserver_10.0\server\bin>java weblogic.WLST
connect('username','password','t3://host:8003')
serverRuntime()
ls()
cd("JMSRuntime")
ls()
cd("server01_01.jms")
ls()
 
dr-\-   Connections
dr-\-   JMSServers
 
-r-\-   ConnectionsCurrentCount                      14
-r-\-   ConnectionsHighCount                         14
-r-\-   ConnectionsTotalCount                        18
-r-\-   HealthState                                  State:HEALTH_OK,ReasonCode:\[\]
-r-\-   JMSServersCurrentCount                       2
-r-\-   JMSServersHighCount                          2
-r-\-   JMSServersTotalCount                         2
-r-\-   Name                                         server01_01.jms
-r-\-   Type                                         JMSRuntime

This will help identify the name of the MBean (in this case server01_01.jms) and the indicative path
to that MBean when using JConsole or SiteScope.

Setting up IIOP access on monitored machines
SiteScope relies on the IIOP protocol to talk to the WL server. In order to get remote access with JMX via IIOP you need to provide a default username and password for the IIOP user.

This can be done via the WL admin console as in:
domServiceBus->Environment->Servers->server01_nn->Protocols [tab]\->IIOP->advanced

Then provide a default username and password for the IIOP user.

Note: IIOP is already enabled by default, but a username and pwd is not. This change requires a server restart for WL. You must make sure that the username and password is >= 8 digits, otherwise the CORBA connection will fail when using SiteScope.

Testing connectivity from JConsole
Before setting up a SiteScope monitor it is worth using JConsole to familiarize yourself with the published MBeans.
Assuming you have the Java SDK installed, launch JConsole as per the following:
C:\Program Files\Java\jdk1.5.0_14\bin\jconsole.exe \-J-Dcom.sun.CORBA.transport.ORBTCPReadTimeouts=10:60000:500:10

Note: you must provide the additional read timeout arguments otherwise you will get a connection error like this:
Unable to retrieve counters: Error receiving browse data: java.rmi.MarshalException: CORBA COMM_FAILURE 1398079696 Maybe; nested exception is: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 208 completed: Maybe

To connect to the target server, click on New Connection->advanced and provide the following details:
JMX URL: service:jmx:rmi:///jndi/iiop://hostname:8003/weblogic.management.mbeanservers.runtime
Username: <whatever you set for the IIOP user>
Password: <whatever you set for the IIOP user>

You will then get a tree-like broswer in which you can look at all available MBeans.

Setting up a SiteScope JMX monitor
Before setting up the monitor, make sure you change the following registry key for SiteScope to avoid the aforementioned timeout errors:
@="-server -Xmx256m -showversion -Dcom.sun.CORBA.transport.ORBTCPReadTimeouts=10:30000:500:10
-Dcatalina.home=\"D:\\SiteScope\\Tomcat\" -Dflipper.param.file=..\\conf\\flipperSiteScope.properties
-Dtopaz.home=.. -cp \"D:\\SiteScope\\Tomcat\\bin\\bootstrap.jar\" org.apache.catalina.startup.Bootstrap start"

Note the added parameter argument:
-Dcom.sun.CORBA.transport.ORBTCPReadTimeouts=10:30000:500:10

You will need to stop/start the SiteScope windows service for this change to take affect.

Now create a new JMX monitor:
Name: 8003 - Managed server
Frequency: 2 minutes
JMX URL: service:jmx:rmi:///jndi/iiop://host:8003/weblogic.management.mbeanservers.runtime
Domain filter:
Username: weblogic
Password: password

Then click Get Counters …

If you used the WLST you’d now know what path the MBean exists on. In this case:
com.bea/server01_01/server01_01.jms/JMSRuntime

And suggested counters to collect might be:
ConnectionsCurrentCount
ConnectionsHighCount
ConnectionsTotalCount
JMSServersCurrentCount
JMSServersHighCount
JMSServersTotalCount

Now you will be able to use SiteScope as the central repository for collecting metrics (long term monitoring). When running a load test scenario, you will be able to poll these same metrics using the SiteScope monitor available to the Controller.