jmx remote using JmxRemoteLifecycleListener

needed to monitor tomcat, and specifically tomcat behind a firewall.

The problem here is that JMX uses two ports, one for the RMI registry and another to export JMX RMI connection objects to which a client is expected to connect into.

• However, the second port is random which of course is a pain given we can’t open all ports on the firewall then Zabbix Java Agent can’t connect to the RMI port on the monitored host.

to overcome this we need to use JmxRemoteLifecycleListener and specify the secondary port, which can then be permitted in the firewall.

to make this work, locate the server.xml  (eg /etc/tomcat6/server.xml) and add the following line

<Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener" rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002"/>

next on the Tomcat startup script (eg /etc/tomcat6/tomcat6.conf)

if you are on a network where the internal DNS name is different to that of the public, add the following

-Djava.rmi.server.hostname=your.public.dns

this address can be either a hostname or IP address

Problems starting Apache

When starting Apache, received this error:

Invalid command 'ExtendedStatus', perhaps misspelled or defined by a module not included in the server configuration

Reason in the httpd conf file, LoadModule status_module modules/mod_status.so was commented out.

uncommenting and restarting the service now allow apache to start.

hint  – to confirm the configuration file, use httpd -t

Tomcat & remote access

I’m using Centos for our production application servers, running Tomcat6.

now the application are working, need to monitor their state – in this example, the servers are all on the same network – see note on JmxRemoteLifecycleListener when using firewalls
exampl

So need to allow remote connections to the tomcat servers.
Default Configuration data is in file –  /etc/tomcat6/tomcat6.conf

Required basic Parameters

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=xxxxx
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false

just add the the above into the conf file and restart the tomcat service. eg

CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10001 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"

however, this isn’t exactly secure…

to achieve this, add the following lines

-Dcom.sun.management.jmxremote.authenticate=true \
-Dcom.sun.management.jmxremote.password.file=$CATALINA_BASE/conf/jmxremote.password \ 
-Dcom.sun.management.jmxremote.access.file==$CATALINA_BASE/conf/jmxremote.access \

• create the password and access files if they don’t exist

example contents for jmxremote.access :

monitorRole readonly 
controlRole readwrite

Apache Server Status

Apache allows us to grab usage statistics

example

Apache Server Status for 192.168.1.80
Server Version: Apache/2.2.15 (Unix) DAV/2 PHP/5.3.3
 Server Built: Feb 13 2012 22:31:42
 Current Time: Tuesday, 26-Feb-2013 17:25:26 GMT
 Restart Time: Tuesday, 26-Feb-2013 17:25:00 GMT
 Parent Server Generation: 0
 Server uptime: 26 seconds
 Total accesses: 19 - Total Traffic: 6 kB
 CPU Usage: u0 s0 cu0 cs0
 .731 requests/sec - 236 B/second - 323 B/request
 1 requests currently being processed, 7 idle workers

The default location for these is http:/localhost/server-stats

Enable server stats

to enable change the apache httpd.conf (/etc/httpd/conf/httpd.conf)

1. Enable line:
ExtendedStatus On

2. uncomment server stats section :

SetHandler server-status
Order deny,allow
Allow from 127.0.0.1
Deny from all


• amend the above to suit local requirements.

check the change config file:

[root]# apachectl configtest

Syntax OK

3. once changed remember to restart the http server :service httpd restart

So, now we’ve grabbed the data, what can we do with it?    Using wget, we can grab the data and pass that onto a monitoring system for subsequent trend or alerting.

I’m using Zabbix to monitor which when configured gives me this example graph:

Method

to achieve this,  choose a method from the Zabbix wiki :

For simplicity, i’m using Method 3

Apache Server

1. down load the script & template from above wiki link.
2. copy the shell script to required path – here i’m using /etc/zabbix/externalscripts
3. on the machine running apache, append to the agent conf file

UserParameter=apache[*],/etc/zabbix/externalscripts/zapache.sh  \$1

4. Restart the zabbix agent : service zabbix-agent restart

Zabbix Server

a good tip before enabling importing any template and subsequent triggers is on the zabbix server use zabbix_get  to check the configuration works.

eg zabbix_get -s [servername] -p 10050 -k apache[Uptime]

1. import the download template
2. assign the template to the apache server
3. check for incoming data/errors etc.
4. create any required graphs

Easy way to Install Tomcat In Centos

Installing Tomcat just takes a few minutes.

Tomcat requires java so install that followed by tomcat.

Method: open terminal, sudo as root type the following commands:

yum -y install java
yum -y install tomcat6 tomcat6-webapps tomcat6-admin-webapps

the start tomcat

/etc/init.d/tomcat6 start

to make it start automatically

chkconfig --add tomcat6
chkconfig tomcat on 235
chkconfig --list tomcat6
chkconfig --list tomcat6
tomcat6 0:off 1:off 2:on 3:on 4:on 5:on 6:off

To connect to Tomcat, browse to port 8080 on the server:

http://[serverip]:8080