Load-balancer plugin in GlassFish now supports Apache 2.2

Support for Apache 2.2 will be coming in GlassFish Server v2.1. Users can download this version of GlassFish server from GlassFish download page. The enterprise edition will even contain installer support for installing load-balancer plugin on Apache 2.2.

For the benefit of GlassFish user, steps to manually configure load-balancer plugin with Apache2.2 is provided in this blog.

Platform supported

Solaris SPARC
Solaris x86
Linux

For exact version of the platform supported, please refer to release notes of GlassFish Server 2.1

NOTE : Apache 2.2 in not supported on windows right now. However Apache 2.0.x is supported on windows.

Download Location

The load-balancer plugin library can be downloaded from http://download.java.net/javaee5/external/<OS>/aslb/jars where OS can be SunOS, SunOS_X86 and Linux.

The latest version of load-balancer plugin available is aslb-9.1.1-b5.jar. User can download any other latest version compared to above version, if available.

Steps to configure load-balancer plugin on Apache2.2

Build Apache 2.2 and install it. If user wants to use auto-apply feature, then it should be build with openssl and server certificate must be installed. You can refer to Apache2.0 documentation for building Apache2.2 with ssl and installing certificate. Use --with-included-apr option when building Apache 2.2 to build bundled apr. Apache 2.2 install directory will be referred to as <apache2.2-install-dir>.
Download load-balancer plugin and unjar it. Then unzip SUNWaspx.zip and SUNWaslb.zip. This directory will be referred to as <lbplugin-unzip-dir>.
Create directory

<apache2.2-install-dir>/modules/errorpages
<apache2.2-install-dir>/modules/resource
<apache2.2-install-dir>/sec_db_files

Copy file - <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/mod_loadbalancer.so to <apache2.2-install-dir>/modules. Also change permission of the file to executable.
Copy files - <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/errorpages/default-error.html and <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/errorpages/sun-http-lberror.html to <apache2.2-install-dir>/modules/errorpages directory.
Copy files - <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/LBPluginDefault_root.res and <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/LBPlugin_root.res to <apache2.2-install-dir>/modules/resource directory.
Copy files - <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/secmod.db, <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/cert8.db and <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/key3.db to <apache2.2-install-dir>/sec_db_files directory.
Copy file <lbplugin-unzip-dir>/lib/install/templates/loadbalancer.xml.example to <apache2.2-install-dir>/conf directory.
Copy files - <lbplugin-unzip-dir>/lib/dtds/sun-loadbalancer_1_1.dtd and <lbplugin-unzip-dir>/lib/dtds/sun-loadbalancer_1_2.dtd to <apache2.2-install-dir>/conf directory.
Changes in file <apache2.2-install-dir>/conf/httpd.conf

Change
FROM:
ServerName www.example.com:80
TO:
ServerName <host-name>:80
Change
FROM:
#Include conf/extra/httpd-mpm.conf
TO:
Include conf/extra/httpd-mpm.conf
Change
FROM:
#Include conf/extra/httpd-vhosts.conf
TO:
Include conf/extra/httpd-vhosts.conf
Change
FROM:
#Include conf/extra/httpd-ssl.conf
TO:
Include conf/extra/httpd-ssl.conf
Append following entry
##BEGIN EE LB Plugin Parameters
LoadFile /usr/lib/libCstd.so.1 (THIS LINE IS ONLY NEEDED ON SOLARIS SPARC)
LoadModule apachelbplugin_module modules/mod_loadbalancer.so
#AddModule apachelbplugin_module
<IfModule apachelbplugin_module>
config-file <apache2.2-install-dir>/conf/loadbalancer.xml
locale en
</IfModule>
##END EE LB Plugin Parameters

Changes in file <apache2.2-install-dir>/conf/extra/httpd-mpm.conf
1. Change prefork MPM settings
  Set StartServers to 1
  Set MaxClients to 1
Changes in file <apache2.2-install-dir>/conf/extra/httpd-vhosts.conf
1. Append
  ##BEGIN EE LB Plugin Parameters
  <VirtualHost <machine-ip-address>>
  ServerName <host-name>
  DocumentRoot "<apache2.2-install-dir>/htdocs"
  </VirtualHost>
  ##END EE LB Plugin Parameters
2. User can remove other existing VirtualHost entries from this file
Changes in file <apache2.2-install-dir>/conf/extra/httpd-ssl.conf
1. Change
  FROM:
  <VirtualHost _default_:443>
  TO:
  <VirtualHost <machine-ip-address>:443>
  b)Change
  FROM:
  ServerName www.example.com:443
  TO:
  ServerName <host-name>:443
If user want to use feature auto-apply feature
1. Export DAS certificate using command : certutil -L -d <application-server-DAS-dir>/config -n s1as -a -o sjsas.crt
2. Copy certificate file(sjsas.crt) to <apache2.2-install-dir>/conf/ssl.crt directory. If directory does not exist, create one.
3. Append in file <apache2.2-install-dir>/conf/extra/httpd-ssl.conf
  ##BEGIN EE LB Plugin Parameters
  <Location /lbconfigupdate >
  SSLVerifyClient require
  SSLVerifyDepth 1
  SSLRequireSSL
  SSLCACertificateFile <apache2.2-install-dir>/conf/ssl.crt/sjsas.crt
  SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)-/ \
  and %{SSL_CLIENT_S_DN_O} eq "Sun Microsystems" \
  and %{SSL_CLIENT_S_DN_OU} eq "Sun Java System Application Server" \
  and %{SSL_CLIENT_M_SERIAL} eq "<Cert-Serial-Number>" )
  </Location>
  <Location /getmonitordata >
  SSLVerifyClient require
  SSLVerifyDepth 1
  SSLRequireSSL
  SSLCACertificateFile <apache2.2-install-dir>/conf/ssl.crt/sjsas.crt
  SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)-/ \
  and %{SSL_CLIENT_S_DN_O} eq "Sun Microsystems" \
  and %{SSL_CLIENT_S_DN_OU} eq "Sun Java System Application Server" \
  and %{SSL_CLIENT_M_SERIAL} eq "<Cert-Serial-Number>" )
  </Location>
  ##END EE LB Plugin Parameters
  
  NOTE:
  1. Cert-Serial-Number need to be generated from provided DAS certificate file(sjsas.crt). Command for that is : keytool -printcert -file sjsas.crt. Note Serial number from output of above command. Take its value and change all lower case characters to upper case.
  2. Application Server name is assumed to be Sun Java System Application Server. If it is different, then it needs to be reflected here as well. Above command will print that as well.
Changes in file <apache2.2-install-dir>/bin/envvars
1. Suffix <lbplugin-unzip-dir>/lib and <apache2.2-install-dir>/modules directory to LD_LIBRARY_PATH on unix platform
For auto-apply feature to work, the run-as user for apache must have writable permission for <apache2.2-install-dir>/conf directory. If security is not an issue, user can even set <apache2.2-install-dir>/conf directory permission to 777.

Posted on: Nov 20, 2008

Posted by: kshitiz

Category: Sun

Permanent link to this entry

Comments:

Thanks for the post. Things were going well but when I try to start apache, the mod_loadbalancer.so tries to load libssl3.so which doesn't appear to exist anywhere on my system.

Its an ubuntu 8.04 installation with apache2.2 installed as an ubuntu package including ssl and using aslb-9.1.1-b10.jar.

I've tried googling but I can't find anything that tells me how to get this .so. Any suggestions?

Posted by Julian Paas on June 12, 2009 at 07:44 AM PDT #

You can run command "locate libssl3.so" to verify if file exists on your system. If it exists then add path to LD_LIBRARY_PATH.
If it does not exist then you can download Sun GlassFish Enterprise Server and install it. It has those libraries bundled.

However load-balancer plugin is not tested on ubuntu. It may have issues running on that platform.

Thanks,
Kshitiz

Posted by Kshitiz on June 12, 2009 at 08:35 AM PDT #

background:
app-server-home = /opt/glassfish
and
i assume DAS home is /opt/glassfish/domains/domain1

first issue step 14: The command ###certutil -L -d /opt/glassfish/domains/domain1/config/ -a -o sjas.crt generates an error message
certutil: function failed: security library: bad database.
with an empty sjas.crt file
Am i exec in the wrong path or is it a configuration issue

second issue having read through the apache log file a noticed a couple of errors
### ERROR:NSS could not be initialized; The issue may be missing security DB files under /opt/apache2/sec_db_files; ...Aborting Plugin initialization ... -- i verified the files all 3 of them are there but the cert file is cert8.db(aslb-9.1.1-b10.jar) instead of cert7.db, renaming didn't help
and lastly
###[error] server reached MaxClients setting, consider raising the MaxClients setting....

is this to be expected having reduced the maxclients to 1

regards,

Posted by tshepiso on June 17, 2009 at 02:26 AM PDT #

Thanks Kshitiz. I was able to get all the .so files I needed. But in the end it fails with an error inside glibc. I had a hard time resolving the dependency on libstdc++-libc6.2-2.so.3. The only way I was able to do so was to download libstdc++2.10-glibc2.2_2.95.4-27_i386.deb and install it manually. I suspect that in the end it failed because this library was i386 architecture and my server is x64.

Part of the reason I used ubuntu is because it is declared as officially supported by glassfish. http://docs.sun.com/app/docs/doc/820-4329/abpak?a=view

But I guess since it is only supported as a developer platform that implies no load balancing.

Posted by Julian Paas on June 17, 2009 at 05:31 AM PDT #

Please post your query to either forum - http://forums.java.net/jive/forum.jspa?forumID=56 or send query to alias dev@glassfish.dev.java.net

Thanks,
Kshitiz

Posted by KSHITIZ SAXENA on June 17, 2009 at 08:10 AM PDT #

I've managed to get the entire stack working with Solaris x86 and apache 2.2.11, there are a few tricks and annoyances:

1) Setting Maxclients to 1 will result in the web server only being able to handle.... yup, 1 client at a time. Given that most web browsers pipeline 4+ concurrent requests to a server (esp if static content is getting served up via the same apache instance), this is more or less worthless. We had to set the values up to something a bit more sane, like 50. By setting the StartServers, MinSpareServers and MaxSpareServers to the same value, you avoid having new instances constantly come up and default to sending requests to the first server listed. The downside is that all of these workers will ping the server if health checking is enabled. (one could go with the worker mpm, but things like php throw warnings about how apache and threads are not well tested and that you will need to proceed at your own risk.

2) for glassfish, the default setting for the max number of threads is 5... again, great for a developer, but two orders of magnitude less that what is required for anything else.

3) the aslb plugin logs debug information at the 'alert' and 'crit' level, resulting in the error log getting loaded up with 7 lines of stuff for every request that is not going to the LB plugin. If you set the LogLevel high enough to actually avoid these, you then miss out on actual errors from the aslb which are logged at the 'warn' level

4) the aslb is linked against dozens of unused libraries (it probably needs one small function from libns-httpd40.so but that lib requires, i kid you not, 22 other shared libs that are outside of the usual system library path). If you also want to use php with ldap or ldap auth for other apps outside of the aslb, load the aslb module _last_ or you will end up in shared library hell.

--john

Posted by John on June 19, 2009 at 03:29 PM PDT #

Hello everybody,

Just want to share with you.
I got the lb-plugin working on Ubuntu 8.04 (2.6.24-24-generic) by using
apache-httpd-2.2.11
openssl-0.9.7g
glassfish2.1

In addition to the procedure explained by Kshitiz, I carried out the following steps.

1 Export the loadbalancer.xml from GF Admin Console: "HTTP-Load Balancer -> lb1 -> Export" to $httpd_home/conf

2. Open $httpd_home/bin/envvars and append

$glassfish_home/lib/webserver-plugin/linux/apache2.2 // contains all important files of load balancer plugin
$sjsws_home/lib // contains libsasl.so

$httpd_home/modules // contains mod_jk.so and mod_loadbalancer.so
into LD_LIBRARY_PATH, as shown below.

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$httpd_home/lib:/opt/httpd2.2.11/modules
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$glassfish_home/lib:$glassfish_home/lib/webserver-plugin/linux/apache2.2:$sjsws_home/lib
export LD_LIBRARY_PATH

SJSWS = Sun Java System Web Server 7

3 Create a security certificate for apache. See Sun GlassFish Enterprise Server 2.1 High Availability
Administration Guide (820-4341)

My special thanks to Kshitiz for your installation and configuration procedure.

Posted by rpimn on June 25, 2009 at 03:15 PM PDT #

On both Solaris 10 x86 and Fedora Core 10 x86 I have been unable to get the LB plugin to function.

Both platforms crash with a signal 11 with every request.

Has anyone seen this, or know how to fix it?

I'm using aslb-9.1.1-b11.jar on both platforms.

Posted by Sean McG on July 15, 2009 at 09:42 PM PDT #

I will request all of you to please post your query to either forum - http://forums.java.net/jive/forum.jspa?forumID=56 or send query to alias dev@glassfish.dev.java.net. That is correct place to issues resolved.

I hope you are not using 64 bit version of apache. It has been well tested on Solaris 10 x86, and should work fine.

Thanks,
Kshitiz

Posted by Kshitiz on July 15, 2009 at 11:03 PM PDT #

Hi kshitiz,
I have the problem

[alert] ERROR:NSS could not be initialized; The issue may be missing security DB files under /opt/apache/sec_db_files; Please ensure that secmod.db, key3.db and cert7.db files are present under /opt/apache/sec_db_files; Refer documentation for more details; Aborting Plugin initialization ...

I am with conditions;
1- Sun DAS is running on different machiine than apache
2- Ubuntu
3- Apache2.0.x

followed these instructions;
http://blogs.sun.com/sprabhu7/entry/install_configure_lb_on_apache#1

these way arounds arent resolving the problem;
http://www.nabble.com/Apache-2.2-%2B-mod_loadbalancer.so-%3D-solved-td22538976.html

Please help. [or private email]
r a o h a m m a d @ gmail. co m

Posted by Hammad on August 09, 2009 at 01:40 AM PDT #

Please post your query to either forum - http://forums.java.net/jive/forum.jspa?forumID=56 or send query to alias dev@glassfish.dev.java.net. That is correct place to issues resolved.

Please check your nss version. Use command
1. ident /usr/lib/libnss3.so
2. nss-config --version

Load-balancer plugin does not work with version 3.12.x. However it works well with 3.11.x.

Thanks,
Kshitiz

Posted by Kshitiz on August 10, 2009 at 03:08 AM PDT #

I'm wondering if you have an ETA on Apache 2.2 support under Windows? This is extremely important to me as I have a module I need to use that only works under 2.2. It seems 2.2 for Windows has been out for a while now so is there some tricky technical issue holding up the plugin from supporting 2.2? Any info would be extremely helpful and I can go down other, less desirable paths if I need to so please provide an honest assessment of if/when this support will arrive. Thank you.

Posted by darkwolf on September 03, 2009 at 12:09 PM PDT #

Truly speaking I do not have any data on when this support will come. This will even depend on Oracle acquisition of Sun. I will update the thread when I have any information on this.

Meanwhile please try using other platform, i.e., Solaris/Linux.

Thanks,
Kshitiz

Posted by Kshitiz on September 06, 2009 at 05:27 AM PDT #

While of course not what I would have liked to hear I am still happy that you provided me a fast and honest assessment so that I can determine how best to proceed. Thank you .

Posted by darkwolf on September 06, 2009 at 09:09 AM PDT #

Kshitiz Saxena: Kshitiz's Blog