You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Scott Tatum <sc...@wcom.com> on 2001/04/06 20:19:49 UTC
Apache+Tomcat+vhosts insights, discussion (long)
I too had the utilization problems associated with Apache and Tomcat
3.2.1. Actually, the problem also manifests itself in Tomcat standalone.
After researching the problem and eventually coming up with a solution,
I realized a couple of things that I have not seen posted to the list,
so I thought I would share them and possibly help some others out.
First of all let me explain my setup. Our group does a lot of Intranet
applications for different groups and we have recently moved from JRun 3
to using Tomcat 3.2.1 for our servlet container. And I must say, even
though the initial setup and administration was tedious, Tomcat is a
faster, more stable environment for us. Our development environment is a
PC running RH6.2, and our production boxes are usually running some
version of Solaris, and in this particular case, Solaris 8. Both setups
are using Apache1.3.19+Tomcat 3.2.1 and Java SDK 1.3.0_02. We run
separate VM's for each virtual host/application.
Our production boxes typically hold several production websites, each
with their own virtual host. The name of the production box itself
doesn't host a site, just the CNAME's we have registered for that host.
For example, the production server may be called pserv, with aliases
app1, app2, app3, etc. We don't have (or want) a website called
pserv.wcomnet.com (wcomnet.com is our Intranet domain), but we do have
the websites app1.wcomnet.com, app2.wcomnet.com, etc. I don't know how
representative this is of people using Apache+Tomcat and virtual hosts,
but it is a good fit for us, and Tomcat handles it well.
The configuration files look something like this:
-workers.properties-
worker.list=app1-ajp12, app2-ajp12, app3-ajp12
worker.app1-ajp12.port=8000
worker.app1-ajp12.host=localhost
worker.app1-ajp12.type=ajp12
worker.app1-ajp12.lbfactor=1
worker.app2-ajp12.port=8001
worker.app2-ajp12.host=localhost
worker.app2-ajp12.type=ajp12
worker.app2-ajp12.lbfactor=1
worker.app3-ajp12.port=8001
worker.app3-ajp12.host=localhost
worker.app3-ajp12.type=ajp12
worker.app3-ajp12.lbfactor=1
-uriworkermap.properties-
/*.jsp=app1-ajp12
/servlet/*=app1-ajp12
/*.jsp=app2-ajp12
/servlet/*=app2-ajp12
/*.jsp=app3-ajp12
/servlet/*=app3-ajp12
-mod_jk.conf-
JkWorkersFile /usr/tomcat/conf/workers.properties
JkLogFile /usr/apache/logs/mod_jk.log
JkLogLevel error
NameVirtualHost 1.1.1.1 # ip's changed to protect the innocent
<VirtualHost 1.1.1.1>
ServerName app1.wcomnet.com
DocumentRoot /usr/tomcat/vhosts/app1
ErrorLog /usr/tomcat/vhosts/app1/WEB-INF/logs/error_log
CustomLog /usr/tomcat/vhosts/app1/WEB-INF/logs/access_log common
JkLogFile /usr/tomcat/vhosts/app1/WEB-INF/logs/mod_jk.log
JkMount /servlet/* app1-ajp12
JkMount /*.jsp app1-ajp12
<Location "/WEB-INF/">
AllowOverride None
deny from all
</Location>
</VirtualHost>
<VirtualHost 1.1.1.1>
ServerName app2.wcomnet.com
DocumentRoot /usr/tomcat/vhosts/app2
ErrorLog /usr/tomcat/vhosts/app2/WEB-INF/logs/error_log
CustomLog /usr/tomcat/vhosts/app2/WEB-INF/logs/access_log common
JkLogFile /usr/tomcat/vhosts/app2/WEB-INF/logs/mod_jk.log
JkMount /servlet/* app2-ajp12
JkMount /*.jsp app2-ajp12
<Location "/WEB-INF/">
AllowOverride None
deny from all
</Location>
</VirtualHost>
<VirtualHost 1.1.1.1>
ServerName app3.wcomnet.com
DocumentRoot /usr/tomcat/vhosts/app3
ErrorLog /usr/tomcat/vhosts/app3/WEB-INF/logs/error_log
CustomLog /usr/tomcat/vhosts/app3/WEB-INF/logs/access_log common
JkLogFile /usr/tomcat/vhosts/app3/WEB-INF/logs/mod_jk.log
JkMount /servlet/* app3-ajp12
JkMount /*.jsp app3-ajp12
<Location "/WEB-INF/">
AllowOverride None
deny from all
</Location>
</VirtualHost>
-server-app1.xml-
# most of this is left out, only the custom bits listed
<Logger name="tc_log"
verbosityLevel = "INFORMATION"
path="/usr/tomcat/vhosts/app1/WEB-INF/logs/tomcat.log"
/>
<Logger name="servlet_log"
path="/usr/tomcat/vhosts/app1/WEB-INF/logs/servlet.log"
verbosityLevel = "INFORMATION"
/>
<Logger name="JASPER_LOG"
path="/usr/tomcat/vhosts/app1/WEB-INF/logs/jasper.log"
verbosityLevel = "INFORMATION" />
<ContextManager debug="0" workDir="/usr/tomcat/vhosts/app1/WEB-INF/work"
show
DebugInfo="true" >
<Connector
className="org.apache.tomcat.service.PoolTcpConnector">
<Parameter name="handler"
value="org.apache.tomcat.service.connector.Ajp12ConnectionHandler"/>
<Parameter name="port" value="8000"/>
</Connector>
<Host name="app1.wcomnet.com">
<Context path="/" docBase="/usr/tomcat/vhosts/app1" />
</Host>
I'll exclude server-app2.xml and server-app3.xml for brevity's sake.
They are the same as app1's xml file, but with the different directory
names and appropriate ports for the Connectors. I'm not sure if the Host
directive is even necessary. In a standalone Tomcat setup it would be,
but I think the mod_jk.conf is handles that now. I have them in there
just in case.
I'm sure some of you are already wondering why I didn't use the webapps
directory. There are a couple of reasons for that. We have several
different developers, each working on their particular applications. I
wanted to have the work directory (the auto-compiled servlets from jsp)
for a given application located under the webapp's directory structure.
That way developers who are mapping drives via Samba can see/manipulate
the generated servlet code along with everything else. I could have done
that through the ContextManager directive while keeping it under
webapps, but that produces some anomalies:
- Tomcat automatically looks in the webapps dir and creates contexts for
everything there. I already have my contexts setup through my config
files, I don't need an extra set. I believe it would allow access to
app1.wcomnet.com through pserv.wcomnet.com/app1, which I definitely
don't want!
- Tomcat automatically creates work directores for ALL webapps in the
webapps dir in every ContextManager-specified directory. So if I have 8
webapps, each with their own server.xml and defined work directory, each
one of those directories now gets _9_ subdirectories, even though 8 of
them aren't used (1 for the vhost and 8 for the contexts it
auto-configured from them being in webapps). I wish there was some way
to stop this behavior in Tomcat through the configuration.
So I created a vhosts directory and moved everything under there.
Everything works great now. The only downside is I get "No apps in
webapps/" each time I start a Tomcat instance. No adverse effects.
Also, I'm not using ajp13 right now, mainly because I read that with
ajp13, if you restart Tomcat, you must restart Apache. This is not good
for us. It would mean if one developer wants to restart his/her Tomcat
to effect a class file change, all the Tomcats would have to be
restarted, disrupting the other developers. Even worse, with several
applications in production, if one needs to be brought down for an
emergency patch, it would mean notifying all users not just for the one
application, but for all applications being served through the virtual
hosts. When/if ajp13 supports reconnecting to Apache I will definitely
move to it. Does anyone know if this will ever happen? I understand that
ajp13 keeps the socket open for speed reasons, but there has to be some
way to do that and provide a mechanism for reconnecting. I'm anxious to
move to it. I'll need SSL before long! :)
With this setup, everything was working great. Until I noticed one day
that the utilization was really high when it shouldn't be. The offending
process was the Tomcat process for app1. If I tried to restart that
process, all the httpd processes would gang up on the processer and
really send the utilization soaring. At first I thought it was a problem
with app1, until I was watching the access logs and the utilization went
up when no one was accessing app1.
In catching up on my usual huge backlog of tomcat-user messages, I came
upon talks of the utilization issue with 3.2.1. I decided this was the
problem. After some more investigating, I was able to make the request
that would spike the utilization and keep it there.
It was easy, just go to http://pserv.wcomnet.com/
So, obviously, the problem was that there was no configuration for pserv
(which isn't used). But first, I tried to figure out why it was always
the app1 process that was spiking. I moved things around in my
workers.properties and uriworkermap.properties but to no avail. I tried
starting the servers in a different order, but app1 would always spike.
I eventually figured out that it was spiking on app1 because app1 has
the first VirtualHost directive in mod_jk.conf. At first I thought I
would have to have another Tomcat running for the pserv name, something
I didn't want to waste the resources on. Well, I found a way around it.
I just made this the first VirtualHost directive in the mod_jk.conf
file:
<VirtualHost 1.1.1.1>
ServerName pserv.wcomnet.com
DocumentRoot /usr/apache/htdocs
</VirtualHost>
All I have in that document root is an index.html that says this is a
production box basically. The group discussions seemed to indicate that
I had to properly configure a root context in Tomcat to avoid this
error. But this setup keeps the utilization problem from happening. If a
request is made to a server, and that name is not configured in Apache,
then it forwards the request on to the first VirtualHost listed in the
conf file. Tomcat apparently gets confused by the differing server
names, and enters the CPU loop. Putting a VirtualHost entry first in the
list without any Tomcat directives stops this problem. Even requests to
pserv.wcomnet.com/somefile.jsp just return a 404 since there are no
JkMount directives for that vhost.
So there is my perspective on the utilization issue. Like I said, I
don't know how representative this configuration is, but I hope someone
else benefits from what I've discovered.
In addition, though the packaged documentation had everything I needed,
it was spread across several files and made it frustrating initially. I
think a separate, concise guide on setting up virtual domains would do a
world of good for those going this route (which I'm betting is a lot).
One that provides the configuration steps for Tomcat standalone,
Apache+Tomcat, as well a section on separate VM's for each host/app. It
should deal only with mod_jk - the docs were mixed mod_jk and mod_jserv,
and this only added to the confusion.
I would be more than willing to write this documentation, as I have
setup several servers now using each of these configurations. I'm
experimenting with ajp13 now, so I would probably have a section on
using ajp12 vs. ajp13. The one thing I wouldn't be able to write on is
using SSL as I haven't done that yet. Would anyone be interested in
this? I see a LOT of mails posted to the group asking about
Apache+Tomcat+vhosts.
If there is enough interest, I will go ahead and start on the
documentation, and if the Tomcat people don't want to include it I can
throw it up on my website. I almost need to to it just for myself, to
make it easier the next time I have to configure a server. :) The
initial version would be configuring for unix, though if I actually do
it, I'll do some setups on Windows boxes and add documentation for that
platform.
-Scott
(p.s. I hope my directives don't get screwed up by the less than's and
greater than's!)
--
Scott Tatum | scott.tatum@wcom.com
Senior Applications Developer, Special Projects
WorldCom | http://www.wcom.com/