You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "Martin Registe, Jr." <ma...@appia.com> on 2012/03/21 19:47:07 UTC
Setup of workers.properties with multiple host and TC instances
Hi All,
Tomcat 6.0.29 with Apache 2.2.17 running on windows 2003R2
Currently the workers.properties files is setup like this and seems to be working fine.
Current Setup
worker.list=part
#worker.list=worker1, worker2
#
# Defining a worker named ajp13w and of type ajp13
# Note that the name and the type do not have to match.
#
worker.jvm1.type=ajp13
worker.jvm1.host=10.17.75.145
worker.jvm1.port=8019
worker.jvm1.lbfactor=10
worker.jvm2.type=ajp13
worker.jvm2.host=10.17.75.145
worker.jvm2.port=8029
worker.jvm2.lbfactor=10
worker.jvm1.type=ajp13
worker.jvm1.host=10.17.75.61
worker.jvm1.port=8019
worker.jvm1.lbfactor=10
worker.jvm2.type=ajp13
worker.jvm2.host=10.17.75.61
worker.jvm2.port=8029
worker.jvm2.lbfactor=10
worker.part.type=lb
worker.part.balance_workers=jvm1,jvm2
worker.part.sticky_session=1
Please note that I have two different IP addresses running under the same worker. My boss thinks that this is causing sessions to jump between the two ip addresses on the same jvm.
Question
Is the current configuration incorrect? I don't think so but can't find anything anywhere else to back me up. Reasoning being that workers.properties has the IP address defined in it. So the request would go to jvm1 ip 10.17.75.61 if jvm1 ip 10.17.75.61 is down or not available it won't try to go to jvm1 10.17.75.145 because the host info (ip address) isn't the same.
Jvm1- TC instance1
Jvm2- TC instance2
Thanks in advance,
Martin
Re: Setup of workers.properties with multiple host and TC instances
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Martin,
On 3/22/12 5:08 PM, Martin Registe, Jr. wrote:
>> First question: do all httpd instances know about all Tomcat
>> instances? Or, does httpd only know about the two instances that
>> are co-located with it on the same physical server?
>
> Httpd only know about the two instances that re collocated with it
> on the same physical server
Okay.
>> If https only knows about its local 2 Tomcat instances, then
>> you'll only need 2 workers, etc. I have no idea how this could
>> possibly work with sticky sessions unless the hw lb knows how to
>> dispatch to the right http/2xTomcat physical server, in which
>> case you are wasting your time with all these extra processes.
>
> The hw loadbalancer sends the traffic to 10.17.75.61 port 80...
> apache takes it and using the workers.properties file sends it to
> one of TC instances then apache sends it back through the hw lb it
> has a sessionid and jvm tag on in the host header so it knows which
> jvm it came from
So, when a client first arrives, they are randomly-assigned an httpd
instance and then randomly-assigned a Tomcat instance that "belongs"
to that httpd instance. The hw lb later uses the "Host" header to
determine which httpd instance gets subsequent requests?
The "Host" header should include the IP address or hostname of the
target server. Are you telling the client that the Host changes when
they are assigned a Tomcat server? I still don't understand why you
don't just have a more-connected cluster where all httpds are
connected directly to all Tomcats.
>> What about <Engine>? It would be nice to confirm that your
>> jvmRoute(s) is(are) set correctly.
>
> TC1 <Engine name="Catalina" defaultHost="localhost"
> jvmRoute="jvm1"> TC2 <Engine name="Catalina"
> defaultHost="localhost" jvmRoute="jvm2">
Looks good.
> My next thoughts I don't know exactly how this is working but it
> definitely is working and doesn't seem to be getting sessions
> jumping from one jvm to the other (on the same physical server).
Uh... okay. I thought you were coming here for help. Do you not
actually have a problem? If your boss simply suspects that there is a
problem, make him prove it via observations. You can't prove a
negative so it's his job to show you a problem, not your job to prove
that no problem exists.
> I will play around with the configuration some more but this has
> been working for over a year. You are saying if I just go with
>
> worker.jvm1.type=ajp13 worker.jvm1.host=localhost
> worker.jvm1.port=8019 worker.jvm1.lbfactor=10
>
> worker.jvm2.type=ajp13 worker.jvm2.host=localhost
> worker.jvm2.port=8029 worker.jvm2.lbfactor=10
>
> worker.part.type=lb worker.part.balance_workers=jvm1,jvm2
> worker.part.sticky_session=1
>
> everything will continue to work, correct?
It should: you are simply balancing between two Tomcat instances and
using sticky sessions. It's about as simple a configuration as you can
have.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk9rneAACgkQ9CaO5/Lv0PA2GACeP3uB1M6znUXhcZO+FWxk5I8Y
tJ0AoK6MLicNEz/68bsJs0ZB50xJ9GGf
=pndJ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Setup of workers.properties with multiple host and TC instances
Posted by "Martin Registe, Jr." <ma...@appia.com>.
Chris,
Let me see if I've got this straight. You've got:
* A hw load-balancer out front
* 5 physical servers each running:
- 1 Apache httpd
- 2 Tomcats
This allows you to have physical-server redundancy (x5) and also the ability to take one TC instance down on each server for maintenance while leaving the other one running. Right?
Answer
Yes this is correct
First question: do all httpd instances know about all Tomcat instances? Or, does httpd only know about the two instances that are co-located with it on
the same physical server?
Answer
Httpd only know about the two instances that re collocated with it on the same physical server
If https only knows about its local 2 Tomcat instances, then you'll only need 2 workers, etc. I have no idea how this could possibly work with sticky sessions unless the hw lb knows how to dispatch to the right http/2xTomcat physical server, in which case you are wasting your time with all these extra processes.
Answer
The hw loadbalancer sends the traffic to 10.17.75.61 port 80...
apache takes it and using the workers.properties file sends it to one of TC instances
then apache sends it back through the hw lb it has a sessionid and jvm tag on in the host header so it knows which jvm it came from
What about <Engine>? It would be nice to confirm that your jvmRoute(s)
is(are) set correctly.
Answer
TC1 <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
TC2 <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm2">
That depends. The host should always be "localhost", since your httpd instances only talk to the co-located Tomcats, right?
Answer
Yes
My next thoughts
I don't know exactly how this is working but it definitely is working and doesn't seem to be getting sessions jumping from one jvm to the other (on the same physical server). I will play around with the configuration some more but this has been working for over a year. You are saying if I just go with
worker.jvm1.type=ajp13
worker.jvm1.host=localhost
worker.jvm1.port=8019
worker.jvm1.lbfactor=10
worker.jvm2.type=ajp13
worker.jvm2.host=localhost
worker.jvm2.port=8029
worker.jvm2.lbfactor=10
worker.part.type=lb
worker.part.balance_workers=jvm1,jvm2
worker.part.sticky_session=1
everything will continue to work, correct?
Thanks,
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Setup of workers.properties with multiple host and TC instances
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Martin,
On 3/22/12 1:18 PM, Martin Registe, Jr. wrote:
> How many *tomcat instances* do you have running ? And can you
> explain, in general terms, what you are trying to do ?
>
> @Andre
>
> I am trying to run two reduant tomcat instances that are each
> running two reduant hosts on the same phyiscal server. I have
> detailed it below.
Let me see if I've got this straight. You've got:
* A hw load-balancer out front
* 5 physical servers each running:
- 1 Apache httpd
- 2 Tomcats
This allows you to have physical-server redundancy (x5) and also the
ability to take one TC instance down on each server for maintenance
while leaving the other one running. Right?
I would argue that you should just take a whole server out of the lb
pool when you want to service it and not bother at all with httpd. If
your hw lb can't properly do stick-sessions, well then you'll still
have to have httpd involved but maybe you can eliminate one of the TC
instances per server by removing the one going down for maintenance
from the mod_jk worker pool.
Anyhow: your picture looks like this to me (hope the formatting
translates well):
/- x.y.145.61 : httpd + 2x Tomcat
lb < - x.y.154.62 : httpd + 2x Tomcat
...
\- x.y.154.65 : httpd + 2x Tomcat
First question: do all httpd instances know about all Tomcat
instances? Or, does httpd only know about the two instances that are
co-located with it on the same physical server?
If httpd knows about all Tomcat instances (x10), then you'll need
every httpd to have a workers.properties file with 10 workers defined
and all of them pooled. They all need distinct jvmRoutes.
If https only knows about its local 2 Tomcat instances, then you'll
only need 2 workers, etc. I have no idea how this could possibly work
with sticky sessions unless the hw lb knows how to dispatch to the
right http/2xTomcat physical server, in which case you are wasting
your time with all these extra processes.
> Then in the server.xml file OnTC1 for jvmRoute=jvm1 OnTC2 for
> jvmRoute=jvm2 And I have host entries for both in each server.xml
> <Host name="10.17.75.145" appBase="webapps" unpackWARs="true"
> autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false">
> There are prameter and other stuff </Host>
>
> <Host name="10.17.75.61" appBase="webapps" unpackWARs="true"
> autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false">
> There are parameters and other stuff </Host>
What about <Engine>? It would be nice to confirm that your jvmRoute(s)
is(are) set correctly.
> We have two redunant tomcat instances running on the same box. So
> as long as there is not a problem on the physical server one of
> the tomcat instances will always be taking traffic. There is a hw
> loadbalancer that sits in front of all of this and there are 5
> other servers setup the same way the hw loadbalancer just directs
> traffic to the particular physical server where apache takes over.
> Each apache only works with its local (2) tomcat instances.
That should have been the first thing you said.
If httpd only knows about its local 2 Tomcat instances, how do sticky
sessions work?
> In the workers.properties if I change .61 to
> worker.jvm3.type=ajp13 worker.jvm3.host=10.17.75.61
> worker.jvm3.port=8019 worker.jvm3.lbfactor=10
>
> worker.jvm4.type=ajp13 worker.jvm4.host=10.17.75.61
> worker.jvm4.port=8019 worker.jvm4.lbfactor=10
>
> and add them to the lb setting worker.part.type=lb
> worker.part.balance_workers=jvm1,jvm2,jvm3,jvm4
> worker.part.sticky_session=1
>
> will traffic get to right host?
That depends. The host should always be "localhost", since your httpd
instances only talk to the co-located Tomcats, right?
> Even though there is no jvmRoute for jvm3 and jvm4.
If httpd gets a request with a session id including a jvmroute, it
will attempt to locate the right worker and send the request to it. IF
you don't have a jvmRoute configured, then it won't work at all.
> I am trying to find a way, or prove that apache won't take a .61
> request and send it to one of the .145 hosts or visa versa.
httpd will only send requests to Tomcat instances defined in
workers.properties. So, if you have 2 workers which are load-balanced,
you'll only be able to send traffic to those two workers.
You said this initially:
> My boss thinks that this is causing sessions to jump between the
> two ip addresses on the same jvm.
First, it shouldn't matter which IP address gets used since the
requests all go to the same place. Second, there's no reason to have a
hostname other than "localhost" when you are communicating between
httpd and Tomcat on the same physical server. Why bother with IP
addresses? Third, I would be more worried in your environment about
requests being served by a random httpd/2xTomcat team by your hw lb.
I'm totally confused, here.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk9rhd4ACgkQ9CaO5/Lv0PBPgQCfTreX+vpBaKj03rN8ojsTWtOZ
m38Ani+KQRXxHGDesRHiuzSSqlYSzTtV
=5bSl
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Setup of workers.properties with multiple host and TC instances
Posted by "Martin Registe, Jr." <ma...@appia.com>.
How many *tomcat instances* do you have running ?
And can you explain, in general terms, what you are trying to do ?
@Andre
I am trying to run two reduant tomcat instances that are each running two reduant hosts on the same phyiscal server. I have detailed it below.
On one physical server I am running two tomcat instances
TC1 - 8019 .145\.61
TC2 - 8029 145.\.61
On the same physical server I have apache which I have two virtual sites running on
Site1 - 10.17.75.145
Site2 - 10.17.75.61
Inside of each virtal host entry in apache I currently have them both mounting jkmount /* part
Then in workers.properties
I have the setup
worker.list=part
#worker.list=worker1, worker2
#
# Defining a worker named ajp13w and of type ajp13
# Note that the name and the type do not have to match.
#
worker.jvm1.type=ajp13
worker.jvm1.host=10.17.75.145
worker.jvm1.port=8019
worker.jvm1.lbfactor=10
worker.jvm2.type=ajp13
worker.jvm2.host=10.17.75.145
worker.jvm2.port=8029
worker.jvm2.lbfactor=10
worker.jvm1.type=ajp13
worker.jvm1.host=10.17.75.61
worker.jvm1.port=8019
worker.jvm1.lbfactor=10
worker.jvm2.type=ajp13
worker.jvm2.host=10.17.75.61
worker.jvm2.port=8029
worker.jvm2.lbfactor=10
worker.part.type=lb
worker.part.balance_workers=jvm1,jvm2
worker.part.sticky_session=1
Then in the server.xml file
OnTC1 for jvmRoute=jvm1
OnTC2 for jvmRoute=jvm2
And I have host entries for both in each server.xml
<Host name="10.17.75.145" appBase="webapps" unpackWARs="true" autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false">
There are prameter and other stuff
</Host>
<Host name="10.17.75.61" appBase="webapps" unpackWARs="true" autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false">
There are parameters and other stuff
</Host>
We have two redunant tomcat instances running on the same box. So as long as there is not a problem on the physical server one of the tomcat instances will always be taking traffic. There is a hw loadbalancer that sits in front of all of this and there are 5 other servers setup the same way the hw loadbalancer just directs traffic to the particular physical server where apache takes over. Each apache only works with its local (2) tomcat instances.
In the workers.properties if I change .61 to
worker.jvm3.type=ajp13
worker.jvm3.host=10.17.75.61
worker.jvm3.port=8019
worker.jvm3.lbfactor=10
worker.jvm4.type=ajp13
worker.jvm4.host=10.17.75.61
worker.jvm4.port=8019
worker.jvm4.lbfactor=10
and add them to the lb setting
worker.part.type=lb
worker.part.balance_workers=jvm1,jvm2,jvm3,jvm4
worker.part.sticky_session=1
will traffic get to right host? Even though there is no jvmRoute for jvm3 and jvm4. Does the worker.xxxx.host=10.17.75.61 play a part in this at all?
@TM
I looked at the modjk.log and I do see the duplicate errors so I defiantly need to fix that. I should have explained that I am running two reduant tomcat instances on the same phyical box and they are listening to 8019 and 8029 already. I am trying to find a way, or prove that apache won't take a .61 request and send it to one of the .145 hosts or visa versa. Eveything looks to be working okay with the current config....)other than the duplicate errors in the modjk.log file) but I need more info
Thanks,
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Setup of workers.properties with multiple host and TC instances
Posted by André Warnier <aw...@ice-sa.com>.
Martin Registe, Jr. wrote:
> -----Original Message-----
> From: Mladen Turk [mailto:mturk@apache.org]
> Sent: Wednesday, March 21, 2012 2:09 PM
> To: users@tomcat.apache.org
> Subject: Re: Setup of workers.properties with multiple host and TC instances
>
> On 03/21/2012 07:47 PM, Martin Registe, Jr. wrote:
>> Hi All,
>>
>>
>> Please note that I have two different IP addresses running under the same worker.
>
> You will need one worker per host:port thus in your case 4 of them not 2.
>
>> My boss thinks that this is causing sessions to jump between the two ip addresses on the same jvm.
>
> You only have contract for two instances. The latest one will overwrite the first setting.
>
>
From Martin's messages above, I believe that there is some confusion as to what "workers"
are, and also about what exactly you are trying to achieve.
In any case, "workers" do not need to be defined in Tomcat's server.xml.
@Martin :
How many *tomcat instances* do you have running ?
And can you explain, in general terms, what you are trying to do ?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Setup of workers.properties with multiple host and TC instances
Posted by Mladen Turk <mt...@apache.org>.
You should learn how to quote messages ;)
On 03/21/2012 08:22 PM, Martin Registe, Jr. wrote:
>>> Please note that I have two different IP addresses running under the same worker.
>> You will need one worker per host:port thus in your case 4 of them not 2.
>>
> Okay so I broke them up and created another lb group (not sure if I had to do that) it looks like this
> worker.list=part, myxx ...
You have typo here. Should be
worker.list=part,myxx
(no spaces around coma)
> I looked online and through my Apache Tomcat book ...
1. You will need to set jvmRoute to match the lb worker name inside Engine
(either 'part' or 'myxx')
2. You can have only one Engine
3. Use two separate physical tomcat instances
one listening on port 8019 and other on 8029
and one having jvmRoute="part" and other jvmRoute="myxx"
Simple :)
Regards
--
^TM
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Setup of workers.properties with multiple host and TC instances
Posted by "Martin Registe, Jr." <ma...@appia.com>.
-----Original Message-----
From: Mladen Turk [mailto:mturk@apache.org]
Sent: Wednesday, March 21, 2012 2:09 PM
To: users@tomcat.apache.org
Subject: Re: Setup of workers.properties with multiple host and TC instances
On 03/21/2012 07:47 PM, Martin Registe, Jr. wrote:
> Hi All,
>
>
> Please note that I have two different IP addresses running under the same worker.
You will need one worker per host:port thus in your case 4 of them not 2.
> My boss thinks that this is causing sessions to jump between the two ip addresses on the same jvm.
You only have contract for two instances. The latest one will overwrite the first setting.
Regards
--
^TM
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Thanks for the quick response TM
Okay so I broke them up and created another lb group (not sure if I had to do that) it looks like this
worker.list=part, myxx
#worker.list=worker1, worker2
#
# Defining a worker named ajp13w and of type ajp13
# Note that the name and the type do not have to match.
#
worker.jvm1.type=ajp13
worker.jvm1.host=10.17.75.145
worker.jvm1.port=8019
worker.jvm1.lbfactor=10
worker.jvm2.type=ajp13
worker.jvm2.host=10.17.75.145
worker.jvm2.port=8029
worker.jvm2.lbfactor=10
worker.jvm3.type=ajp13
worker.jvm3.host=10.17.75.61
worker.jvm3.port=8019
worker.jvm3.lbfactor=10
worker.jvm4.type=ajp13
worker.jvm4.host=10.17.75.61
worker.jvm4.port=8029
worker.jvm4.lbfactor=10
worker.part.type=lb
worker.part.balance_workers=jvm1,jvm2
worker.part.sticky_session=1
worker.myxx.type=lb
worker.myxx.balance_workers=jvm3,jvm4
worker.myxx.sticky_session=1
But I get the below error in the mod_jk.log
[Tue Mar 20 15:32:10.978 2012] [27476:35628] [error] jk_worker.c (151): factory for lb failed for myxx
[Tue Mar 20 15:32:10.978 2012] [27476:35628] [error] jk_worker.c (262): failed to create worker myxx
I looked online and through my Apache Tomcat book but I haven't been able to fine anything when it comes to running two instances and on the same box sharing the same IP addresses on multiple ports. I don't even know if I really need to make this change. I don't think this option will work because jvm3 and jvm4 aren't defined in the server.xml files on the tomcat side
Thanks,
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Setup of workers.properties with multiple host and TC instances
Posted by Mladen Turk <mt...@apache.org>.
On 03/21/2012 07:47 PM, Martin Registe, Jr. wrote:
> Hi All,
>
>
> Please note that I have two different IP addresses running under the same worker.
You will need one worker per host:port thus in your case 4 of them not 2.
> My boss thinks that this is causing sessions to jump between the two ip addresses on the same jvm.
You only have contract for two instances. The latest one will overwrite the first setting.
Regards
--
^TM
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org