You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Nenad Kovacevic <mi...@yahoo.com> on 2009/05/08 22:42:19 UTC
Getting the Right High Availability Architecture for Tomcat
We are setting our infrastructure to run tomcat servers with stateful
two-tier Web UI applications (WEB Tier (Tomcat) --jdbc--> Database).
Broadly speaking the infrastructure has to satisfy 99.9% of up time as far
as HA is concerned. As far as applications are concerned, in case a server
fails it is OK to fail the in-flight-request, however the user's session
should be preserved. Another requirement is of course performance /
scalability. Even though it has not been explicitly specified (i.e. in terms
of number of concurrent users/requests and maximum allowed response times)
it is expected that when the response times start getting poor this would be
resolved by putting additional nodes in the infrastructure. I realize
addressing scalability does not end in the web tier and that in large
portion it depends on the scalability of the data tier, but for the sake of
this discussion please abstract data tier out and assume that scalability is
entirely constrained by the web tier.
The architecture we were thinking of implementing would have two nodes (for
starters) each running a Tomcat server. Tomcats would be configured to run
within one cluster (referring to Tomcat's clustering capabilities here - not
clustering software like Veritas etc.). The cluster would be fronted by a
pair of LocalDirectors (Cisco load-balancing appliance) that would implement
load balancing over the servers. Pretty much all applications would run over
https where SSL sessions would terminate on tomcat servers, not on the load
balancers. Note that this means that request's payload would not be
available to balancers so to be able to implement request stickiness based
on JSESSIONID nor any other piece of content within the request.
So all of that means that an https request would be forwarded to a tomcat
instance, and in the course of processing it any change in session state
would be broadcast to all cluster members. Since the session is now in the
"collective memory" of the cluster any subsequent request is free to be
routed to any available tomcat instance not just the initial one that
created the session.
When the load increases to the point that responses start becoming too slow,
we just add another tomcat node.
Sounds too simple to be true so I am wondering if I am missing something,
and is a setup like this supposed to work?
1) First of all I have found many posts where almost every load balancing
setup with Tomcat mentions implementing sticky session where load balancers
(usually apache httpd with tomcat's mod_jk module) would forward requests to
the instance that created the session in the first place and it would only
be ever routed to a different instance in case the original one is not
available to process it (e.g. crashes). Is stickiness required and why? If
the session is replicated across the cluster should it not be irrelevant
where a request gets processed? Or it is recommended because session
replication can not be 100% guaranteed especially under heavy load (and
having an incomplete and out-dated session is better then no session at all
in failure scenarios)?
2) How to ensure session uniqueness across the cluster? Is Tomcat smart
enough not to generate an ID that was already created by another instance in
the cluster or one has to ensure uniqueness through configuration using
jvmRoute attribute of Tomcat's /server/service/engine element (inside
server.xml configuration file)? The problem I am seeing with using jvmRoute
in the setup we are considering is that JSESSIONID changes every time a
request gets routed to a different instance. Even though that does not seem
to affect session state (which seems preserved) it worries me that this is
not a right thing to do...
Please let me know your thoughts on this, any problems you see with such a
setup, and if there are better solutions.
Thanks
Nash
--
View this message in context: http://www.nabble.com/Getting-the-Right-High-Availability-Architecture-for-Tomcat-tp23452018p23452018.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Getting the Right High Availability Architecture for Tomcat
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Robin,
On 5/11/2009 11:47 AM, Robin Wilson wrote:
> Actually, <script> tags load sequentially, I believe. (Not sure about
> css.)
That's a pretty bold statement. The truth is, it's complicated.
http://www.stevesouders.com/blog/2009/04/27/loading-scripts-without-blocking/
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkoIh3MACgkQ9CaO5/Lv0PDJhACgjsNV9XMP8/rpYhn14JsS/Vky
D7IAoJmqaYaK/7FAfp/ylb8TSPpaX4ld
=vzs/
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Getting the Right High Availability Architecture for Tomcat
Posted by Robin Wilson <rw...@kingsisle.com>.
Actually, <script> tags load sequentially, I believe. (Not sure about
css.)
--
Robin D. Wilson
On May 11, 2009, at 10:28 AM, "André Warnier" <aw...@ice-sa.com> wrote:
> Nenad Kovacevic wrote:
>>
>> Caldarale, Charles R wrote:
>>>
>>> Have you considered doing the SSL processing in the load
>>> balancer(s)? It
>>> would make life simpler.
>>>
>>> - Chuck
>>>
>>>
>>
>>> From application's perspective it really does not make much of a
>>> difference
>> where SSL is done - actually it does make now after your
>> explanation -
>> thanks for that - however this change would need to be run and
>> approved by
>> our networking and security people first. In any case I am afraid
>> that even
>> if they are willing to move SSL processing to the balancers this
>> change may
>> not happen in time for our first application, so we might end up
>> with the
>> setup as I described it in my original post.
>>
>> Our applications do not issue concurrent requests to the servers,
>> i.e. they
>> are classical web applications where the user activates a control
>> on a page
>> and then waits for a page to refresh or a new page to load.
>> Therefore under
>> normal usage scenarios concurrent requests should really not
>> happen. I say
>> normal as it is possible for a user to resubmit a request by
>> reloading a
>> page using browser controls. However we warn the users to use only
>> controls
>> on the page and gray-out submit buttons once a request is submitted
>> so
>> hopefully this should not be an issue.
>>
>> With such an application in mind would you see an issue with not
>> implementing sticky session? Again, I was able to test that and the
>> only
>> issue that I am seeing is that JSESSIONID changes depending on what
>> Tomcat
>> instance processed it, but again, I am not sure if that is really
>> an issue
>> or not?
>>
> Not sure if it is relevant here :
> a browser will make (quasi-)concurrent requests to the server, for
> example when you load a html "frames" document. The first request
> will
> be for the frames document itself, but as soon as that one is
> returned,
> each <frame> in it will be the object of a new request.
> A similar case happens when a document merely contains <img> or
> <style>
> or <script> tags. To "fill these slots", the browser will issue
> several
> requests (and probably establish several connections) in parallel.
> I know that this kind of thing can play havoc with some authentication
> schemes for instance. Again, I don't know if it is really a cause for
> concern in this situation.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: Getting the Right High Availability Architecture for Tomcat
Posted by André Warnier <aw...@ice-sa.com>.
Nenad Kovacevic wrote:
>
> Caldarale, Charles R wrote:
>>
>> Have you considered doing the SSL processing in the load balancer(s)? It
>> would make life simpler.
>>
>> - Chuck
>>
>>
>
>>>From application's perspective it really does not make much of a difference
> where SSL is done - actually it does make now after your explanation -
> thanks for that - however this change would need to be run and approved by
> our networking and security people first. In any case I am afraid that even
> if they are willing to move SSL processing to the balancers this change may
> not happen in time for our first application, so we might end up with the
> setup as I described it in my original post.
>
> Our applications do not issue concurrent requests to the servers, i.e. they
> are classical web applications where the user activates a control on a page
> and then waits for a page to refresh or a new page to load. Therefore under
> normal usage scenarios concurrent requests should really not happen. I say
> normal as it is possible for a user to resubmit a request by reloading a
> page using browser controls. However we warn the users to use only controls
> on the page and gray-out submit buttons once a request is submitted so
> hopefully this should not be an issue.
>
> With such an application in mind would you see an issue with not
> implementing sticky session? Again, I was able to test that and the only
> issue that I am seeing is that JSESSIONID changes depending on what Tomcat
> instance processed it, but again, I am not sure if that is really an issue
> or not?
>
Not sure if it is relevant here :
a browser will make (quasi-)concurrent requests to the server, for
example when you load a html "frames" document. The first request will
be for the frames document itself, but as soon as that one is returned,
each <frame> in it will be the object of a new request.
A similar case happens when a document merely contains <img> or <style>
or <script> tags. To "fill these slots", the browser will issue several
requests (and probably establish several connections) in parallel.
I know that this kind of thing can play havoc with some authentication
schemes for instance. Again, I don't know if it is really a cause for
concern in this situation.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Getting the Right High Availability Architecture for Tomcat
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Nenad Kovacevic [mailto:micamicic@yahoo.com]
> Subject: RE: Getting the Right High Availability Architecture for
> Tomcat
>
> Our applications do not issue concurrent requests to the servers, i.e.
> they are classical web applications where the user activates a control
> on a page and then waits for a page to refresh or a new page to load.
Unless the pages are incredibly simplistic, there will be multiple concurrent requests. Each embedded object in the page along with any .css files will cause the browser to issue multiple requests in parallel. You also have no control over the user getting impatient and clicking more links or buttons before the prior one responds.
> However we warn the users to use only controls on the page and gray-out
> submit buttons once a request is submitted so hopefully this should not
> be an issue.
Sorry, but that's rather naive. You cannot base the integrity of your application on trusting the user to always do the right thing.
> With such an application in mind would you see an issue with not
> implementing sticky session?
Yes, I would.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Getting the Right High Availability Architecture for Tomcat
Posted by Nenad Kovacevic <mi...@yahoo.com>.
Nenad Kovacevic wrote:
>
>
> Caldarale, Charles R wrote:
>>
>>
>> Have you considered doing the SSL processing in the load balancer(s)? It
>> would make life simpler.
>>
>> - Chuck
>>
>>
>
> From application's perspective it really does not make much of a
> difference where SSL is done - actually it does make now after your
> explanation - thanks for that - however this change would need to be run
> and approved by our networking and security people first. In any case I am
> afraid that even if they are willing to move SSL processing to the
> balancers this change may not happen in time for our first application, so
> we might end up with the setup as I described it in my original post.
>
> Our applications do not issue concurrent requests to the servers, i.e.
> they are classical web applications where the user activates a control on
> a page and then waits for a page to refresh or a new page to load.
> Therefore under normal usage scenarios concurrent requests should really
> not happen. I say normal as it is possible for a user to resubmit a
> request by reloading a page using browser controls. However we warn the
> users to use only controls on the page and gray-out submit buttons once a
> request is submitted so hopefully this should not be an issue.
>
> With such an application in mind would you see an issue with not
> implementing sticky session? Again, I was able to test that and the only
> issue that I am seeing is that JSESSIONID changes depending on what Tomcat
> instance processed it, but again, I am not sure if that is really an issue
> or not?
>
> Thanks
> Nash
>
>
Sorry, forgot to mention we are running Tomcat 6.0.18 on SunOS 5.8, our
current java is 1.5 update 11.
--
View this message in context: http://www.nabble.com/Getting-the-Right-High-Availability-Architecture-for-Tomcat-tp23452018p23485700.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Getting the Right High Availability Architecture for Tomcat
Posted by Nenad Kovacevic <mi...@yahoo.com>.
Caldarale, Charles R wrote:
>
>
> Have you considered doing the SSL processing in the load balancer(s)? It
> would make life simpler.
>
> - Chuck
>
>
>From application's perspective it really does not make much of a difference
where SSL is done - actually it does make now after your explanation -
thanks for that - however this change would need to be run and approved by
our networking and security people first. In any case I am afraid that even
if they are willing to move SSL processing to the balancers this change may
not happen in time for our first application, so we might end up with the
setup as I described it in my original post.
Our applications do not issue concurrent requests to the servers, i.e. they
are classical web applications where the user activates a control on a page
and then waits for a page to refresh or a new page to load. Therefore under
normal usage scenarios concurrent requests should really not happen. I say
normal as it is possible for a user to resubmit a request by reloading a
page using browser controls. However we warn the users to use only controls
on the page and gray-out submit buttons once a request is submitted so
hopefully this should not be an issue.
With such an application in mind would you see an issue with not
implementing sticky session? Again, I was able to test that and the only
issue that I am seeing is that JSESSIONID changes depending on what Tomcat
instance processed it, but again, I am not sure if that is really an issue
or not?
Thanks
Nash
--
View this message in context: http://www.nabble.com/Getting-the-Right-High-Availability-Architecture-for-Tomcat-tp23452018p23485311.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Getting the Right High Availability Architecture for Tomcat
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Nenad Kovacevic [mailto:micamicic@yahoo.com]
> Subject: Getting the Right High Availability Architecture for Tomcat
>
> If the session is replicated across the cluster should it not be
> irrelevant where a request gets processed?
The problem is that multiple requests associated with a single session may be occurring simultaneously. Unless you have sticky sessions, you would have to replicate each session modification as it occurs, using appropriate cross-node locking. (This is usually considered to be a performance killer, as you might imagine.) Replication normally occurs only at the completion of a request, and, with sticky sessions, normal Java locking techniques can be used to insure that concurrent request processing maintains session integrity.
You could implement some form of session ownership so that only one node has the updatable copy at any given time, and only give up ownership when needed. Still complicated, and still has performance issues.
> Is Tomcat smart enough not to generate an ID that was already
> created by another instance in the cluster or one has to ensure
> uniqueness through configuration using jvmRoute attribute
The latter is required at the moment. It would be fairly trivial to modify the session ID generation logic to limit each node to a specific range of values (it's only an integer); this would eliminate the need to set jvmRoute.
Have you considered doing the SSL processing in the load balancer(s)? It would make life simpler.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org