Re: Load balancing - fail-over support with mod_webapp

[Mathias Herberts <Ma...@iroise.net>]

Hi, I have been working on load balancing Apache/Tomcat clusters.

In mod_jk, the fail-over is done in the function
get_most_suitable_worker. Basically if the worker who initiated the
session is available (i.e. not in error state), use it, otherwise loop
through the workers and select either the one with the lowest load
balancing value OR one that has been in error for at least
WAIT_BEFORE_RECOVER seconds if such worker exists. Thus the faulted
workers are privileged when selecting a worker for a request whose
worker designated by the jvmroute is not available.

There is though no algorithm implemented which will map all requests to
the same 'backup' worker, thus either the worker which created the
session (the one in the jvmroute) is available, either another worker
will be selected, but in the latter case there is no guarantee that
'second choice' will be the same for the next requests.

Therefore re-routing does exist in mod_jk and works, but it is not
optimal for your case, the worst thing being the
dispatching of incoming requests to a different worker each time, your
distributed session management would therefore give the parenting of the
session to a different worker each time, far from optimal!

Mathias.

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

[Mathias Herberts <Ma...@iroise.net>]

> Another solution would be to calculate some sort
> of checksum on each session id received for a vm that is
> no longer on the air. Then modulo that number by the number of
> available tc instances in the cluster, and route the request to the
> the jth server in the cluster (where j is the calculated modulo).
> This would spread the load without requiring anyone to start
> maintaining copious session routing tables. I don't know what
> you'd do if the number of available tc's changes.

You implicitly make the assumption that in a setup with several
frontend Apache servers they all use the same workers and that
there is an order relationship between those. Choosing the jth
worker only makes sense if the workers are in the same order
on all mod_jk configs.

You could very well have a setup where a load balancing equipment
first routes a request according to a client ip for example to a
given Apache server, being in a 'GOLD' customer all my requests
are forwarded to a fast Apache with fast workers. Now if I am
a 'COPPER' customer my requests will be routed to the slower
Apache server using slower workers.

The 'GOLD' setup has fail-over, that is the slow workers can
handle the requests coming from the 'GOLD' Apache but 'COPPER'
clients get no fail-over. The modulo trick is really tricky in
such a setup.

> Another solution would be to send a cookie back to the
> client that contains the new jvmrouting info. But, that only
> works for clients that allow cookies.

If the webapp uses URL rewriting where needed, the new value
containing the new jvmroute can very well be included where
it belongs (in all encoded URLs).

The problem I see can arise is the atomicity of the setting of
the new value. Somehow we need a 'lock' in the cluster of
workers so that the first one to receive old session ID S1 will
turn it into S2 and all subsequent requests for S1 will also
turn it into S2, even if one of those requests is dispatched to
a worker which would, if it had received the first request with
S1, have turned it into S3. The mapping S1->S2 should be kept
for the life of the session, so are possible further mappings,
S1->S3 and/or S2->S'3, etc, should further workers fail.

Mathias.

> Another solution would be to choose a new 'owner', and
> store this mapping (of session id to owner vm) in a hashtable.
> Then, in subsequent requests, do a hashtable lookup to find
> the owner vm. Solutions that require a large routing table
> are probably not desirable, but would probably result in the
> greatest flexibility.

Yep, sort of what I talked about in my previous paragraph, but I would
see this mapping stored among the workers, not in Apache modules.
 
> Tom Drake

Mathias.

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

[Tom Drake <rt...@pobox.com>]

Adding the vmroute may help with load balancing, but it doesn't
help with fail-over. What happens when the tomcat instance that
created a session goes down, and then the user sends another
request? It should be routed to 'another' tomcat in the cluster.
This 'other' tc instance should probably become the new 'owner'
of the session. All subsequent requests for that session should
be routed to the new owner (until / unless the new owner goes
down).

One simple solution is to simply keep track of the 'vmroute's that
are 'down' or no longer available and substitue each down vm
with one other vm in the cluster. Anytime a request is received
from a client that is destined for one of the 'down' vm's, Apache
can route the request to the 'stand-in' vm instead. The obvious
disadvantage is that it will unfairly burden the 'stand-in' vm's,
which may have already been handling their fair share of traffic.
This, does provide fail-over. Which, is an important feature to
have for ha environments.

Another solution would be to calculate some sort
of checksum on each session id received for a vm that is
no longer on the air. Then modulo that number by the number of
available tc instances in the cluster, and route the request to the
the jth server in the cluster (where j is the calculated modulo).
This would spread the load without requiring anyone to start
maintaining copious session routing tables. I don't know what
you'd do if the number of available tc's changes.

Another solution would be to send a cookie back to the
client that contains the new jvmrouting info. But, that only
works for clients that allow cookies.

Another solution would be to choose a new 'owner', and
store this mapping (of session id to owner vm) in a hashtable.
Then, in subsequent requests, do a hashtable lookup to find
the owner vm. Solutions that require a large routing table
are probably not desirable, but would probably result in the
greatest flexibility.

Tom Drake

----- Original Message -----
From: <co...@covalent.net>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: Tuesday, December 18, 2001 2:14 PM
Subject: Re: Load balancing - fail-over support with mod_webapp


| On Tue, 18 Dec 2001, Craig R. McClanahan wrote:
|
| > The load balancer routing from JK hasn't ever been implemented in
| > Catalina yet.  Patches welcome :-).
|
| Can we get a 'setRequest' method on Manager ? I'm trying to find
| workarounds, but that would be the simple solution.
|
| Again, adding support for load balancer is simple - the Manager just need
| to append the vmroute to the generated session id. How you get the session
| id is the problem - it is available in request, or it could be configured
| in server.xml if getting from the request is not possible.
|
| ( right now I'm trying to finish jk2 asap - when I'm done I'll try again
| to fix the lb for 40 if nobody does it before )
|
| Costin
|
|
| --
| To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
| For additional commands, e-mail:
<ma...@jakarta.apache.org>
|
|
|


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

[Bill Barker <wb...@wilshire.com>]

I'm going to ignore the complete problem case where you have configured
Apache to be the "default servlet", and index.html is a frameset with 'n'
jsp frames (which will typically result in 'n' different sessions).  This is
why mod_webapp only supports forwardAll, and forwardAll is the default for
Ajp.

Most 3.3 methods map well to Catalina.  It's either an xxxListener or a
Valve.  This is one of the problem cases that falls through the cracks.  As
such, I personally can't see how to do this better than Costin's idea of
adding a jvmroute attribute to the Manager.  Instead, I'm going to try and
describe the programmatic requirements and hope that someone smarter than me
can see how to implement it.

Apache knows who it sent the request to, and can (and does in 3.x) pass this
information on to the Request object.  If, as it happens, that the servlet
that eventually gets executed at the end of the pipeline (or a rogue
Valve/Filter) creates a new Session (e.g. Listener), then somebody with
access to the Request object (e.g. Valve)  needs to re-set the SessionId to
be SessionId += "." +request.getJvmroute().  This has to happen before the
servlet regains control (so that response.encodeURL works), and before the
cookie is set (or at least sent).

Of course, what I care about is Apache.  However in the above you can
s/Apache/IIS/ or s/Apache/IPlanet/, and get to the same point.
----- Original Message -----
From: "Craig R. McClanahan" <cr...@apache.org>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: Tuesday, December 18, 2001 6:09 PM
Subject: Re: Load balancing - fail-over support with mod_webapp


>
>
> On Tue, 18 Dec 2001 costinm@covalent.net wrote:
>
> > Date: Tue, 18 Dec 2001 14:14:10 -0800 (PST)
> > From: costinm@covalent.net
> > Reply-To: Tomcat Developers List <to...@jakarta.apache.org>
> > To: Tomcat Developers List <to...@jakarta.apache.org>
> > Subject: Re: Load balancing - fail-over support with mod_webapp
> >
> > On Tue, 18 Dec 2001, Craig R. McClanahan wrote:
> >
> > > The load balancer routing from JK hasn't ever been implemented in
> > > Catalina yet.  Patches welcome :-).
> >
> > Can we get a 'setRequest' method on Manager ? I'm trying to find
> > workarounds, but that would be the simple solution.
> >
>
> In other words, a *single* "request" property for the entire Manager?
> Wouldn't that have problems when you had simultaneous requests in
> progress?
>
> Same thing would happen (but on a smaller scale) if you added setRequest
> on the session instead -- it's legal to have multiple simultaneous
> requests.
>
> > Again, adding support for load balancer is simple - the Manager just
need
> > to append the vmroute to the generated session id. How you get the
session
> > id is the problem - it is available in request, or it could be
configured
> > in server.xml if getting from the request is not possible.
> >
>
> The request (and therefore the response because they maintain mutual
> references) knows what the session id is ...  is that good enough?
>
> > ( right now I'm trying to finish jk2 asap - when I'm done I'll try again
> > to fix the lb for 40 if nobody does it before )
> >
> > Costin
> >
>
> Craig
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
>


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

["Craig R. McClanahan" <cr...@apache.org>]

On Tue, 18 Dec 2001 costinm@covalent.net wrote:

> Date: Tue, 18 Dec 2001 14:14:10 -0800 (PST)
> From: costinm@covalent.net
> Reply-To: Tomcat Developers List <to...@jakarta.apache.org>
> To: Tomcat Developers List <to...@jakarta.apache.org>
> Subject: Re: Load balancing - fail-over support with mod_webapp
>
> On Tue, 18 Dec 2001, Craig R. McClanahan wrote:
>
> > The load balancer routing from JK hasn't ever been implemented in
> > Catalina yet.  Patches welcome :-).
>
> Can we get a 'setRequest' method on Manager ? I'm trying to find
> workarounds, but that would be the simple solution.
>

In other words, a *single* "request" property for the entire Manager?
Wouldn't that have problems when you had simultaneous requests in
progress?

Same thing would happen (but on a smaller scale) if you added setRequest
on the session instead -- it's legal to have multiple simultaneous
requests.

> Again, adding support for load balancer is simple - the Manager just need
> to append the vmroute to the generated session id. How you get the session
> id is the problem - it is available in request, or it could be configured
> in server.xml if getting from the request is not possible.
>

The request (and therefore the response because they maintain mutual
references) knows what the session id is ...  is that good enough?

> ( right now I'm trying to finish jk2 asap - when I'm done I'll try again
> to fix the lb for 40 if nobody does it before )
>
> Costin
>

Craig


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

[co...@covalent.net]

On Tue, 18 Dec 2001, Craig R. McClanahan wrote:

> The load balancer routing from JK hasn't ever been implemented in
> Catalina yet.  Patches welcome :-).

Can we get a 'setRequest' method on Manager ? I'm trying to find
workarounds, but that would be the simple solution.

Again, adding support for load balancer is simple - the Manager just need
to append the vmroute to the generated session id. How you get the session
id is the problem - it is available in request, or it could be configured
in server.xml if getting from the request is not possible.

( right now I'm trying to finish jk2 asap - when I'm done I'll try again
to fix the lb for 40 if nobody does it before )

Costin


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

["Craig R. McClanahan" <cr...@apache.org>]

On Tue, 18 Dec 2001, Tom Drake wrote:

> Date: Tue, 18 Dec 2001 13:25:07 -0800
> From: Tom Drake <rt...@pobox.com>
> Reply-To: Tomcat Developers List <to...@jakarta.apache.org>,
>      Tom Drake <rt...@pobox.com>
> To: Tomcat Developers List <to...@jakarta.apache.org>
> Subject: Re: Load balancing - fail-over support with mod_webapp
>
> Mathias:
>
> In Tomcat 4, unless I'm not reading the code correctly, or completely,
> it doesn't appear that that any routing information is stored in the session
> id.
>
> Can anyone comment on this?
>

The load balancer routing from JK hasn't ever been implemented in
Catalina yet.  Patches welcome :-).

> Tom
>

Craig


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

[Tom Drake <rt...@pobox.com>]

Mathias:

In Tomcat 4, unless I'm not reading the code correctly, or completely,
it doesn't appear that that any routing information is stored in the session
id.

Can anyone comment on this?

Tom


----- Original Message -----
From: "Mathias Herberts" <Ma...@iroise.net>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: Tuesday, December 18, 2001 12:23 PM
Subject: Re: Load balancing - fail-over support with mod_webapp


| Tom Drake wrote:
| >
| > Mathias;
| >
| > Thanks for your very informative reply. From your reply, it seems
| > that mod_jk comes close to doing what I want, and should actually
| > provide the fail-over I'm looking for, however not in the most
| > optimal way.
| >
| > What would be involved in adding (optionally configured) logic to
| > mod_jk that would prefer another running instance over a downed
| > instance, and then remember this as the 'jvmroute' for the given
| > JSESSIONID?
|
| Humm the design Gal Shachor did for mod_jk is pretty slick, there
| is no need to maintain any session table in Apache memory as the
| jvmroute
| is part of the session id.
|
| Therefore I do not see any way coherent with that design to re-route
| a given session id (containing a jvmroute not to be used) to a new
| Tomcat in a sticky way.
|
| My best shot would be, the first time a request is received on a Tomcat
| which did not issue it, to reissue Set-Cookie and Set-Cookie2 headers to
| force a new session id to be used. Therefore the mapping would be
| explicit in
| the new session id. The new session id would have to be used also in URL
| rewritings.
|
| This solution would provide the re-mapping you requested and the new
| Tomcat would
| become the new session parent.
|
| There are though several drawbacks to issueing a new session id.
|
| First the marketing guys won't be happy :-) Their client tracking would
| probably suffer from this change of session id, even if the end user
| doesn't notice anything.
|
| Second, depending if you use load balancing equipment before the Apache
| servers, and what kind of stickyness you
| use those equipments with, you could end up with non optimal routings of
| requests, but that's only minor since
| it will only apply to sessions that were on the downed worker initially.
|
| > Can you provide any insights into mod_webapp?
|
| Nope, I only dug into mod_jk for now.
|
| Mathias.
|
| --
| To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
| For additional commands, e-mail:
<ma...@jakarta.apache.org>
|
|
|


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

[Mathias Herberts <Ma...@iroise.net>]

Tom Drake wrote:
> 
> Mathias;
> 
> Thanks for your very informative reply. From your reply, it seems
> that mod_jk comes close to doing what I want, and should actually
> provide the fail-over I'm looking for, however not in the most
> optimal way.
> 
> What would be involved in adding (optionally configured) logic to
> mod_jk that would prefer another running instance over a downed
> instance, and then remember this as the 'jvmroute' for the given
> JSESSIONID?

Humm the design Gal Shachor did for mod_jk is pretty slick, there
is no need to maintain any session table in Apache memory as the
jvmroute
is part of the session id.

Therefore I do not see any way coherent with that design to re-route
a given session id (containing a jvmroute not to be used) to a new
Tomcat in a sticky way.

My best shot would be, the first time a request is received on a Tomcat
which did not issue it, to reissue Set-Cookie and Set-Cookie2 headers to
force a new session id to be used. Therefore the mapping would be
explicit in
the new session id. The new session id would have to be used also in URL
rewritings.

This solution would provide the re-mapping you requested and the new
Tomcat would
become the new session parent.

There are though several drawbacks to issueing a new session id.

First the marketing guys won't be happy :-) Their client tracking would
probably suffer from this change of session id, even if the end user
doesn't notice anything.

Second, depending if you use load balancing equipment before the Apache
servers, and what kind of stickyness you
use those equipments with, you could end up with non optimal routings of
requests, but that's only minor since
it will only apply to sessions that were on the downed worker initially.

> Can you provide any insights into mod_webapp?

Nope, I only dug into mod_jk for now.

Mathias.

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

[Tom Drake <rt...@pobox.com>]

Mathias;

Thanks for your very informative reply. From your reply, it seems
that mod_jk comes close to doing what I want, and should actually
provide the fail-over I'm looking for, however not in the most
optimal way.

What would be involved in adding (optionally configured) logic to
mod_jk that would prefer another running instance over a downed
instance, and then remember this as the 'jvmroute' for the given
JSESSIONID?

Can you provide any insights into mod_webapp?

Tom

----- Original Message -----
From: "Mathias Herberts" <Ma...@iroise.net>
To: "Tomcat Developers List" <to...@jakarta.apache.org>;
<to...@jakarta.apache.org>
Sent: Tuesday, December 18, 2001 10:28 AM
Subject: Re: Load balancing - fail-over support with mod_webapp


| Hi, I have been working on load balancing Apache/Tomcat clusters.
|
| In mod_jk, the fail-over is done in the function
| get_most_suitable_worker. Basically if the worker who initiated the
| session is available (i.e. not in error state), use it, otherwise loop
| through the workers and select either the one with the lowest load
| balancing value OR one that has been in error for at least
| WAIT_BEFORE_RECOVER seconds if such worker exists. Thus the faulted
| workers are privileged when selecting a worker for a request whose
| worker designated by the jvmroute is not available.
|
| There is though no algorithm implemented which will map all requests to
| the same 'backup' worker, thus either the worker which created the
| session (the one in the jvmroute) is available, either another worker
| will be selected, but in the latter case there is no guarantee that
| 'second choice' will be the same for the next requests.
|
| Therefore re-routing does exist in mod_jk and works, but it is not
| optimal for your case, the worst thing being the
| dispatching of incoming requests to a different worker each time, your
| distributed session management would therefore give the parenting of the
| session to a different worker each time, far from optimal!
|
| Mathias.
|
| --
| To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
| For additional commands, e-mail:
<ma...@jakarta.apache.org>
|
|
|


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Load balancing - fail-over support with mod_webapp

[co...@covalent.net]

Hi Mathias,

Maybe you could contribute some docs, you seem very familiar with the
subject ! Code would be even better :-)

In jk2 it should be possible for 'callbacks' to change anything, and all
the information about lb is exposed in accessible structures ( private
data has been greatly reduced ).

That means a tomcat that takes over for a falled one could ( during the
negotiation phase ) add a mapping ( probably we need a 'redirect' table
in uriMap ).

BTW, one thing that we should add is 'greacefull shutdown' - that will
also be done ( probably ) using a callback ( i.e. have only sticky
requests forwarded to a tomcat instance ).

Costin

On Tue, 18 Dec 2001, Mathias Herberts wrote:

> Hi, I have been working on load balancing Apache/Tomcat clusters.
>
> In mod_jk, the fail-over is done in the function
> get_most_suitable_worker. Basically if the worker who initiated the
> session is available (i.e. not in error state), use it, otherwise loop
> through the workers and select either the one with the lowest load
> balancing value OR one that has been in error for at least
> WAIT_BEFORE_RECOVER seconds if such worker exists. Thus the faulted
> workers are privileged when selecting a worker for a request whose
> worker designated by the jvmroute is not available.
>
> There is though no algorithm implemented which will map all requests to
> the same 'backup' worker, thus either the worker which created the
> session (the one in the jvmroute) is available, either another worker
> will be selected, but in the latter case there is no guarantee that
> 'second choice' will be the same for the next requests.
>
> Therefore re-routing does exist in mod_jk and works, but it is not
> optimal for your case, the worst thing being the
> dispatching of incoming requests to a different worker each time, your
> distributed session management would therefore give the parenting of the
> session to a different worker each time, far from optimal!
>
> Mathias.
>
> --
> To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>