You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by sh...@brm.com on 2000/12/21 19:54:05 UTC
Tomcat session replicator
Hi,
Two weeks ago I posted note here saying I'm going to write patch for Tomcat
3.2 to support redundancy, in manner of having session information stored
between reloads and shared between tomcat instances.
In order to support tomcat redundancy I thought on implementing that in two
phases:
1. (dumping) While shutting down Tomcat it will save sessions in file
and reload them while going up. This feature will allow restarting tomcat
without loosing state.
2. (replication) Sending session from one Tomcat to another (AKA:
buddy). This will allow the session to have backup on another machine.
mod_jk will send user connection requests to the second machine while the
original machine is unavailable.
Number 1. above is already implemented, and 2. is going to be finish soon.
The question I have is: Should I implement the session replicator listener
as stand-alone connector, or incorporate in into Ajpv13? This will require
adding two commands to the protocol implementation.
Options:
1. Stand-alone connector. This will require another listener.
2. Incorporate it into Ajp13.
3. Incorporate it into Ajp13 and calling it Ajp14.
Other ideas?
Implementation details will be followed soon.
________________________
Shai Fultheim
Chief Technology Officer
BRM Seed <http://www.brm.com/>
E-Mail: shai@brm.com <ma...@brm.com>
Mobile: 972-53-866-459
Office: 972-2-5891-459
Re: Tomcat session replicator
Posted by Jason Brittain <ja...@olliance.com>.
Craig R. McClanahan wrote:
> At ApacheCon Europe (it was also presented in Orlando last March, but I missed
> it then), there was a very interesting session by Theo Schlossnagle
> <je...@cnds.jhu.edu> about the Backhand Project, which is focused on providing
> load balancing support for Apache as a web server. There are some good ideas we
> can glean from this approach (and possibly even some code???). More info at:
>
> http://www.cnds.jhu.edu/~jesus/
>
> Craig
I attended his session at ApacheCon in Orlando. I also spoke with him
about Backhand
and other high-availability and load balancing solutions (one that I
helped create).
Backhand has many features that only commercial software had before.
Many of the tricks/approaches/patters that Backhand uses have been
around for quite
a while, but just not implemented as open source software. Yes, now we
have a very
good open source example.
--
Jason Brittain
Software Engineer, Olliance Inc. http://www.Olliance.com
Current Maintainer, Locomotive Project http://www.Locomotive.org
Re: Tomcat session replicator
Posted by "Craig R. McClanahan" <Cr...@eng.sun.com>.
Jason Brittain wrote:
> Hi guys..
>
> I'm one of the other folks that have voiced some desire to implement
> distributed sessions and distributed servlet container functionality in
> Tomcat (although I only want to work on 4.0!). I'd love to collaborate
> with others on the design and implementation. More below:
>
At ApacheCon Europe (it was also presented in Orlando last March, but I missed
it then), there was a very interesting session by Theo Schlossnagle
<je...@cnds.jhu.edu> about the Backhand Project, which is focused on providing
load balancing support for Apache as a web server. There are some good ideas we
can glean from this approach (and possibly even some code???). More info at:
http://www.cnds.jhu.edu/~jesus/
Craig
Re: Tomcat session replicator
Posted by Jason Brittain <ja...@olliance.com>.
Hi guys..
I'm one of the other folks that have voiced some desire to implement
distributed sessions and distributed servlet container functionality in
Tomcat (although I only want to work on 4.0!). I'd love to collaborate
with others on the design and implementation. More below:
Craig R. McClanahan wrote:
>
> The biggest problem I have with integrating session replication into
> AJPxx is that it means this will only work if you're running Apache
> connected, versus when Tomcat is running stand alone. Even in that
> environment, does the Apache instance really need to know what's going
> on? Wouldn't it just be a case of updating the cookie to include the
> new "router" information when you migrate a session from one JVM to
> another?
>
> Personally, I'd like to see a session migration mechanism that works
> between the "back end" Tomcat instances. There are probably few
> enough of those (in most use cases) that you can maintain a spiderweb
> of persistent connections that is used only to communicate session
> migration information.
>
I'm still wrestling with how it's best designed, but I agree that it
probably has nothing to do
with AJP. It should work no matter what web server you're using. I
believe that it can.
Also, the idea of a spiderweb of TCP connections is what I had in mind
also, since then if one
machine goes down or is unreachable, the system knows almost
immediately, and can
take appropriate action with the remaining servers in the cluster. I
like it.
> Just as a small technical detail -- the servlet spec says that an
> application *must* run within a single JVM unless it marks itself as
> <distributable/> in the deployment descriptor -- don't forget to check
> for that.
>
Yep, this is a feature of "distributed servlet containers" as defined by
the Servlet 2.2 and
2.3 specs.
> As to what version of Tomcat to work on, no matter what the future
> holds it seems like the current 3.2 code base is not really
> appropriate for major innovation. I would only point out that the 4.0
> architecture has an interface (org.apache.catalina.Store) behind which
> session migration stuff can easily be hidden -- as well as support for
> persistence, swapping, and other advanced features -- without
> modifying the remainder of the servlet container.
>
I'm looking into that already, and have been following all discussion about
making (any version of) Tomcat distributable. But, I haven't begun
designing/coding yet (mostly due to lack of time).
Anyone who has ideas: I'm listening, and I'd sure like to discuss
how you think we should make this work.
--
Jason Brittain
Software Engineer, Olliance Inc. http://www.Olliance.com
Current Maintainer, Locomotive Project http://www.Locomotive.org
Re: Tomcat session replicator
Posted by "Craig R. McClanahan" <Cr...@eng.sun.com>.
Shai,
I apologize for not responding to you earlier ... substantive discussions are
getting a little lost in the noise at the moment.
For some reason, my Netscape mail reader won't let me intersperse comments -- so
I've put them at the end.
I think I read into your earlier comment that you'd be interested in being a
committer to work on this stuff. If you're still interested, you've got my +1.
Craig
shai@brm.com wrote:
> Hi,
>
> Two weeks ago I posted note here saying Im going to write patch for Tomcat
> 3.2 to support redundancy, in manner of having session information stored
> between reloads and shared between tomcat instances.
>
>
>
> In order to support tomcat redundancy I thought on implementing that in two
> phases:
>
> 1. (dumping) While shutting down Tomcat it will save sessions in file and
> reload them while going up. This feature will allow restarting tomcat
> without loosing state.
> 2. (replication) Sending session from one Tomcat to another (AKA: buddy).
> This will allow the session to have backup on another machine.
> mod_jk will send user connection requests to the second machine while
> the original machine is unavailable.
>
> Number 1. above is already implemented, and 2. is going to be finish soon.
>
> The question I have is: Should I implement the session replicator listener as
> stand-alone connector, or incorporate in into Ajpv13? This will require adding
> two commands to the protocol implementation.
>
> Options:
>
> 1. Stand-alone connector. This will require another listener.
> 2. Incorporate it into Ajp13.
> 3. Incorporate it into Ajp13 and calling it Ajp14.
>
> Other ideas?
>
> Implementation details will be followed soon.
>
> ________________________
>
> Shai Fultheim
>
> Chief Technology Officer
>
> BRM Seed
>
> E-Mail: shai@brm.com
>
> Mobile: 972-53-866-459
>
> Office: 972-2-5891-459
>
The biggest problem I have with integrating session replication into AJPxx is
that it means this will only work if you're running Apache connected, versus
when Tomcat is running stand alone. Even in that environment, does the Apache
instance really need to know what's going on? Wouldn't it just be a case of
updating the cookie to include the new "router" information when you migrate a
session from one JVM to another?
Personally, I'd like to see a session migration mechanism that works between the
"back end" Tomcat instances. There are probably few enough of those (in most
use cases) that you can maintain a spiderweb of persistent connections that is
used only to communicate session migration information.
Just as a small technical detail -- the servlet spec says that an application
*must* run within a single JVM unless it marks itself as <distributable/> in the
deployment descriptor -- don't forget to check for that.
As to what version of Tomcat to work on, no matter what the future holds it
seems like the current 3.2 code base is not really appropriate for major
innovation. I would only point out that the 4.0 architecture has an interface
(org.apache.catalina.Store) behind which session migration stuff can easily be
hidden -- as well as support for persistence, swapping, and other advanced
features -- without modifying the remainder of the servlet container. Several
other folks have expressed interest in working on this at various times, so
collaboration would be quite useful.
Craig McClanahan
Re: Tomcat session replicator
Posted by "yhs@mimic.onesourcecorp.com" <yh...@mimic.onesourcecorp.com>.
On Fri, 22 Dec 2000, Jason Brittain wrote:
> Craig R. McClanahan wrote:
>
> > "yhs@mimic.onesourcecorp.com" wrote:
> >
> >> why not do iot in shared storage and implement SSI ? thats what the
> >> mod_jserv shm file was for...a shared hunk of disk store.
> >> -Ys-
> >> yhs@mimic.onesourcecorp.com
> >
> >
> > This is certainly one valid approach. It works for cases where the servers are all
<SNIP>
>
> Comments? Suggestions?
>
> --
> Jason Brittain
do geographical load balancing too. servers closest to the user should
be able to get the hit. thats one piece im missing in my SSI
implementation. you can do it all in java. the w3c does it BTW for
their webservers.
-Ys-
yhs@mimic.onesourcecorp.com
Re: Tomcat session replicator
Posted by Jason Brittain <ja...@olliance.com>.
Craig R. McClanahan wrote:
> "yhs@mimic.onesourcecorp.com" wrote:
>
>> why not do iot in shared storage and implement SSI ? thats what the
>> mod_jserv shm file was for...a shared hunk of disk store.
>> -Ys-
>> yhs@mimic.onesourcecorp.com
>
>
> This is certainly one valid approach. It works for cases where the servers are all
> on the same machine. But you still need an approach that works on a multiple
> machine environment as well.
Yes! I want a system where you can have as many Tomcats as you
want on each machine, and as many of those machines as you want,
and they all act together as one big distributed farm of Tomcats.
The administrator should be able to configure which Tomcat instance
replicates to which other Tomcat instance(s). And, this should be
transparent to users regardless of the web server being used.
Unless I'm missing something, I don't see why we should change the
C-side of the AJP code to make all this work..
> It would be worth someone taking the time to articulate proposed use cases, from
> which we can identify desired features.
Yes, and I'll see what I can do..
> My bet is that we will end up wanting a
> variety of pluggable implementations with different functionality and performance
> characteristics.
I sure do.
Some features I'd like to add to Tomcat 4 (and use!):
* Add the functionality to Catalina to operate as a distributed Servlet
container as described in the latest Servlet specification in
stand-alone operation. This should support pluggable communication
mechanisms (plug in the protocol of your choice at runtime via a custom
config), including RMI and IP-Multicast technologies like Weblogic uses.
* Add a load balancing and fault tolerance Valve to Catalina that makes
the web server act as a load balancing HTTP redirect server. The goal
is to make it easy to install, run, and maintain a cluster of Tomcat "web
containers". This Valve should later be able to take advantage of Avalon
if Catalina is running within Avalon, and if not it should still work in
standalone mode.
More detail:
This Valve makes the web server serve HTTP redirects (HTTP 302 - Moved
Temporarily) when the request should be moved to another Tomcat instance.
It acts in concert with one or more other Tomcat instances in a
completely decentralized way. For instance, no single machine acts as
the central brain for the system.. each redirect server tries to cooperate
with any other redirect servers in the system, but each one has authority
over its own requests should it not be able to communicate with other
redirect servers in the system. These redirect servers keep track of how
many content servers there are in the system, and distribute HTTP requests
to each of the content servers in a manner dependent on the load balancing
module.
The load balancing module should be able to use many different load
balancing algorithms. The system administrator should be able to
configure which algorithm(s) to use for any given context. Load
balancing algorithms may include:
- Least Loaded Server: load can be calculated in one of many ways,
including counting requests per unit of time, asking the operating
system for the load average, current number of request Threads
running, etc.
- Weighted Percentage: larger, more powerful servers in the cluster
are given a higher percentage of the requests that each redirect
server must forward. Smaller, less capable servers will receive
a smaller percentage of the requests/workload.
- Fastest Path: a module at each data center will use ICMP pings to
determine which data center has the fastest path back to the client.
At that time, the fastest data center's own cluster of content
servers are the only remaining candidates for forwarding and serving
the request. Note that the pinger can't be written in the Java
programming language (no support for ICMP -- a potential security
issue for consumers), so that component needs to be written in
another programming language.
- Sticky Sessions: Any user's request will be sent to the same content
server that the user's last request was sent to. Identification of
a user is usually done in the form of cookies, or jsessionids.
- Round Robin: perhaps the least intelligent algorithm, but the simplest
to understand, set up, and try out. Redirect servers simply redirect
requests one to each server in the list, looping back to the first
server when the last server in the list is used.
The load balancing module should allow pluggable load balancing
algorithms to work in conjunction with each other. The administrator
should be able to assign a decision making "weights" to each
algorithm so that the decision about where to send a request can
be made after considering more than just one algorithm's suggestion.
Some algorithms, though, should never be used in combination with
others (for example, Sticky Sessions.. what if you use it with two
other algorithms and they override the Sticky Session and send the
request elsewhere where it doesn't have its session replicated?).
So, there should be two main types of load balancing algorithms:
Democratic algorithms, and Dictator algorithms. Dictator algorithms
don't vote on where to send the request, they just send it.
Democratic algorithms work together, voting about where to send
a request, and each algorithm's vote is weighted.
Regardless of the load balancing algorithm(s) used, the request will
never be redirected to a server that is down. For this to work, the
load balancer must be able to maintain contact with each content server.
This contact should be configurable as well. Possible choices may
include:
- TCP Socket Keepalive: each redirect server opens a single TCP socket
connection to each content server (to something like an echo service)
and keeps the connection alive by sending data to the content server
to be echoed back to the redirect server. If the TCP connection
breaks and cannot be reestablished, the redirect server considers
that content server to be unavailable and will not send any requests
to it again until the TCP connection can be reestablished.
- UDP Ping: each redirect server periodically sends a UDP message out
to a content server (the content server implements a UDP echo service)
to be echoed back to the redirect server. If packets are lost for
longer than a configurable timeout interval, then the redirect server
considers that content server to be down until it begins echoing back
the UDP ping messages.
Comments? Suggestions?
--
Jason Brittain
Software Engineer, Olliance Inc. http://www.Olliance.com
Current Maintainer, Locomotive Project http://www.Locomotive.org
Re: Tomcat session replicator
Posted by "yhs@mimic.onesourcecorp.com" <yh...@mimic.onesourcecorp.com>.
On Fri, 22 Dec 2000, Craig R. McClanahan wrote:
> "yhs@mimic.onesourcecorp.com" wrote:
>
> > why not do iot in shared storage and implement SSI ? thats what the
> > mod_jserv shm file was for...a shared hunk of disk store.
> > -Ys-
> > yhs@mimic.onesourcecorp.com
>
> This is certainly one valid approach. It works for cases where the servers are all
> on the same machine. But you still need an approach that works on a multiple
> machine environment as well.
>
> It would be worth someone taking the time to articulate proposed use cases, from
> which we can identify desired features. My bet is that we will end up wanting a
> variety of pluggable implementations with different functionality and performance
> characteristics.
>
> Craig
>
theres no reason to have the shared disk store PER SERVER like
mod_jserv. you simply distribute the data to all the servers which are in
the shared pool. i've done that and thats how i implemented
it. unfortunately ive signed an NDA so i cant distribute...but my point is
that if i can do it im sure you guys can do it better.
-Ys-
yhs@mimic.onesourcecorp.com
Re: Tomcat session replicator
Posted by "Craig R. McClanahan" <Cr...@eng.sun.com>.
"yhs@mimic.onesourcecorp.com" wrote:
> why not do iot in shared storage and implement SSI ? thats what the
> mod_jserv shm file was for...a shared hunk of disk store.
> -Ys-
> yhs@mimic.onesourcecorp.com
This is certainly one valid approach. It works for cases where the servers are all
on the same machine. But you still need an approach that works on a multiple
machine environment as well.
It would be worth someone taking the time to articulate proposed use cases, from
which we can identify desired features. My bet is that we will end up wanting a
variety of pluggable implementations with different functionality and performance
characteristics.
Craig
Re: Tomcat session replicator
Posted by "yhs@mimic.onesourcecorp.com" <yh...@mimic.onesourcecorp.com>.
On Thu, 21 Dec 2000 cmanolache@yahoo.com wrote:
> > Two weeks ago I posted note here saying I'm going to write patch for Tomcat
> > 3.2 to support redundancy, in manner of having session information stored
> > between reloads and shared between tomcat instances.
> > In order to support tomcat redundancy I thought on implementing that in two
> > phases:
>
>
> > 1. Stand-alone connector. This will require another listener.
> > 2. Incorporate it into Ajp13.
> > 3. Incorporate it into Ajp13 and calling it Ajp14.
>
> My preference would be wither 2 or 3.
>
> I think we should be able to add new message types to Ajp13 - that's
> needed to support the autoconfiguration anyway, and will be needed in
> future.
>
> If that's not possible - we should add what's needed and make it 14.
> ( AFAIK it should be - I remember this was a thing Gal had in mind when he
> wrote the code )
>
> Costin
>
why not do iot in shared storage and implement SSI ? thats what the
mod_jserv shm file was for...a shared hunk of disk store.
-Ys-
yhs@mimic.onesourcecorp.com
Re: Tomcat session replicator
Posted by cm...@yahoo.com.
> Two weeks ago I posted note here saying I'm going to write patch for Tomcat
> 3.2 to support redundancy, in manner of having session information stored
> between reloads and shared between tomcat instances.
> In order to support tomcat redundancy I thought on implementing that in two
> phases:
> 1. Stand-alone connector. This will require another listener.
> 2. Incorporate it into Ajp13.
> 3. Incorporate it into Ajp13 and calling it Ajp14.
My preference would be wither 2 or 3.
I think we should be able to add new message types to Ajp13 - that's
needed to support the autoconfiguration anyway, and will be needed in
future.
If that's not possible - we should add what's needed and make it 14.
( AFAIK it should be - I remember this was a thing Gal had in mind when he
wrote the code )
Costin