adding communication between mod_jk and servlet code

[Chris Lamprecht <cl...@gmail.com>]

Hi all,
I previously submitted the lb 'busyness' patch for mod_jk, and it has worked
well for us.

Now we need some additional functionality from mod_jk and I'd like to get
feedback here first before I start patching.  I'd prefer to implement
something that is generally useful to others, and submit the patch, so it
becomes part of the mod_jk codebase.

First, the problem.  Currently there isn't any direct way for a servlet to
tell mod_jk that it should be marked as being in 'Error' state (as being
down).  We have a backend HTTP service that requires high availability.  If
the servlet detects that something is wrong, it should "take itself out" of
the mod_jk load balancer, at least temporarily.  There are two ways I can
think of to accomplish this right now.  The first, which I've done, is set
the reply_timeout to a fairly low value, like 4000ms, and then make the
servlet intentionally sleep for 5000ms to cause it to go into Error state.
This works, but having the reply_timeout so low is a risk -- if the backend
servers are slow for some reason (copying large files, etc), they can all be
marked as down -- oops.  The second method, which I have not tried, is to
have the servlet do a GET request to the mod_jk JkStatus page, to take
itself out of rotation.  While it may work, I just don't have good feelings
about this method.  If anyone has  other suggestions on ways for a servlet
to force itself to be marked as in 'Error' state, please reply.

So, I looked through mod_jk code hoping that I could have the servlet throw
an HTTP 50x error that mod_jk would interpret as "down", but no such luck.
There has been talk before about writing a "meta" communication layer
between servlets and mod_jk.  This could open up the doors to some cool
things:  Servlets could provide JK with load information, average response
times, available webapp contexts, etc.  See this thread:
http://mail-archives.apache.org/mod_mbox/httpd-dev/200606.mbox/%3C449703AA.2070703@kippdata.de%3E

For my current purposes, I just need the servlet to tell mod_jk "even though
I'm available, mark me as in Error state!".   How should we do this?  Using
a "hidden" HTTP response header might work for communicating general
strings, or returning certain "reserved" HTTP responses (HTTP 599 = error,
etc) -- these are two options that come to mind.  I'm willing to implement
and test the mods, including under heavy load.


Thanks,
Chris

Re: adding communication between mod_jk and servlet code

[Rainer Jung <ra...@kippdata.de>]

Hello Chris,

thanks for your original "Busyness" contribution.

I'm not a big fan of transporting container meta information on top of 
usual request/response traffic. I know we don't have anything else we 
could easily use yet.

1) There is no built in protocol support in ajp13 for other types of 
information. There is no effort in realising ajp14 at the moment.

2) There is no easy way of supporting parallel management activity 
besides normal workload (creating management threads etc.), because we 
are multi-platform and multi-webserver.

I'm thinking about adding more functionality to the status worker as an 
interesting compromise:

- the status worker can run in parallel to usual request load. It needs 
to be triggered externally, but at least we don't pack something on top 
of every request handling.

- the status worker can change state in the shared memory segment we 
use. Any load balancer already uses this state.

So one could think about:

- adding a backend status URL to each load balancer (configuration)
- adding the ability to send requests via ajp13 to backends from inside 
the status worker (we have to fake the request) and to extract 
information from the response
- defining a good response content scheme
- reacting on the response by disabling/stopping/erring/reenabling etc. 
load balancer members
- defining request parameters (cmd=probe&...) to the status worker, that 
trigger the request to every member of a load balancer and the 
corresponding action depending on the response

How does that sound?

Regards,

Rainer

Chris Lamprecht wrote:
> Hi all,
> I previously submitted the lb 'busyness' patch for mod_jk, and it has 
> worked
> well for us.
> 
> Now we need some additional functionality from mod_jk and I'd like to get
> feedback here first before I start patching.  I'd prefer to implement
> something that is generally useful to others, and submit the patch, so it
> becomes part of the mod_jk codebase.
> 
> First, the problem.  Currently there isn't any direct way for a servlet to
> tell mod_jk that it should be marked as being in 'Error' state (as being
> down).  We have a backend HTTP service that requires high availability.  If
> the servlet detects that something is wrong, it should "take itself out" of
> the mod_jk load balancer, at least temporarily.  There are two ways I can
> think of to accomplish this right now.  The first, which I've done, is set
> the reply_timeout to a fairly low value, like 4000ms, and then make the
> servlet intentionally sleep for 5000ms to cause it to go into Error state.
> This works, but having the reply_timeout so low is a risk -- if the backend
> servers are slow for some reason (copying large files, etc), they can 
> all be
> marked as down -- oops.  The second method, which I have not tried, is to
> have the servlet do a GET request to the mod_jk JkStatus page, to take
> itself out of rotation.  While it may work, I just don't have good feelings
> about this method.  If anyone has  other suggestions on ways for a servlet
> to force itself to be marked as in 'Error' state, please reply.
> 
> So, I looked through mod_jk code hoping that I could have the servlet throw
> an HTTP 50x error that mod_jk would interpret as "down", but no such luck.
> There has been talk before about writing a "meta" communication layer
> between servlets and mod_jk.  This could open up the doors to some cool
> things:  Servlets could provide JK with load information, average response
> times, available webapp contexts, etc.  See this thread:
> http://mail-archives.apache.org/mod_mbox/httpd-dev/200606.mbox/%3C449703AA.2070703@kippdata.de%3E 
> 
> 
> For my current purposes, I just need the servlet to tell mod_jk "even 
> though
> I'm available, mark me as in Error state!".   How should we do this?  Using
> a "hidden" HTTP response header might work for communicating general
> strings, or returning certain "reserved" HTTP responses (HTTP 599 = error,
> etc) -- these are two options that come to mind.  I'm willing to implement
> and test the mods, including under heavy load.
> 
> 
> Thanks,
> Chris

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org