Re: mod_jk and session stickiness

[Christopher Schultz <ch...@christopherschultz.net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

All,

On 6/17/14, 10:43 AM, Christopher Schultz wrote:
> All,
> 
> I've been using sticky sessions with mod_jk and I can see that
> there is a bit of a problem when attempting to take a backend
> Tomcat server out of load-balanced rotation: a user who never (or
> rarely) restarts their web browser will keep the same JSESSIONID
> cookie forever and therefore end up with the same backend server
> whether it has been disabled or not.
> 
> Quick series of events:
> 
> 1. User visits load-balancer and gets a randomly-assigned backend 
> server/route. We'll call this route "X". The JSESSIONID cookie set
> by the backend server is therefore foo.X.
> 
> 2. User's requests are routed by mod_jk to route X.
> 
> 3. Route X is disabled using mod_jk's status worker
> 
> 4. User's session on server X expires.
> 
> [Technically, 3 and 4 can happen in either order]
> 
> 5. User makes a new request to the load-balancer, and mod_jk sees
> the JSESSIONID cookie still set to foo.X. mod_jk sends the request
> to route X which allows the user to login, etc.
> 
> Thus, it takes more time than necessary to bleed all the traffic
> from route X for maintenance, etc.
> 
> Is there a way for mod_jk to ask route X if the session is *still* 
> valid? It seems that mod_jk will not re-route a request that looks 
> like it's got a valid session id to a new (active) backend server 
> unless the backend server X is actually down.
> 
> Any ideas?

Any takers?

I was thinking that the following strategy might work:

1. Extend mod_jk's load-balancer worker to include a new directive:
rebalance_statuses, then set the value for that directive to some very
rare HTTP status code(s) like 502.

2. Add a Valve/Filter/Whatever to Tomcat that, when activated because
you want to take a node out of service) returns a 502 when a session
identifier is used that does not map to a currently-valid session and
there is no "force-new-session" header included in the request with
some unpredictable value to prevent end-users from sending their own
valid headers. This could also be handled by the same component
(Manager) that current creates sessions with a new configuration option.

These two together would then bleed-off the aforementioned users: in
step 5, the request would be routed to Tomcat X, then rejected with a
502. mod_jk would detect this and re-balance the worker. If mod_jk
can't re-balance the worker to another backend (e.g. they are all
disabled, stopped, or in a failure state), then mod_jk would send the
request (back) to Tomcat X with the force-new-session header set.

Other than the above (or something similar), or actively probing a
backend Tomcat to see if a session is valid, does anyone have any
ideas for how to get a backend Tomcat into a completely idle state
before pulling the plug?

Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTu+RDAAoJEBzwKT+lPKRYWtUP/Rln9w5ou2QO78qwZZ2WX7Cz
PM13BKdnqHZmG1cyDlOPC17bhGynI8xkee+o/ivk1aIsiBeaavhF5M3iBwH9I9fO
/GX/ILKUGn11I3X8E61dGHFxziUVXpBMotOtSvb3+JMo0eMOI1rTo/5gi/GnvXSr
cFkegZ/0IR1PCHVEB++I6HayoyvXLladT5F4FG8vQjQdKHqPX2lt3RFmHdjoyC6D
tgG215w2WPn0arzLvrCvmP07mOeEXNkUf2ai/u9FEx6Dm8poDXIPD9MMS30aUAm9
pFwL0Nc7MhvtUYMj8XmbnnfEMVW/SaXd8u6du2s5d4+2fON7zc27rK0HWAL8OSJQ
0S59q0G3x17JqMrWdQ98eOxCMjYfBR7vBmXXwMiLaVpknxyGRR9AM1sEOQ1zRQCZ
MhHT+hm457LrRu6pSjfUL9L9tGY0kLxX/KwkbKacIoll9rC4gkTHMgwjdPso0YaZ
WMABwDxoymPJpAM1khGjeQSHtBhq31a/H/kfnZSpMZ2Ugl7giBgLVzmJ8Q9HcUqD
/nvLPj1d9Vra3a/bYwkeUiHdpIRNmUUNnu1hoVL0/g+pT0JWSk7jBxznBv5Z/h4o
LNclTOzTP2ijAYJSFMJUKFw4Hg7n91QfSq1BccLQEWg7M5VD5fPKnp/AvOlDGptu
j+2HZ8VdoriHBYoSKuzc
=/IRy
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: mod_jk and session stickiness

[Rainer Jung <ra...@kippdata.de>]

On 08.07.2014 14:29, Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> All,
>
> On 6/17/14, 10:43 AM, Christopher Schultz wrote:
>> All,
>>
>> I've been using sticky sessions with mod_jk and I can see that
>> there is a bit of a problem when attempting to take a backend
>> Tomcat server out of load-balanced rotation: a user who never (or
>> rarely) restarts their web browser will keep the same JSESSIONID
>> cookie forever and therefore end up with the same backend server
>> whether it has been disabled or not.
>>
>> Quick series of events:
>>
>> 1. User visits load-balancer and gets a randomly-assigned backend
>> server/route. We'll call this route "X". The JSESSIONID cookie set
>> by the backend server is therefore foo.X.
>>
>> 2. User's requests are routed by mod_jk to route X.
>>
>> 3. Route X is disabled using mod_jk's status worker
>>
>> 4. User's session on server X expires.
>>
>> [Technically, 3 and 4 can happen in either order]
>>
>> 5. User makes a new request to the load-balancer, and mod_jk sees
>> the JSESSIONID cookie still set to foo.X. mod_jk sends the request
>> to route X which allows the user to login, etc.
>>
>> Thus, it takes more time than necessary to bleed all the traffic
>> from route X for maintenance, etc.
>>
>> Is there a way for mod_jk to ask route X if the session is *still*
>> valid? It seems that mod_jk will not re-route a request that looks
>> like it's got a valid session id to a new (active) backend server
>> unless the backend server X is actually down.
>>
>> Any ideas?
>
> Any takers?
>
> I was thinking that the following strategy might work:
>
> 1. Extend mod_jk's load-balancer worker to include a new directive:
> rebalance_statuses, then set the value for that directive to some very
> rare HTTP status code(s) like 502.
>
> 2. Add a Valve/Filter/Whatever to Tomcat that, when activated because
> you want to take a node out of service) returns a 502 when a session
> identifier is used that does not map to a currently-valid session and
> there is no "force-new-session" header included in the request with
> some unpredictable value to prevent end-users from sending their own
> valid headers. This could also be handled by the same component
> (Manager) that current creates sessions with a new configuration option.
>
> These two together would then bleed-off the aforementioned users: in
> step 5, the request would be routed to Tomcat X, then rejected with a
> 502. mod_jk would detect this and re-balance the worker. If mod_jk
> can't re-balance the worker to another backend (e.g. they are all
> disabled, stopped, or in a failure state), then mod_jk would send the
> request (back) to Tomcat X with the force-new-session header set.
>
> Other than the above (or something similar), or actively probing a
> backend Tomcat to see if a session is valid, does anyone have any
> ideas for how to get a backend Tomcat into a completely idle state
> before pulling the plug?

The status code (plus header) idea could be done. Probably not to 
difficult. Will not necessarily work for POST, but will suffice in most 
cases.

The check-valid-session stuff would be harder. First we need to run an 
http request on our own (which would be useful also for regular 
probing), but then it is probably also a performance issue. We can't 
simply piggyback the session id on cping/cpong, because the container 
needs to at least get the URL to determine the correct context and run 
the mapper. IMHO to much to force on every request, even for a node with 
activation disabled.

See also my other response.

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org