You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2014/06/17 16:43:54 UTC
mod_jk and session stickiness
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
All,
I've been using sticky sessions with mod_jk and I can see that there
is a bit of a problem when attempting to take a backend Tomcat server
out of load-balanced rotation: a user who never (or rarely) restarts
their web browser will keep the same JSESSIONID cookie forever and
therefore end up with the same backend server whether it has been
disabled or not.
Quick series of events:
1. User visits load-balancer and gets a randomly-assigned backend
server/route. We'll call this route "X". The JSESSIONID cookie set by
the backend server is therefore foo.X.
2. User's requests are routed by mod_jk to route X.
3. Route X is disabled using mod_jk's status worker
4. User's session on server X expires.
[Technically, 3 and 4 can happen in either order]
5. User makes a new request to the load-balancer, and mod_jk sees the
JSESSIONID cookie still set to foo.X. mod_jk sends the request to
route X which allows the user to login, etc.
Thus, it takes more time than necessary to bleed all the traffic from
route X for maintenance, etc.
Is there a way for mod_jk to ask route X if the session is *still*
valid? It seems that mod_jk will not re-route a request that looks
like it's got a valid session id to a new (active) backend server
unless the backend server X is actually down.
Any ideas?
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJToFQqAAoJEBzwKT+lPKRYQgEP/0oIOU1bmm6z0aKlvuW4oVnB
5ozq6hyXCQlpIN1/eORIZdnpz3vkqsFSmsVfyZLpfgJcWjn3yjiKt9M1Ywe9MQyH
mezY1Pd5ZJoUfoopHTyw3i2cxeWfvKMJTAe1nZ1XaOHoYc+iYGGrRxDYExQq2CDV
V6d55ftH9iFuMADPwcgdWhydap8A0ojeskpKsn6UEV3c1qO8uGMKvDpDb4/bnh1S
m5Ri5/Jc8mFnIBQEWOeAIFlUu9VciFQ7XF560ERCAoZ6NGhBVFJjdeEFM1iWprO/
kBawz5IXUAcNHqcZn5CNTvumsQKpGQpw54l2+iUkGUmkbBWrR2CULIzulLLdLnHF
BteWdt7qZFWgth7l4nY+SR5MpzNH3NBzzD7Cr+l861OGDzTG+A4rw81B5mS/Hmuf
/bsbnnIKeS+uREx1tR/F2+/omhhFaRpleyRFrH+NAEtU7sw1th+HO4mB3eSoioiN
xNhk/SsWWsslJH3MrD3Bi/iw3ULp5JYiaLCoD+yX18P06TzBxO7aQBK9LuCU58G4
1gN4rm/2+z33YS9URhEohWDJtTXzIEbwA7g94+oB1eAdPksWy1+0oPz7IAKyzCyb
bHZnI+v/D5w9h8lcMwJbfEx7Lg+0vxnAe4PBef6rZfhTynB2tYKsC6U/nFgwwh73
s6N1b1A+N1uKM5Ab62Jl
=X1DE
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rainer,
On 1/30/15 5:57 AM, Rainer Jung wrote:
> Am 30.01.2015 um 03:06 schrieb Christopher Schultz:
>> On 7/23/14 5:14 PM, Christopher Schultz wrote:
>>> Rainer,
>>>
>>> On 7/23/14, 4:10 PM, Rainer Jung wrote:
>>>> Am 23.07.2014 um 21:03 schrieb Christopher Schultz:
>>>
>>>>> On 7/23/14, 1:49 PM, Rainer Jung wrote:
>>>
>>>>>> The other case, a request with an invalid session ID
>>>>>> accessing a tomcat instance with activation disabled can
>>>>>> IMHO be handled by a filter that
>>>>>>
>>>>>> - checks whether the request has a valid session using
>>>>>> getSession(false), if it has one, let the request
>>>>>> proceed
>>>>>>
>>>>>> - checks activation state, if "active", let the request
>>>>>> proceed
>>>>>>
>>>>>> - checks the request method, if not GET, let the request
>>>>>> proceed
>>>>>>
>>>>>> - otherwise:
>>>>>>
>>>>>> - set the session cookie, e.g. JSESSIONID the an empty
>>>>>> value - issue an external redirect to the same request
>>>>>> URL - optional redirect loop detection: add a query
>>>>>> string param or cookie that gets the local jvmRoute
>>>>>> appended during each redirect. Before doing the redirect,
>>>>>> check that the local jvmRoute is not already part of that
>>>>>> token (we have already been here before)
>>>>>>
>>>>>> This would not really interfere with your saved
>>>>>> requests: they would get a redirect which the browser
>>>>>> follwos automatically and after that you will observe the
>>>>>> normal behavior.
>>>>>
>>>>> This is exactly what I have implemented -- as a Filter
>>>>> since we can insert it before SecurityFilter intercepts the
>>>>> requests -- and my tests suggest that it will work
>>>>> correctly.
>>>>>
>>>>> I added code to strip-out any ;jsessionid path parameter
>>>>> from the URL
>>>
>>>> ACK
>>>
>>>>> if it exists, but haven't done any of the redirect loop
>>>>> detection (yet). I think the loop detection is going to
>>>>> have to keep a running list of visited nodes which, in
>>>>> large clusters, could grow very large especially if the
>>>>> node names are long. I'll post my code when it's a little
>>>>> more featureful.
>>>
>>>> As long as nothing goes wrong, the first redirect - having
>>>> no more session info - should not end up in getting
>>>> redirected as well, because it should only be send to an
>>>> active node by mod_jk. So you can even stop redirecting if
>>>> you detect, that it already happened once. For a large
>>>> cluster that might be better.
>>>
>>>> Multiple might happen (didn't check the code), if all active
>>>> members are in error state. Even then one would not like to
>>>> produce many of those redirects for one request, so again,
>>>> only allowing one redirect might be the right solution.
>>>
>>> That's a good thought, and simplifies things. The request
>>> parameter could then just be a boolean flag and not actually
>>> identify the node that produced the redirect: any node
>>> considering redirecting simply would not redirect if that flag
>>> had been set.
>>
>> So, I've implemented all of the above and just given it a real
>> test-drive in production. It seems to work well on the first
>> shot.
>
> Good!
>
>> I'll be testing it a bit more as time does on, actually. I also
>> converted it to a Valve that I will propose to include in Tomcat
>> once I get some real mileage on it.
>
> Was a Valve more useful than a filter? Did you need to access
> Tomcat internals?
I can do everything without using Tomcat internals *except* for
determining whether or not the cookie path should end with a "/". I'll
have to make that part of the configuration, and have it default to
"true". That's the only part that isn't exposed by the Servlet API.
(This was one of the things that bit me when deploying this recently.)
I got crickets when posting a while back to the dev list:
http://markmail.org/thread/4oldeombollmp2dw
Feel free to lend your opinion.
>> I also discovered that I'd like to have a new feature: the
>> ability for a whitelisted client to ignore the re-balance rules
>> and continue to contact the server. So, if you have a cluster
>> member called, say, "foo" and you want to force your way into
>> "foo", you can make a request to any page with a jsessionid
>> parameter:
>>
>> http://host/some_page;jsessionid=1234.foo
>>
>> mod_jk will send you to the "foo" node, even though it is in the
>> DISABLED state.
>>
>> Normally, my Filter/Valve would see that you have no valid
>> session ("1234.foo" is not valid) and perform a redirect without
>> the "jsessionid" path parameter added, thus re-balancing you to
>> another node.
>>
>> Instead, I've added the ability to configure an "ignore" cookie
>> and cookie value (like "ignore=SECRETCODE"). If the client
>> presents this cookie to the Filter/Valve, then it's business as
>> usual.
>>
>> This allows a trusted user to log into the DISABLED node through
>> the load-balancer to take a look around, make sure that things
>> are working properly, etc. before re-enabling the node in the
>> cluster. I'll give this a shot in the next few days, clean it all
>> up, and publish it (soon?).
>
> That sounds useful.
I'm testing it in development right now.
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJUy5G7AAoJEBzwKT+lPKRYn3MP/3KTIAq8QP8CqyEQR1ckXLWM
668XN5YfolSnwWloaeoihEUgwbZBoXNDK9jdEqXcMEzf0cwsHipCRp/PHhejSFNy
Q7iiDGbHsiXr3fzWXEOCdKiiHLHvMYr0KVff4TefQUvK06LgS8zupgp/PJ3mymfr
xjBmBrzpg5rljZ8PeYVI6csr85E0lL15WWwCSDS+AM/NHYR6UELcvXUCwMDOINe+
0UgZwgbYvbSpIPHZ2i30WZHQCA3DJJ4RpNpm7pT6jhptp2WflAWlAAPQYaifHVlz
+VbMVzrbPLSs/JLa+rIAih5jXqYMLcsYJ1RxxkrbTHGRFcBY2IAq2eFuCxyZ5Wfq
hECQXqFJyosADx1fBNvdmnDOXb60JmYnX6r1nu0aZ1jZImsZSgsW4Ca9IoSBKv4A
NY27n1kmMvqyqskizB5IYpo/7M3PDqBhdzhF9D9aDR4Llpx5oi7PlLa/xUPghPzV
BAxO2w/qUi9UCjzCI7MHDkEV29BzmJYhNhfcaRZBLFiU86xY8no/2b2Ezxq6pOC/
J/Fmv0pZ+vWPa/F+K5abwdLr3FTiLPJa4DFazry/BfBCaH00Lb0vFXUG677dXSmK
o2CfxzSB8tXR6qAuLQHtas/FrBd/Gm9guHNh8ncjRim4kurHFFsU4v5YWeW5hP/u
IYX8/t+qznF263F9G+8U
=lZOs
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Rainer Jung <ra...@kippdata.de>.
Am 30.01.2015 um 03:06 schrieb Christopher Schultz:
> On 7/23/14 5:14 PM, Christopher Schultz wrote:
>> Rainer,
>>
>> On 7/23/14, 4:10 PM, Rainer Jung wrote:
>>> Am 23.07.2014 um 21:03 schrieb Christopher Schultz:
>>
>>>> On 7/23/14, 1:49 PM, Rainer Jung wrote:
>>
>>>>> The other case, a request with an invalid session ID
>>>>> accessing a tomcat instance with activation disabled can IMHO
>>>>> be handled by a filter that
>>>>>
>>>>> - checks whether the request has a valid session using
>>>>> getSession(false), if it has one, let the request proceed
>>>>>
>>>>> - checks activation state, if "active", let the request
>>>>> proceed
>>>>>
>>>>> - checks the request method, if not GET, let the request
>>>>> proceed
>>>>>
>>>>> - otherwise:
>>>>>
>>>>> - set the session cookie, e.g. JSESSIONID the an empty value
>>>>> - issue an external redirect to the same request URL -
>>>>> optional redirect loop detection: add a query string param or
>>>>> cookie that gets the local jvmRoute appended during each
>>>>> redirect. Before doing the redirect, check that the local
>>>>> jvmRoute is not already part of that token (we have already
>>>>> been here before)
>>>>>
>>>>> This would not really interfere with your saved requests:
>>>>> they would get a redirect which the browser follwos
>>>>> automatically and after that you will observe the normal
>>>>> behavior.
>>>>
>>>> This is exactly what I have implemented -- as a Filter since
>>>> we can insert it before SecurityFilter intercepts the requests
>>>> -- and my tests suggest that it will work correctly.
>>>>
>>>> I added code to strip-out any ;jsessionid path parameter from
>>>> the URL
>>
>>> ACK
>>
>>>> if it exists, but haven't done any of the redirect loop
>>>> detection (yet). I think the loop detection is going to have
>>>> to keep a running list of visited nodes which, in large
>>>> clusters, could grow very large especially if the node names
>>>> are long. I'll post my code when it's a little more
>>>> featureful.
>>
>>> As long as nothing goes wrong, the first redirect - having no
>>> more session info - should not end up in getting redirected as
>>> well, because it should only be send to an active node by mod_jk.
>>> So you can even stop redirecting if you detect, that it already
>>> happened once. For a large cluster that might be better.
>>
>>> Multiple might happen (didn't check the code), if all active
>>> members are in error state. Even then one would not like to
>>> produce many of those redirects for one request, so again, only
>>> allowing one redirect might be the right solution.
>>
>> That's a good thought, and simplifies things. The request
>> parameter could then just be a boolean flag and not actually
>> identify the node that produced the redirect: any node considering
>> redirecting simply would not redirect if that flag had been set.
>
> So, I've implemented all of the above and just given it a real
> test-drive in production. It seems to work well on the first shot.
Good!
> I'll be testing it a bit more as time does on, actually. I also
> converted it to a Valve that I will propose to include in Tomcat once
> I get some real mileage on it.
Was a Valve more useful than a filter? Did you need to access Tomcat
internals?
> I also discovered that I'd like to have a new feature: the ability for
> a whitelisted client to ignore the re-balance rules and continue to
> contact the server. So, if you have a cluster member called, say,
> "foo" and you want to force your way into "foo", you can make a
> request to any page with a jsessionid parameter:
>
> http://host/some_page;jsessionid=1234.foo
>
> mod_jk will send you to the "foo" node, even though it is in the
> DISABLED state.
>
> Normally, my Filter/Valve would see that you have no valid session
> ("1234.foo" is not valid) and perform a redirect without the
> "jsessionid" path parameter added, thus re-balancing you to another node.
>
> Instead, I've added the ability to configure an "ignore" cookie and
> cookie value (like "ignore=SECRETCODE"). If the client presents this
> cookie to the Filter/Valve, then it's business as usual.
>
> This allows a trusted user to log into the DISABLED node through the
> load-balancer to take a look around, make sure that things are working
> properly, etc. before re-enabling the node in the cluster. I'll give
> this a shot in the next few days, clean it all up, and publish it (soon?).
That sounds useful.
Thanks!
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rainer,
On 7/23/14 5:14 PM, Christopher Schultz wrote:
> Rainer,
>
> On 7/23/14, 4:10 PM, Rainer Jung wrote:
>> Am 23.07.2014 um 21:03 schrieb Christopher Schultz:
>
>>> On 7/23/14, 1:49 PM, Rainer Jung wrote:
>
>>>> The other case, a request with an invalid session ID
>>>> accessing a tomcat instance with activation disabled can IMHO
>>>> be handled by a filter that
>>>>
>>>> - checks whether the request has a valid session using
>>>> getSession(false), if it has one, let the request proceed
>>>>
>>>> - checks activation state, if "active", let the request
>>>> proceed
>>>>
>>>> - checks the request method, if not GET, let the request
>>>> proceed
>>>>
>>>> - otherwise:
>>>>
>>>> - set the session cookie, e.g. JSESSIONID the an empty value
>>>> - issue an external redirect to the same request URL -
>>>> optional redirect loop detection: add a query string param or
>>>> cookie that gets the local jvmRoute appended during each
>>>> redirect. Before doing the redirect, check that the local
>>>> jvmRoute is not already part of that token (we have already
>>>> been here before)
>>>>
>>>> This would not really interfere with your saved requests:
>>>> they would get a redirect which the browser follwos
>>>> automatically and after that you will observe the normal
>>>> behavior.
>>>
>>> This is exactly what I have implemented -- as a Filter since
>>> we can insert it before SecurityFilter intercepts the requests
>>> -- and my tests suggest that it will work correctly.
>>>
>>> I added code to strip-out any ;jsessionid path parameter from
>>> the URL
>
>> ACK
>
>>> if it exists, but haven't done any of the redirect loop
>>> detection (yet). I think the loop detection is going to have
>>> to keep a running list of visited nodes which, in large
>>> clusters, could grow very large especially if the node names
>>> are long. I'll post my code when it's a little more
>>> featureful.
>
>> As long as nothing goes wrong, the first redirect - having no
>> more session info - should not end up in getting redirected as
>> well, because it should only be send to an active node by mod_jk.
>> So you can even stop redirecting if you detect, that it already
>> happened once. For a large cluster that might be better.
>
>> Multiple might happen (didn't check the code), if all active
>> members are in error state. Even then one would not like to
>> produce many of those redirects for one request, so again, only
>> allowing one redirect might be the right solution.
>
> That's a good thought, and simplifies things. The request
> parameter could then just be a boolean flag and not actually
> identify the node that produced the redirect: any node considering
> redirecting simply would not redirect if that flag had been set.
So, I've implemented all of the above and just given it a real
test-drive in production. It seems to work well on the first shot.
I'll be testing it a bit more as time does on, actually. I also
converted it to a Valve that I will propose to include in Tomcat once
I get some real mileage on it.
I also discovered that I'd like to have a new feature: the ability for
a whitelisted client to ignore the re-balance rules and continue to
contact the server. So, if you have a cluster member called, say,
"foo" and you want to force your way into "foo", you can make a
request to any page with a jsessionid parameter:
http://host/some_page;jsessionid=1234.foo
mod_jk will send you to the "foo" node, even though it is in the
DISABLED state.
Normally, my Filter/Valve would see that you have no valid session
("1234.foo" is not valid) and perform a redirect without the
"jsessionid" path parameter added, thus re-balancing you to another node.
Instead, I've added the ability to configure an "ignore" cookie and
cookie value (like "ignore=SECRETCODE"). If the client presents this
cookie to the Filter/Valve, then it's business as usual.
This allows a trusted user to log into the DISABLED node through the
load-balancer to take a look around, make sure that things are working
properly, etc. before re-enabling the node in the cluster. I'll give
this a shot in the next few days, clean it all up, and publish it (soon?).
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJUyucgAAoJEBzwKT+lPKRYi8wQAIi+G/hb0IxXem+yp56n8Lw1
M2TfALSw6JeSmTTrqj6tRDW91b26zj7+3pSZQp0HBkZNiWEZfZNmurLEHe5baWCV
KmKY//zSX8LPOCrTR+hRQ95VAmRIw918OYdgFlRYEQ11Tx45UOr5ivNsAJRfARPw
KdBf3mo3mOKlxjxPEphlZdL6CJojjz0psToc412t3hPf3u2ttxKo0wSn8hDi47O3
bbdpwoA6vpCW7vLRPI22kZoJe4nbsFz0IS26smzVXjarbyrzt/YNHMPhuFSrXSIn
kQhvZeF/61/ZDbOS/54VmfoVugRlcoUiw1nm4Fpvv1xWeezZw7DteCJ/SuqvjbVo
1Y+6wYomMj91Viw8HqNBxh1nXcLUP8tYKqCT1H7MM5sjgMQFonoIdtKMSWbLO8qQ
zDi4qvJudWyGazxWsjTcy9iRyrVqjbkZ0SiVt6trKHyu7NPWBK2CcO3BoznJXs1M
fFsmBTvAFMZLicqenlniPazMQhGzrdfl2QaixPI0moUhnscYoZfUXFhrZaQ0z83v
GKL/MiQDeImZLsi1hup2C8/ML4lkPIj3TS0fP6BxMQZ1Bk6NB6EJ0yJjLq0aH/Kd
6dyax4kfRbG88wxtlzmAF1ryceoq2/hU7wHryhDBOuzLTmWCb9jfEw3pHTn6UOsZ
GKyVmm7oT9//JH8KGXQB
=n5w9
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rainer,
On 7/23/14, 4:10 PM, Rainer Jung wrote:
> Am 23.07.2014 um 21:03 schrieb Christopher Schultz:
>
>> On 7/23/14, 1:49 PM, Rainer Jung wrote:
>
>>> The other case, a request with an invalid session ID accessing
>>> a tomcat instance with activation disabled can IMHO be handled
>>> by a filter that
>>>
>>> - checks whether the request has a valid session using
>>> getSession(false), if it has one, let the request proceed
>>>
>>> - checks activation state, if "active", let the request
>>> proceed
>>>
>>> - checks the request method, if not GET, let the request
>>> proceed
>>>
>>> - otherwise:
>>>
>>> - set the session cookie, e.g. JSESSIONID the an empty value -
>>> issue an external redirect to the same request URL - optional
>>> redirect loop detection: add a query string param or cookie
>>> that gets the local jvmRoute appended during each redirect.
>>> Before doing the redirect, check that the local jvmRoute is not
>>> already part of that token (we have already been here before)
>>>
>>> This would not really interfere with your saved requests: they
>>> would get a redirect which the browser follwos automatically
>>> and after that you will observe the normal behavior.
>>
>> This is exactly what I have implemented -- as a Filter since we
>> can insert it before SecurityFilter intercepts the requests --
>> and my tests suggest that it will work correctly.
>>
>> I added code to strip-out any ;jsessionid path parameter from the
>> URL
>
> ACK
>
>> if it exists, but haven't done any of the redirect loop
>> detection (yet). I think the loop detection is going to have to
>> keep a running list of visited nodes which, in large clusters,
>> could grow very large especially if the node names are long. I'll
>> post my code when it's a little more featureful.
>
> As long as nothing goes wrong, the first redirect - having no more
> session info - should not end up in getting redirected as well,
> because it should only be send to an active node by mod_jk. So you
> can even stop redirecting if you detect, that it already happened
> once. For a large cluster that might be better.
>
> Multiple might happen (didn't check the code), if all active
> members are in error state. Even then one would not like to produce
> many of those redirects for one request, so again, only allowing
> one redirect might be the right solution.
That's a good thought, and simplifies things. The request parameter
could then just be a boolean flag and not actually identify the node
that produced the redirect: any node considering redirecting simply
would not redirect if that flag had been set.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJT0CW9AAoJEBzwKT+lPKRYPlsQAJUFuK+nBO3jty473UgIqZCK
of3S+yux9FxzowMUCIkwnXi61zazDJqOmEuPO/qvWdmxPb9/cEj8U7h9V8lC04zI
zc3Z0Fq4m9mMgmMmhi4MEu2vXuW01pw6YaePxyMsP0LqNxCb6xvp0UkQsBDCEd46
X7GJpt3q1MreFCD4lPYqDhiKm5oR3HqsJC+CmajpcfWMTg5VT4k8jX9HM5xS1p7N
bvCYbxFB8I7/QMOG7cQ2+Dok5C9h/j9MGQvlL3UXCgWS7LRLPBFNzoZn9UfUhZNz
VsEWJspIn8KETP02AGV5vFd00ydpi53ZwlySUwjznrNixrPt7R4O+yhR5dB8tyNm
Yx8NA1rtWKVoKOeBiB/r3WBnNWg2f+AzND92DmaAvRZevSoOK1OY6oeAYnoBYVg6
2Oipes/dUEalq+oeNqAh2/jAmzo195MuOfo/vCDwGLTE26j+sjEDFRqKOcnR0EmO
g2Y13hFa60ueswrI5Ixq5beFz0SRd2St6V6FBQRl7+7TRPI9wwEbYnZwT3iCL4qT
8jWP7XvWWOySLM7rEKldbcVpEElobWVtfhhjLWjQWhjXgnsdOAj0elfAzWAS+kFq
YeKJRHCbi8dka5TGv31fhfGSNgTnlUyMaZ+J+idzCNw0O+yBgIgZWPMPUsC9RqDq
5gBLNkXTUQb+VSzBbteQ
=Uwwp
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Rainer Jung <ra...@kippdata.de>.
Am 23.07.2014 um 21:03 schrieb Christopher Schultz:
> On 7/23/14, 1:49 PM, Rainer Jung wrote:
>> The other case, a request with an invalid session ID accessing a
>> tomcat instance with activation disabled can IMHO be handled by a
>> filter that
>>
>> - checks whether the request has a valid session using
>> getSession(false), if it has one, let the request proceed
>>
>> - checks activation state, if "active", let the request proceed
>>
>> - checks the request method, if not GET, let the request proceed
>>
>> - otherwise:
>>
>> - set the session cookie, e.g. JSESSIONID the an empty value -
>> issue an external redirect to the same request URL - optional
>> redirect loop detection: add a query string param or cookie that
>> gets the local jvmRoute appended during each redirect. Before
>> doing the redirect, check that the local jvmRoute is not already
>> part of that token (we have already been here before)
>>
>> This would not really interfere with your saved requests: they
>> would get a redirect which the browser follwos automatically and
>> after that you will observe the normal behavior.
>
> This is exactly what I have implemented -- as a Filter since we can
> insert it before SecurityFilter intercepts the requests -- and my
> tests suggest that it will work correctly.
>
> I added code to strip-out any ;jsessionid path parameter from the URL
ACK
> if it exists, but haven't done any of the redirect loop detection
> (yet). I think the loop detection is going to have to keep a running
> list of visited nodes which, in large clusters, could grow very large
> especially if the node names are long. I'll post my code when it's a
> little more featureful.
As long as nothing goes wrong, the first redirect - having no more
session info - should not end up in getting redirected as well, because
it should only be send to an active node by mod_jk. So you can even stop
redirecting if you detect, that it already happened once. For a large
cluster that might be better.
Multiple might happen (didn't check the code), if all active members are
in error state. Even then one would not like to produce many of those
redirects for one request, so again, only allowing one redirect might be
the right solution.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rainer,
On 7/23/14, 1:49 PM, Rainer Jung wrote:
> Am 23.07.2014 um 17:59 schrieb Christopher Schultz:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>
>> Rainer,
>>
>> On 7/23/14, 11:23 AM, Rainer Jung wrote:
>>> On 23.07.2014 16:37, Christopher Schultz wrote:
>>>
>>>> On 7/18/14, 12:13 PM, Rainer Jung wrote:
>>>
>>>>> On 17.06.2014 16:43, Christopher Schultz wrote:
>>>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>>>>>
>>>>>> All,
>>>>>>
>>>>>> I've been using sticky sessions with mod_jk and I can
>>>>>> see that there is a bit of a problem when attempting to
>>>>>> take a backend Tomcat server out of load-balanced
>>>>>> rotation: a user who never (or rarely) restarts their web
>>>>>> browser will keep the same JSESSIONID cookie forever and
>>>>>> therefore end up with the same backend server whether it
>>>>>> has been disabled or not.
>>>>>>
>>>>>> Quick series of events:
>>>>>>
>>>>>> 1. User visits load-balancer and gets a
>>>>>> randomly-assigned backend server/route. We'll call this
>>>>>> route "X". The JSESSIONID cookie set by the backend
>>>>>> server is therefore foo.X.
>>>>>>
>>>>>> 2. User's requests are routed by mod_jk to route X.
>>>>>>
>>>>>> 3. Route X is disabled using mod_jk's status worker
>>>>>>
>>>>>> 4. User's session on server X expires.
>>>>>>
>>>>>> [Technically, 3 and 4 can happen in either order]
>>>>>>
>>>>>> 5. User makes a new request to the load-balancer, and
>>>>>> mod_jk sees the JSESSIONID cookie still set to foo.X.
>>>>>> mod_jk sends the request to route X which allows the user
>>>>>> to login, etc.
>>>>>>
>>>>>> Thus, it takes more time than necessary to bleed all the
>>>>>> traffic from route X for maintenance, etc.
>>>>>>
>>>>>> Is there a way for mod_jk to ask route X if the session
>>>>>> is *still* valid? It seems that mod_jk will not re-route
>>>>>> a request that looks like it's got a valid session id to
>>>>>> a new (active) backend server unless the backend server X
>>>>>> is actually down.
>>>>>>
>>>>>> Any ideas?
>>>>>
>>>>> Not exactly what you want, but you can build something
>>>>> around it:
>>>>>
>>>>> 1) Switch off stickyness for specific URLs
>>>>>
>>>>> If you know that users will come via specific URLs, like a
>>>>> login page, and you want that page to be handled non-sticky
>>>>> to optimize load balancing even if users have an old
>>>>> cookie, you can do that by setting the Apache envvar
>>>>> JK_STICKY_IGNORE. Look for JK_STICKY_IGNORE on:
>>>>>
>>>>> http://tomcat.apache.org/connectors-doc/reference/apache.html
>>>>
>>>>
>>>>>
This seems like a reasonable way to do things, except that we
>>>> still want to support requests to protected resources being
>>>> saved and redirected to the login page. If we did this
>>>> (JK_STICKY_IGNORE), then we'd end up "forgetting" the saved
>>>> request (because the client would be re-balanced to another
>>>> node for the login page) and ending up with a (useless)
>>>> session on the node we are trying to take down.
>>>>
>>>> We'd like to retain the request-saving capabilities of the
>>>> container.
>>>
>>> Whatever "saved" exactly means: if you can identify that
>>> situation in terms of any part of the request (e.g. something
>>> in the referer header etc.), you can add that as a positive or
>>> negative condition via RewriteCond (or the 2.4 unified
>>> expression parser) and set JK_STICKY_IGNORE in a RewriteRule
>>> that does not change the URI, only sets the variable and also
>>> only of the RewriteConds apply. You'd have to analyze the
>>> request-response stream e.g. with a browser plugin to look for
>>> a useful header or similar which can be used to distinguish the
>>> "saved" and the "normal" case.
>>
>> When I say "saved" I mean this workflow:
>>
>> 1. User requests a protected resource 2. Container saves the
>> request (which creates a new session), presents the login screen
>> 3. User authenticates 4. User is redirected to request saved in
>> step #1
>
> So where's the interaction with putting a JK_STICKY_IGNORE on the
> login page if directly retrieved?
No problem there.
> The above sequence of steps does not retrieve the login page from
> the outside via its real URL, e.g. /login, does it?
Yes, it does: the request in #2 results in a *redirect* to the login
page. Thus, the saved request would be lost if we stripped the
stickiness of the request.
> That env var is only useful for URIs for which you know that they
> either don't need a session (unprotected static content) or will
> start a new session (login page accessed by URI).
That why I think we might want to start to use a different URI for
"regular" logins versus logins prompted by the container (those that
are for "saved" requests).
The above poses the following problem: users with expired session ids
will create "saved request sessions" on a node that should not be
used, then they will be redirected to another node in the cluster,
thereby losing the saved session. I'd like to avoid both parts of that
problem and I believe it can be done with a Valve (but not a Filter).
> The other case, a request with an invalid session ID accessing a
> tomcat instance with activation disabled can IMHO be handled by a
> filter that
>
> - checks whether the request has a valid session using
> getSession(false), if it has one, let the request proceed
>
> - checks activation state, if "active", let the request proceed
>
> - checks the request method, if not GET, let the request proceed
>
> - otherwise:
>
> - set the session cookie, e.g. JSESSIONID the an empty value -
> issue an external redirect to the same request URL - optional
> redirect loop detection: add a query string param or cookie that
> gets the local jvmRoute appended during each redirect. Before
> doing the redirect, check that the local jvmRoute is not already
> part of that token (we have already been here before)
>
> This would not really interfere with your saved requests: they
> would get a redirect which the browser follwos automatically and
> after that you will observe the normal behavior.
This is exactly what I have implemented -- as a Filter since we can
insert it before SecurityFilter intercepts the requests -- and my
tests suggest that it will work correctly.
I added code to strip-out any ;jsessionid path parameter from the URL
if it exists, but haven't done any of the redirect loop detection
(yet). I think the loop detection is going to have to keep a running
list of visited nodes which, in large clusters, could grow very large
especially if the node names are long. I'll post my code when it's a
little more featureful.
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJT0Ab/AAoJEBzwKT+lPKRYG7wQAModddUmQTUFOS1XCFkFMSEM
Ccudk+haFhMlNgI8F69VC/Y/SKd+QAeyyv/f2a5mwz+2/xg6+/yoJtw39HKrQjy5
4wKNGGuO0wiHPqmndXi9R5QPWRrV59U++W4VOU2i2Hc8kxid0Wh95oDPMkePrrxz
DKDo53nVLbeL1KmN2gb53ykcxsbhUObWFW73Y8R1izewn1FgNEPs7+OjnBanf8eh
S7tPkIbASYOMdaGouDxe9XBM0PToeMkWFQPlsd29OLZ1UdRgAHmWrWyjX07Rmzff
RGt1w9GWu96y9MzC5F+XMFzs+3CNNfiQoIaEG7/1M/Imh8XI5Ymj6tYmCC6cN9xi
hnXMZZSFxar61q1B4yoQMslP18O2a8C54LReousNnIKSpLjdy0d5anue+OPmHjPD
JzoCAfwsH5e8REkS1pp+b1Eizt2djbpOylcU6g5cnmsZ99N50fB4lKD4sEqGaq3/
SYgOptl5m1RAx+c4Shoy9yELTN95+nddRZi5xjh223M2+h6pV3f1yfiORikCDyTH
ME+fZKXTIiMd7PR+F0K6QAPnhu+HeDHqJyZqcnXSOS8Z1nBO3dxnhKTtJqrggAup
cLuU/FUEH60N9ImUZFzMTId8HOr1lGYRRJrkGcuxF0qSPjJ+jpw4zWmf9in8yuYD
jxm0uRqU3wlrWrOnIUjY
=v6RU
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Rainer Jung <ra...@kippdata.de>.
Am 23.07.2014 um 17:59 schrieb Christopher Schultz:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Rainer,
>
> On 7/23/14, 11:23 AM, Rainer Jung wrote:
>> On 23.07.2014 16:37, Christopher Schultz wrote:
>>
>>> On 7/18/14, 12:13 PM, Rainer Jung wrote:
>>
>>>> On 17.06.2014 16:43, Christopher Schultz wrote:
>>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>>>>
>>>>> All,
>>>>>
>>>>> I've been using sticky sessions with mod_jk and I can see
>>>>> that there is a bit of a problem when attempting to take a
>>>>> backend Tomcat server out of load-balanced rotation: a user
>>>>> who never (or rarely) restarts their web browser will keep
>>>>> the same JSESSIONID cookie forever and therefore end up with
>>>>> the same backend server whether it has been disabled or not.
>>>>>
>>>>> Quick series of events:
>>>>>
>>>>> 1. User visits load-balancer and gets a randomly-assigned
>>>>> backend server/route. We'll call this route "X". The
>>>>> JSESSIONID cookie set by the backend server is therefore
>>>>> foo.X.
>>>>>
>>>>> 2. User's requests are routed by mod_jk to route X.
>>>>>
>>>>> 3. Route X is disabled using mod_jk's status worker
>>>>>
>>>>> 4. User's session on server X expires.
>>>>>
>>>>> [Technically, 3 and 4 can happen in either order]
>>>>>
>>>>> 5. User makes a new request to the load-balancer, and mod_jk
>>>>> sees the JSESSIONID cookie still set to foo.X. mod_jk sends
>>>>> the request to route X which allows the user to login, etc.
>>>>>
>>>>> Thus, it takes more time than necessary to bleed all the
>>>>> traffic from route X for maintenance, etc.
>>>>>
>>>>> Is there a way for mod_jk to ask route X if the session is
>>>>> *still* valid? It seems that mod_jk will not re-route a
>>>>> request that looks like it's got a valid session id to a new
>>>>> (active) backend server unless the backend server X is
>>>>> actually down.
>>>>>
>>>>> Any ideas?
>>>>
>>>> Not exactly what you want, but you can build something around
>>>> it:
>>>>
>>>> 1) Switch off stickyness for specific URLs
>>>>
>>>> If you know that users will come via specific URLs, like a
>>>> login page, and you want that page to be handled non-sticky to
>>>> optimize load balancing even if users have an old cookie, you
>>>> can do that by setting the Apache envvar JK_STICKY_IGNORE. Look
>>>> for JK_STICKY_IGNORE on:
>>>>
>>>> http://tomcat.apache.org/connectors-doc/reference/apache.html
>>>
>>> This seems like a reasonable way to do things, except that we
>>> still want to support requests to protected resources being saved
>>> and redirected to the login page. If we did this
>>> (JK_STICKY_IGNORE), then we'd end up "forgetting" the saved
>>> request (because the client would be re-balanced to another node
>>> for the login page) and ending up with a (useless) session on the
>>> node we are trying to take down.
>>>
>>> We'd like to retain the request-saving capabilities of the
>>> container.
>>
>> Whatever "saved" exactly means: if you can identify that situation
>> in terms of any part of the request (e.g. something in the referer
>> header etc.), you can add that as a positive or negative condition
>> via RewriteCond (or the 2.4 unified expression parser) and set
>> JK_STICKY_IGNORE in a RewriteRule that does not change the URI,
>> only sets the variable and also only of the RewriteConds apply.
>> You'd have to analyze the request-response stream e.g. with a
>> browser plugin to look for a useful header or similar which can be
>> used to distinguish the "saved" and the "normal" case.
>
> When I say "saved" I mean this workflow:
>
> 1. User requests a protected resource
> 2. Container saves the request (which creates a new session), presents
> the login screen
> 3. User authenticates
> 4. User is redirected to request saved in step #1
So where's the interaction with putting a JK_STICKY_IGNORE on the login
page if directly retrieved? The above sequence of steps does not
retrieve the login page from the outside via its real URL, e.g. /login,
does it? That env var is only useful for URIs for which you know that
they either don't need a session (unprotected static content) or will
start a new session (login page accessed by URI).
The other case, a request with an invalid session ID accessing a tomcat
instance with activation disabled can IMHO be handled by a filter that
- checks whether the request has a valid session using
getSession(false), if it has one, let the request proceed
- checks activation state, if "active", let the request proceed
- checks the request method, if not GET, let the request proceed
- otherwise:
- set the session cookie, e.g. JSESSIONID the an empty value
- issue an external redirect to the same request URL
- optional redirect loop detection: add a query string param or
cookie that gets the local jvmRoute appended during each redirect.
Before doing the redirect, check that the local jvmRoute is not already
part of that token (we have already been here before)
This would not really interfere with your saved requests: they would get
a redirect which the browser follwos automatically and after that you
will observe the normal behavior.
>>> One option we have here is to provide separate URLs for
>>> "regular" logins versus the saved-request logins mentioned above:
>>> that would probably solve both problems.
>>>
>>>> 2) Improve handling of sessions for node with activation
>>>> "disabled"
>>>>
>>>> If you switch a node to activation "disabled", mod_jk will not
>>>> send requests there, that have no session id (and of course
>>>> also not when the session route points to another node). But
>>>> the old cookie requests might still go there.
>>>
>>> Yes, this is what we would normally do.
>>>
>>>> For that you can use the feature, that mod_jk forwards the
>>>> activation state to the Tomcat node. The node can then decide
>>>> on itself, whether it will handle a request coming in with an
>>>> invalid session id, or for example clears the session cookie
>>>> and does a self-referential redirect, which then by mod_jk is
>>>> balanced on the fully enabled nodes.
>>>
>>> This sounds like a promising technique. I was thinking we'd just
>>> tell the Tomcat node directly (e.g. set a context-scoped flag)
>>> that it was "disabled", but having AJP forward that information
>>> would be even better, so the state can be managed entirely by the
>>> status worker on the httpd side.
>>>
>>>> This logic can be put into a servlet filter.
>>>
>>> I'm not sure it can be in a Filter because of the interaction
>>> with the saved-request features described above. If in a Filter,
>>> the request would be saved before the Filter has a chance to see
>>> it, then authentication would take place, etc.
>>>
>>> I think this has to be in a Valve, and it has to happen before
>>> the AuthenticatorValve sees the request. Do you see a way around
>>> using a Valve? Assuming that such a Valve would be required, I
>>> think we should provide one with Tomcat. I'd be happy to write
>>> such a Valve.
>>
>> Can't comment because I don't know what exactly the "saved
>> request" feature means, resp. how it works.
>
> Well, the lb of course doesn't know if the session is valid. If it's
> valid, the request *should* go to the requested node (sticky). If the
> session is invalid, it should be re-balanced. Only the node itself can
> determine the validity of the session.
>
> Since the container's authenticator Valve intercepts requests to
> protected resources in the above step #1 and takes-over, no Filter can
> be used to intercede and redirect (to cause a re-balance to occur)
> before step 2 occurs which defeats the whole process.
>
> I'm working on a Filter that will work in our environment (we use
> SecurityFilter, which, being a Filter, can be subverted by an
> earlier-executing Filter). Once I have that proof-of concept, I'll
> propose it as a new Valve available but not enabled by default.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rainer,
On 7/23/14, 11:23 AM, Rainer Jung wrote:
> On 23.07.2014 16:37, Christopher Schultz wrote:
>
>> On 7/18/14, 12:13 PM, Rainer Jung wrote:
>
>>> On 17.06.2014 16:43, Christopher Schultz wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>>>
>>>> All,
>>>>
>>>> I've been using sticky sessions with mod_jk and I can see
>>>> that there is a bit of a problem when attempting to take a
>>>> backend Tomcat server out of load-balanced rotation: a user
>>>> who never (or rarely) restarts their web browser will keep
>>>> the same JSESSIONID cookie forever and therefore end up with
>>>> the same backend server whether it has been disabled or not.
>>>>
>>>> Quick series of events:
>>>>
>>>> 1. User visits load-balancer and gets a randomly-assigned
>>>> backend server/route. We'll call this route "X". The
>>>> JSESSIONID cookie set by the backend server is therefore
>>>> foo.X.
>>>>
>>>> 2. User's requests are routed by mod_jk to route X.
>>>>
>>>> 3. Route X is disabled using mod_jk's status worker
>>>>
>>>> 4. User's session on server X expires.
>>>>
>>>> [Technically, 3 and 4 can happen in either order]
>>>>
>>>> 5. User makes a new request to the load-balancer, and mod_jk
>>>> sees the JSESSIONID cookie still set to foo.X. mod_jk sends
>>>> the request to route X which allows the user to login, etc.
>>>>
>>>> Thus, it takes more time than necessary to bleed all the
>>>> traffic from route X for maintenance, etc.
>>>>
>>>> Is there a way for mod_jk to ask route X if the session is
>>>> *still* valid? It seems that mod_jk will not re-route a
>>>> request that looks like it's got a valid session id to a new
>>>> (active) backend server unless the backend server X is
>>>> actually down.
>>>>
>>>> Any ideas?
>>>
>>> Not exactly what you want, but you can build something around
>>> it:
>>>
>>> 1) Switch off stickyness for specific URLs
>>>
>>> If you know that users will come via specific URLs, like a
>>> login page, and you want that page to be handled non-sticky to
>>> optimize load balancing even if users have an old cookie, you
>>> can do that by setting the Apache envvar JK_STICKY_IGNORE. Look
>>> for JK_STICKY_IGNORE on:
>>>
>>> http://tomcat.apache.org/connectors-doc/reference/apache.html
>>
>> This seems like a reasonable way to do things, except that we
>> still want to support requests to protected resources being saved
>> and redirected to the login page. If we did this
>> (JK_STICKY_IGNORE), then we'd end up "forgetting" the saved
>> request (because the client would be re-balanced to another node
>> for the login page) and ending up with a (useless) session on the
>> node we are trying to take down.
>>
>> We'd like to retain the request-saving capabilities of the
>> container.
>
> Whatever "saved" exactly means: if you can identify that situation
> in terms of any part of the request (e.g. something in the referer
> header etc.), you can add that as a positive or negative condition
> via RewriteCond (or the 2.4 unified expression parser) and set
> JK_STICKY_IGNORE in a RewriteRule that does not change the URI,
> only sets the variable and also only of the RewriteConds apply.
> You'd have to analyze the request-response stream e.g. with a
> browser plugin to look for a useful header or similar which can be
> used to distinguish the "saved" and the "normal" case.
When I say "saved" I mean this workflow:
1. User requests a protected resource
2. Container saves the request (which creates a new session), presents
the login screen
3. User authenticates
4. User is redirected to request saved in step #1
>> One option we have here is to provide separate URLs for
>> "regular" logins versus the saved-request logins mentioned above:
>> that would probably solve both problems.
>>
>>> 2) Improve handling of sessions for node with activation
>>> "disabled"
>>>
>>> If you switch a node to activation "disabled", mod_jk will not
>>> send requests there, that have no session id (and of course
>>> also not when the session route points to another node). But
>>> the old cookie requests might still go there.
>>
>> Yes, this is what we would normally do.
>>
>>> For that you can use the feature, that mod_jk forwards the
>>> activation state to the Tomcat node. The node can then decide
>>> on itself, whether it will handle a request coming in with an
>>> invalid session id, or for example clears the session cookie
>>> and does a self-referential redirect, which then by mod_jk is
>>> balanced on the fully enabled nodes.
>>
>> This sounds like a promising technique. I was thinking we'd just
>> tell the Tomcat node directly (e.g. set a context-scoped flag)
>> that it was "disabled", but having AJP forward that information
>> would be even better, so the state can be managed entirely by the
>> status worker on the httpd side.
>>
>>> This logic can be put into a servlet filter.
>>
>> I'm not sure it can be in a Filter because of the interaction
>> with the saved-request features described above. If in a Filter,
>> the request would be saved before the Filter has a chance to see
>> it, then authentication would take place, etc.
>>
>> I think this has to be in a Valve, and it has to happen before
>> the AuthenticatorValve sees the request. Do you see a way around
>> using a Valve? Assuming that such a Valve would be required, I
>> think we should provide one with Tomcat. I'd be happy to write
>> such a Valve.
>
> Can't comment because I don't know what exactly the "saved
> request" feature means, resp. how it works.
Well, the lb of course doesn't know if the session is valid. If it's
valid, the request *should* go to the requested node (sticky). If the
session is invalid, it should be re-balanced. Only the node itself can
determine the validity of the session.
Since the container's authenticator Valve intercepts requests to
protected resources in the above step #1 and takes-over, no Filter can
be used to intercede and redirect (to cause a re-balance to occur)
before step 2 occurs which defeats the whole process.
I'm working on a Filter that will work in our environment (we use
SecurityFilter, which, being a Filter, can be subverted by an
earlier-executing Filter). Once I have that proof-of concept, I'll
propose it as a new Valve available but not enabled by default.
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTz9v+AAoJEBzwKT+lPKRYNC0P/AjnJjhOiQIS9KXidoxLPUL6
3AFux9EAMPQVDARGrkdRVgvN7+Fzs/KJ0/1qWL99cZkXAR1DdA0OnbhEOAEp/s2k
EjNTivGZuPg3xLBro6hcanl69vjNmTEJ7h5XC8mo8qPulI9lwBc916KdKlyz560p
smrI6aJaXTuGV9+fHiJ8mkwcplpzOpfAD111sq3coadUFQYy9Z1YnX01rokQuvgN
PO4NmCn7PI1ffXtr0vGloMuPGg2Hm0mJDcJs/ZJCcuYaRlh6aOSVi353Fvijd36W
fqIUjE+WZqg2qr8iJkdIwy2BI13mUHgM7aF0F/0onD6xPoIxji9EPdrqwLPlMN+K
Bp1gtZT0axF4YvaDn0Cts75GrP9/xUi+mjPT+S9L5aV3zbm6exuHejF5+wsv8zJj
1vNcS6lxn6U9fwvNRv+22XYG6hz2cqcfSa0uP+VI+5BEWHkn1CQv/0E6jjFq3Uzk
NzJBtaqb0UC8iUkLwW4wf1gm/zrwG7TbApIGLRxlPve40IV/wNCAJxdX0S+5fxf9
S+Eg4LKiCfWoPIvkaOSbYgpVFLlfEJ7nJJ2x/TwyMQGedrbaXu9TI4wZB1zJWBGU
/H8mXTojzTaS2ceZNbh1bRUuZNkMlOe4GfWFNQhbKDO8X6h7BlUNmmIjBxjAnCxX
X/OCKmhFx0Za/RZNAuAj
=SZWZ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Rainer Jung <ra...@kippdata.de>.
On 23.07.2014 16:37, Christopher Schultz wrote:
> On 7/18/14, 12:13 PM, Rainer Jung wrote:
>> On 17.06.2014 16:43, Christopher Schultz wrote:
>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>>
>>> All,
>>>
>>> I've been using sticky sessions with mod_jk and I can see that
>>> there is a bit of a problem when attempting to take a backend
>>> Tomcat server out of load-balanced rotation: a user who never (or
>>> rarely) restarts their web browser will keep the same JSESSIONID
>>> cookie forever and therefore end up with the same backend server
>>> whether it has been disabled or not.
>>>
>>> Quick series of events:
>>>
>>> 1. User visits load-balancer and gets a randomly-assigned
>>> backend server/route. We'll call this route "X". The JSESSIONID
>>> cookie set by the backend server is therefore foo.X.
>>>
>>> 2. User's requests are routed by mod_jk to route X.
>>>
>>> 3. Route X is disabled using mod_jk's status worker
>>>
>>> 4. User's session on server X expires.
>>>
>>> [Technically, 3 and 4 can happen in either order]
>>>
>>> 5. User makes a new request to the load-balancer, and mod_jk sees
>>> the JSESSIONID cookie still set to foo.X. mod_jk sends the
>>> request to route X which allows the user to login, etc.
>>>
>>> Thus, it takes more time than necessary to bleed all the traffic
>>> from route X for maintenance, etc.
>>>
>>> Is there a way for mod_jk to ask route X if the session is
>>> *still* valid? It seems that mod_jk will not re-route a request
>>> that looks like it's got a valid session id to a new (active)
>>> backend server unless the backend server X is actually down.
>>>
>>> Any ideas?
>>
>> Not exactly what you want, but you can build something around it:
>>
>> 1) Switch off stickyness for specific URLs
>>
>> If you know that users will come via specific URLs, like a login
>> page, and you want that page to be handled non-sticky to optimize
>> load balancing even if users have an old cookie, you can do that by
>> setting the Apache envvar JK_STICKY_IGNORE. Look for
>> JK_STICKY_IGNORE on:
>>
>> http://tomcat.apache.org/connectors-doc/reference/apache.html
>
> This seems like a reasonable way to do things, except that we still
> want to support requests to protected resources being saved and
> redirected to the login page. If we did this (JK_STICKY_IGNORE), then
> we'd end up "forgetting" the saved request (because the client would
> be re-balanced to another node for the login page) and ending up with
> a (useless) session on the node we are trying to take down.
>
> We'd like to retain the request-saving capabilities of the container.
Whatever "saved" exactly means: if you can identify that situation in
terms of any part of the request (e.g. something in the referer header
etc.), you can add that as a positive or negative condition via
RewriteCond (or the 2.4 unified expression parser) and set
JK_STICKY_IGNORE in a RewriteRule that does not change the URI, only
sets the variable and also only of the RewriteConds apply. You'd have to
analyze the request-response stream e.g. with a browser plugin to look
for a useful header or similar which can be used to distinguish the
"saved" and the "normal" case.
> One option we have here is to provide separate URLs for "regular"
> logins versus the saved-request logins mentioned above: that would
> probably solve both problems.
>
>> 2) Improve handling of sessions for node with activation
>> "disabled"
>>
>> If you switch a node to activation "disabled", mod_jk will not
>> send requests there, that have no session id (and of course also
>> not when the session route points to another node). But the old
>> cookie requests might still go there.
>
> Yes, this is what we would normally do.
>
>> For that you can use the feature, that mod_jk forwards the
>> activation state to the Tomcat node. The node can then decide on
>> itself, whether it will handle a request coming in with an invalid
>> session id, or for example clears the session cookie and does a
>> self-referential redirect, which then by mod_jk is balanced on the
>> fully enabled nodes.
>
> This sounds like a promising technique. I was thinking we'd just tell
> the Tomcat node directly (e.g. set a context-scoped flag) that it was
> "disabled", but having AJP forward that information would be even
> better, so the state can be managed entirely by the status worker on
> the httpd side.
>
>> This logic can be put into a servlet filter.
>
> I'm not sure it can be in a Filter because of the interaction with the
> saved-request features described above. If in a Filter, the request
> would be saved before the Filter has a chance to see it, then
> authentication would take place, etc.
>
> I think this has to be in a Valve, and it has to happen before the
> AuthenticatorValve sees the request. Do you see a way around using a
> Valve? Assuming that such a Valve would be required, I think we should
> provide one with Tomcat. I'd be happy to write such a Valve.
Can't comment because I don't know what exactly the "saved request"
feature means, resp. how it works.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rainer,
On 7/23/14, 10:40 AM, Christopher Schultz wrote:
> Rainer,
>
> On 7/23/14, 10:37 AM, Christopher Schultz wrote:
>>> On 17.06.2014 16:43, Christopher Schultz wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>>>
>>>> All,
>>>>
>>>> I've been using sticky sessions with mod_jk and I can see
>>>> that there is a bit of a problem when attempting to take a
>>>> backend Tomcat server out of load-balanced rotation: a user
>>>> who never (or rarely) restarts their web browser will keep
>>>> the same JSESSIONID cookie forever and therefore end up with
>>>> the same backend server whether it has been disabled or not.
>>>>
>>>> Quick series of events:
>>>>
>>>> 1. User visits load-balancer and gets a randomly-assigned
>>>> backend server/route. We'll call this route "X". The
>>>> JSESSIONID cookie set by the backend server is therefore
>>>> foo.X.
>>>>
>>>> 2. User's requests are routed by mod_jk to route X.
>>>>
>>>> 3. Route X is disabled using mod_jk's status worker
>>>>
>>>> 4. User's session on server X expires.
>>>>
>>>> [Technically, 3 and 4 can happen in either order]
>>>>
>>>> 5. User makes a new request to the load-balancer, and mod_jk
>>>> sees the JSESSIONID cookie still set to foo.X. mod_jk sends
>>>> the request to route X which allows the user to login, etc.
>>>>
>>>> Thus, it takes more time than necessary to bleed all the
>>>> traffic from route X for maintenance, etc.
>>>>
>>>> Is there a way for mod_jk to ask route X if the session is
>>>> *still* valid? It seems that mod_jk will not re-route a
>>>> request that looks like it's got a valid session id to a new
>>>> (active) backend server unless the backend server X is
>>>> actually down.
>>>>
>>>> Any ideas?
>
>>> Not exactly what you want, but you can build something around
>>> it:
>
>>> 1) Switch off stickyness for specific URLs
>
>>> If you know that users will come via specific URLs, like a
>>> login page, and you want that page to be handled non-sticky to
>>> optimize load balancing even if users have an old cookie, you
>>> can do that by setting the Apache envvar JK_STICKY_IGNORE. Look
>>> for JK_STICKY_IGNORE on:
>
>>> http://tomcat.apache.org/connectors-doc/reference/apache.html
>
>> This seems like a reasonable way to do things, except that we
>> still want to support requests to protected resources being
>> saved and redirected to the login page. If we did this
>> (JK_STICKY_IGNORE), then we'd end up "forgetting" the saved
>> request (because the client would be re-balanced to another node
>> for the login page) and ending up with a (useless) session on the
>> node we are trying to take down.
>
>> We'd like to retain the request-saving capabilities of the
>> container.
>
>> One option we have here is to provide separate URLs for "regular"
>> logins versus the saved-request logins mentioned above: that
>> would probably solve both problems.
>
>>> 2) Improve handling of sessions for node with activation
>>> "disabled"
>
>>> If you switch a node to activation "disabled", mod_jk will not
>>> send requests there, that have no session id (and of course
>>> also not when the session route points to another node). But
>>> the old cookie requests might still go there.
>
>> Yes, this is what we would normally do.
>
>>> For that you can use the feature, that mod_jk forwards the
>>> activation state to the Tomcat node. The node can then decide
>>> on itself, whether it will handle a request coming in with an
>>> invalid session id, or for example clears the session cookie
>>> and does a self-referential redirect, which then by mod_jk is
>>> balanced on the fully enabled nodes.
>
>> This sounds like a promising technique. I was thinking we'd just
>> tell the Tomcat node directly (e.g. set a context-scoped flag)
>> that it was "disabled", but having AJP forward that information
>> would be even better, so the state can be managed entirely by the
>> status worker on the httpd side.
>
>>> This logic can be put into a servlet filter.
>
>> I'm not sure it can be in a Filter because of the interaction
>> with the saved-request features described above. If in a Filter,
>> the request would be saved before the Filter has a chance to see
>> it, then authentication would take place, etc.
>
>> I think this has to be in a Valve, and it has to happen before
>> the AuthenticatorValve sees the request. Do you see a way around
>> using a Valve? Assuming that such a Valve would be required, I
>> think we should provide one with Tomcat. I'd be happy to write
>> such a Valve.
>
>>> You have to be careful though to not produce redirecting
>>> cycles, e.g. in case of errors or all other nodes are down. You
>>> could add the name of the first node the the URL as a query
>>> param, and if you see it a second time, then do not redirect
>>> again.
>
>> I think that's a good way of doing it. One could also use
>> cookies if they can be relied upon, if you don't want to modify
>> the URL.
>
>>> The request attribute is named "JK_LB_ACTIVATION". Search for
>>> that name on
>
>>> http://tomcat.apache.org/connectors-doc/generic_howto/loadbalancers.html
>
>>>
> I'm
>>>
> not seeing this in my request attributes. Do I have to specifically
> enable it using JkEnvVar?
Duh:
"
You can retrieve the variables on Tomcat as request attributes via
request.getAttribute(attributeName). Note that the variables send via
JkEnvVar will not be listed in request.getAttributeNames().
"
The request attribute is in fact there... it's just not advertised
through request.getAttributeNames, just like any other JK_ environment
variable passed-over from the web server.
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTz8tvAAoJEBzwKT+lPKRYYdEP/0yqDT6Ek4mNYfIYbbP8BxXh
1Sq5ubDkzEFy7x1uBEfh4sMPfdgZZE7ACzshRZgij51eJh61X4NIcAHgbiAFzsJP
QQOPnqOBV5GG46T7oeWwCudS1MjrTIBJuWQOJSieFnGJZ3GmZ6S6tdb67B3Kr6of
fw9KePDc10jjxl+bHJxGAI3iXeS4kFXoZV8oLdP99XsQTKp3tOHaZF50k6wQjRks
t9tTiQ9ADM4AAqXcw7lgYl0t1lF5yt2KxWPvAvk6B9fP0k6oUTBmvlVVjt6rxm7G
THXpAXDaZWOkyMN6JbKxj7NzYgcJq9OyNOm1BXrAdKJ0RBRKgGhdYCdH5qVsvaZZ
+JfyNDB1xciJ9XekTltgyFozpwoI5awgedeK985Qga+TlIvSzge9td8YDZ8KnG+U
zyjzP74tT9MjpZCR6xW9lktIhowZy5jJU2O/1audE/Y8CjZLCxs+hYO4ATfXCspW
0v0N6g4e72CC85g660mQnhUhqqSoT8cl7rT1zmRCUXbZVAzHYngwg6+E8uUyvln9
qIQOpdLBR8V6qWzhh5WCd563nWCJr1x54xIJ8y3xOYpXxxBtV+JIpKFCQEuq66sj
zR9MiTpn4cpJ3NJLgmakm1Ad9zeGEu9XGxAAGPWFNPBjh307tCeyUMWOlTFw3C9w
kn7h73+wkOdBAls8R3gw
=DsVT
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rainer,
On 7/23/14, 10:37 AM, Christopher Schultz wrote:
>> On 17.06.2014 16:43, Christopher Schultz wrote:
>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>>
>>> All,
>>>
>>> I've been using sticky sessions with mod_jk and I can see that
>>> there is a bit of a problem when attempting to take a backend
>>> Tomcat server out of load-balanced rotation: a user who never
>>> (or rarely) restarts their web browser will keep the same
>>> JSESSIONID cookie forever and therefore end up with the same
>>> backend server whether it has been disabled or not.
>>>
>>> Quick series of events:
>>>
>>> 1. User visits load-balancer and gets a randomly-assigned
>>> backend server/route. We'll call this route "X". The
>>> JSESSIONID cookie set by the backend server is therefore
>>> foo.X.
>>>
>>> 2. User's requests are routed by mod_jk to route X.
>>>
>>> 3. Route X is disabled using mod_jk's status worker
>>>
>>> 4. User's session on server X expires.
>>>
>>> [Technically, 3 and 4 can happen in either order]
>>>
>>> 5. User makes a new request to the load-balancer, and mod_jk
>>> sees the JSESSIONID cookie still set to foo.X. mod_jk sends
>>> the request to route X which allows the user to login, etc.
>>>
>>> Thus, it takes more time than necessary to bleed all the
>>> traffic from route X for maintenance, etc.
>>>
>>> Is there a way for mod_jk to ask route X if the session is
>>> *still* valid? It seems that mod_jk will not re-route a
>>> request that looks like it's got a valid session id to a new
>>> (active) backend server unless the backend server X is actually
>>> down.
>>>
>>> Any ideas?
>
>> Not exactly what you want, but you can build something around
>> it:
>
>> 1) Switch off stickyness for specific URLs
>
>> If you know that users will come via specific URLs, like a login
>> page, and you want that page to be handled non-sticky to
>> optimize load balancing even if users have an old cookie, you can
>> do that by setting the Apache envvar JK_STICKY_IGNORE. Look for
>> JK_STICKY_IGNORE on:
>
>> http://tomcat.apache.org/connectors-doc/reference/apache.html
>
> This seems like a reasonable way to do things, except that we
> still want to support requests to protected resources being saved
> and redirected to the login page. If we did this
> (JK_STICKY_IGNORE), then we'd end up "forgetting" the saved request
> (because the client would be re-balanced to another node for the
> login page) and ending up with a (useless) session on the node we
> are trying to take down.
>
> We'd like to retain the request-saving capabilities of the
> container.
>
> One option we have here is to provide separate URLs for "regular"
> logins versus the saved-request logins mentioned above: that would
> probably solve both problems.
>
>> 2) Improve handling of sessions for node with activation
>> "disabled"
>
>> If you switch a node to activation "disabled", mod_jk will not
>> send requests there, that have no session id (and of course also
>> not when the session route points to another node). But the old
>> cookie requests might still go there.
>
> Yes, this is what we would normally do.
>
>> For that you can use the feature, that mod_jk forwards the
>> activation state to the Tomcat node. The node can then decide on
>> itself, whether it will handle a request coming in with an
>> invalid session id, or for example clears the session cookie and
>> does a self-referential redirect, which then by mod_jk is
>> balanced on the fully enabled nodes.
>
> This sounds like a promising technique. I was thinking we'd just
> tell the Tomcat node directly (e.g. set a context-scoped flag) that
> it was "disabled", but having AJP forward that information would be
> even better, so the state can be managed entirely by the status
> worker on the httpd side.
>
>> This logic can be put into a servlet filter.
>
> I'm not sure it can be in a Filter because of the interaction with
> the saved-request features described above. If in a Filter, the
> request would be saved before the Filter has a chance to see it,
> then authentication would take place, etc.
>
> I think this has to be in a Valve, and it has to happen before the
> AuthenticatorValve sees the request. Do you see a way around using
> a Valve? Assuming that such a Valve would be required, I think we
> should provide one with Tomcat. I'd be happy to write such a
> Valve.
>
>> You have to be careful though to not produce redirecting cycles,
>> e.g. in case of errors or all other nodes are down. You could
>> add the name of the first node the the URL as a query param, and
>> if you see it a second time, then do not redirect again.
>
> I think that's a good way of doing it. One could also use cookies
> if they can be relied upon, if you don't want to modify the URL.
>
>> The request attribute is named "JK_LB_ACTIVATION". Search for
>> that name on
>
>> http://tomcat.apache.org/connectors-doc/generic_howto/loadbalancers.html
I'm
>>
not seeing this in my request attributes. Do I have to
specifically enable it using JkEnvVar?
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTz8lZAAoJEBzwKT+lPKRY7YcQAL23/2/XSQhekfTaB+5IL7IL
UQSz7QInnxbRjrHo/oOElIFNn/AoThpXYFtYLAUauZ2sYyT56gDPyVCRwZXn3XqB
gEVZsnQLivW+wEVb79htUS4yUn1j3O8lijmtg/BXRWL+2/KDlF6kbLSBOOGRbwtS
PRTmXnqLr+YQVm5e/UUj1Ctz4t5lFnVz7ahrj6dfOUntm1wUvF2U7QWsEbhcXmNA
gyvfYVmXJTahsEgAWmHEiyqrMY8P2gzexain9Dk9RwZpEQPC1apsMqvalXJl7JIr
wCwFB4L6NXTgkhVc4ROM4i4r+oKvPNPWstRQGbyZ6hzWMGlGqqE3nqd7exHws9fp
lVEUWMfeCGQjcnr+SKDpYvzMgXEbk5N6rigjUX6c94pVtRq79f5ietcH91tM6FpX
tLvcUiFLA61/ZbWmNXTiqQTNL3ECsVrqN8bckLwMjaUD6Z5FtiWWVqQPIWn04jza
ISehwLE2MePdTWR+IrTu6FCDf/Jlaib1DW/SLlxGGGDGkWiZTLXu6BxL3mkqR3r9
BLarkvtfqrMTV3ixqDyahs6uCbQoyeRbdt7iqC3ORUdMleMhuQ0J3Q6wq9ZEepc5
TfWmwUdTbAyCoEysxx2/lCSrMBtPzXtxsbxeJNnGSW6H6vKTB74gtxpNyyH7cgZf
xr2eDDrw5cSMMvJYeAOL
=Kt74
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rainer,
On 7/18/14, 12:13 PM, Rainer Jung wrote:
> late answer but at least an answer. See below.
I appreciate the late answer, because it was very helpful. I have some
concerns, though. Please see below.
> On 17.06.2014 16:43, Christopher Schultz wrote:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>
>> All,
>>
>> I've been using sticky sessions with mod_jk and I can see that
>> there is a bit of a problem when attempting to take a backend
>> Tomcat server out of load-balanced rotation: a user who never (or
>> rarely) restarts their web browser will keep the same JSESSIONID
>> cookie forever and therefore end up with the same backend server
>> whether it has been disabled or not.
>>
>> Quick series of events:
>>
>> 1. User visits load-balancer and gets a randomly-assigned
>> backend server/route. We'll call this route "X". The JSESSIONID
>> cookie set by the backend server is therefore foo.X.
>>
>> 2. User's requests are routed by mod_jk to route X.
>>
>> 3. Route X is disabled using mod_jk's status worker
>>
>> 4. User's session on server X expires.
>>
>> [Technically, 3 and 4 can happen in either order]
>>
>> 5. User makes a new request to the load-balancer, and mod_jk sees
>> the JSESSIONID cookie still set to foo.X. mod_jk sends the
>> request to route X which allows the user to login, etc.
>>
>> Thus, it takes more time than necessary to bleed all the traffic
>> from route X for maintenance, etc.
>>
>> Is there a way for mod_jk to ask route X if the session is
>> *still* valid? It seems that mod_jk will not re-route a request
>> that looks like it's got a valid session id to a new (active)
>> backend server unless the backend server X is actually down.
>>
>> Any ideas?
>
> Not exactly what you want, but you can build something around it:
>
> 1) Switch off stickyness for specific URLs
>
> If you know that users will come via specific URLs, like a login
> page, and you want that page to be handled non-sticky to optimize
> load balancing even if users have an old cookie, you can do that by
> setting the Apache envvar JK_STICKY_IGNORE. Look for
> JK_STICKY_IGNORE on:
>
> http://tomcat.apache.org/connectors-doc/reference/apache.html
This seems like a reasonable way to do things, except that we still
want to support requests to protected resources being saved and
redirected to the login page. If we did this (JK_STICKY_IGNORE), then
we'd end up "forgetting" the saved request (because the client would
be re-balanced to another node for the login page) and ending up with
a (useless) session on the node we are trying to take down.
We'd like to retain the request-saving capabilities of the container.
One option we have here is to provide separate URLs for "regular"
logins versus the saved-request logins mentioned above: that would
probably solve both problems.
> 2) Improve handling of sessions for node with activation
> "disabled"
>
> If you switch a node to activation "disabled", mod_jk will not
> send requests there, that have no session id (and of course also
> not when the session route points to another node). But the old
> cookie requests might still go there.
Yes, this is what we would normally do.
> For that you can use the feature, that mod_jk forwards the
> activation state to the Tomcat node. The node can then decide on
> itself, whether it will handle a request coming in with an invalid
> session id, or for example clears the session cookie and does a
> self-referential redirect, which then by mod_jk is balanced on the
> fully enabled nodes.
This sounds like a promising technique. I was thinking we'd just tell
the Tomcat node directly (e.g. set a context-scoped flag) that it was
"disabled", but having AJP forward that information would be even
better, so the state can be managed entirely by the status worker on
the httpd side.
> This logic can be put into a servlet filter.
I'm not sure it can be in a Filter because of the interaction with the
saved-request features described above. If in a Filter, the request
would be saved before the Filter has a chance to see it, then
authentication would take place, etc.
I think this has to be in a Valve, and it has to happen before the
AuthenticatorValve sees the request. Do you see a way around using a
Valve? Assuming that such a Valve would be required, I think we should
provide one with Tomcat. I'd be happy to write such a Valve.
> You have to be careful though to not produce redirecting cycles,
> e.g. in case of errors or all other nodes are down. You could add
> the name of the first node the the URL as a query param, and if you
> see it a second time, then do not redirect again.
I think that's a good way of doing it. One could also use cookies if
they can be relied upon, if you don't want to modify the URL.
> The request attribute is named "JK_LB_ACTIVATION". Search for that
> name on
>
> http://tomcat.apache.org/connectors-doc/generic_howto/loadbalancers.html
I'll
>
definitely look at using this.
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTz8isAAoJEBzwKT+lPKRYVDcP/RexqazYXjbzrIu2StWr8ChL
eCQzG4ZOPiqz2Q/1bXChH+QLgEw5hdc9OczD/KhHyCUmayKVeSsZVKuS1tzgruPU
7YcTx6ioftlR/fWFGN5K8RprqUX+nkGI/rKn7VpyJe2OdBy149qWbk1DnnDdjFrT
UteUZ1penC63n3DlSaTPYdKAkNPkdFhOxasC1tB2hDNMgzcpdi06ci6nIqLMUltd
0dA2pP8MqGnTzdBZUy9s0Y37kU49nSgZ4Qp7dElBYh0bnoz3f1EeyDUIeozNzj7U
vXm//7ZQ/ie89pDcGrCEMQFxPBeblPaoAUgYBGsDzNI6kfsB8bp7hR+uskBQnznI
IC5tUyvkNKzoOIElSVsmAxDhB+Rtm1pVa1Jd0XaxkBVE+kBHbXIAEfiUlbWIRIUK
JgXcXIQaXFBI0kQ9OWeYxDg6ds7RQYmyyzr7haxDmcveqvvZWepnrDtxH3nMve8W
HYkIllz5UTShV2S/k5JQmeRw2ZJ8mDXLsH+wPb/qUgxqZF29p7vW08ILAOwXr9Rn
GjSeCAT0ruOeyFx4se05h67QcdJt4aBghcaOcHDXJZ50bOyVoD3lDsoclIExwNSv
cjfaEev2ERYKDMCd4o2MEYSBzD0LB+ehA28AjcQsZGCRCwcYqYqai8s3Nu5TbhvX
ae13hnoiLH/GTyAYDsym
=pUfD
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Rainer Jung <ra...@kippdata.de>.
Hi Chris,
late answer but at least an answer. See below.
On 17.06.2014 16:43, Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> All,
>
> I've been using sticky sessions with mod_jk and I can see that there
> is a bit of a problem when attempting to take a backend Tomcat server
> out of load-balanced rotation: a user who never (or rarely) restarts
> their web browser will keep the same JSESSIONID cookie forever and
> therefore end up with the same backend server whether it has been
> disabled or not.
>
> Quick series of events:
>
> 1. User visits load-balancer and gets a randomly-assigned backend
> server/route. We'll call this route "X". The JSESSIONID cookie set by
> the backend server is therefore foo.X.
>
> 2. User's requests are routed by mod_jk to route X.
>
> 3. Route X is disabled using mod_jk's status worker
>
> 4. User's session on server X expires.
>
> [Technically, 3 and 4 can happen in either order]
>
> 5. User makes a new request to the load-balancer, and mod_jk sees the
> JSESSIONID cookie still set to foo.X. mod_jk sends the request to
> route X which allows the user to login, etc.
>
> Thus, it takes more time than necessary to bleed all the traffic from
> route X for maintenance, etc.
>
> Is there a way for mod_jk to ask route X if the session is *still*
> valid? It seems that mod_jk will not re-route a request that looks
> like it's got a valid session id to a new (active) backend server
> unless the backend server X is actually down.
>
> Any ideas?
Not exactly what you want, but you can build something around it:
1) Switch off stickyness for specific URLs
If you know that users will come via specific URLs, like a login page,
and you want that page to be handled non-sticky to ptimize load
balancing even if users have an old cookie, you can do that by setting
the Apache envvar JK_STICKY_IGNORE. Look for JK_STICKY_IGNORE on:
http://tomcat.apache.org/connectors-doc/reference/apache.html
2) Improve handling of sessions for node with activation "disabled"
If you switch a node to activation "disabled", mod_jk will not send
requests there, that have no session id (and of course also not when the
session route points to another node). But the old cookie requests might
still go there. For that you can use the feature, that mod_jk forwards
the activation state to the Tomcat node. The node can then decide on
itself, whether it will handle a request coming in with an invalid
session id, or for example clears the session cookie and does a
self-referential redirect, which then by mod_jk is balanced on the fully
enabled nodes. This logic can be put into a servlet filter. You have to
be careful though to not produce redirecting cycles, e.g. in case of
errors or all other nodes are down. You could add the name of the first
node the the URL as a query param, and if you see it a second time, then
do not redirect again.
The request attribute is named "JK_LB_ACTIVATION". Search for that name on
http://tomcat.apache.org/connectors-doc/generic_howto/loadbalancers.html
HTH
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Rainer Jung <ra...@kippdata.de>.
On 08.07.2014 14:29, Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> All,
>
> On 6/17/14, 10:43 AM, Christopher Schultz wrote:
>> All,
>>
>> I've been using sticky sessions with mod_jk and I can see that
>> there is a bit of a problem when attempting to take a backend
>> Tomcat server out of load-balanced rotation: a user who never (or
>> rarely) restarts their web browser will keep the same JSESSIONID
>> cookie forever and therefore end up with the same backend server
>> whether it has been disabled or not.
>>
>> Quick series of events:
>>
>> 1. User visits load-balancer and gets a randomly-assigned backend
>> server/route. We'll call this route "X". The JSESSIONID cookie set
>> by the backend server is therefore foo.X.
>>
>> 2. User's requests are routed by mod_jk to route X.
>>
>> 3. Route X is disabled using mod_jk's status worker
>>
>> 4. User's session on server X expires.
>>
>> [Technically, 3 and 4 can happen in either order]
>>
>> 5. User makes a new request to the load-balancer, and mod_jk sees
>> the JSESSIONID cookie still set to foo.X. mod_jk sends the request
>> to route X which allows the user to login, etc.
>>
>> Thus, it takes more time than necessary to bleed all the traffic
>> from route X for maintenance, etc.
>>
>> Is there a way for mod_jk to ask route X if the session is *still*
>> valid? It seems that mod_jk will not re-route a request that looks
>> like it's got a valid session id to a new (active) backend server
>> unless the backend server X is actually down.
>>
>> Any ideas?
>
> Any takers?
>
> I was thinking that the following strategy might work:
>
> 1. Extend mod_jk's load-balancer worker to include a new directive:
> rebalance_statuses, then set the value for that directive to some very
> rare HTTP status code(s) like 502.
>
> 2. Add a Valve/Filter/Whatever to Tomcat that, when activated because
> you want to take a node out of service) returns a 502 when a session
> identifier is used that does not map to a currently-valid session and
> there is no "force-new-session" header included in the request with
> some unpredictable value to prevent end-users from sending their own
> valid headers. This could also be handled by the same component
> (Manager) that current creates sessions with a new configuration option.
>
> These two together would then bleed-off the aforementioned users: in
> step 5, the request would be routed to Tomcat X, then rejected with a
> 502. mod_jk would detect this and re-balance the worker. If mod_jk
> can't re-balance the worker to another backend (e.g. they are all
> disabled, stopped, or in a failure state), then mod_jk would send the
> request (back) to Tomcat X with the force-new-session header set.
>
> Other than the above (or something similar), or actively probing a
> backend Tomcat to see if a session is valid, does anyone have any
> ideas for how to get a backend Tomcat into a completely idle state
> before pulling the plug?
The status code (plus header) idea could be done. Probably not to
difficult. Will not necessarily work for POST, but will suffice in most
cases.
The check-valid-session stuff would be harder. First we need to run an
http request on our own (which would be useful also for regular
probing), but then it is probably also a performance issue. We can't
simply piggyback the session id on cping/cpong, because the container
needs to at least get the URL to determine the correct context and run
the mapper. IMHO to much to force on every request, even for a node with
activation disabled.
See also my other response.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk and session stickiness
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
All,
On 6/17/14, 10:43 AM, Christopher Schultz wrote:
> All,
>
> I've been using sticky sessions with mod_jk and I can see that
> there is a bit of a problem when attempting to take a backend
> Tomcat server out of load-balanced rotation: a user who never (or
> rarely) restarts their web browser will keep the same JSESSIONID
> cookie forever and therefore end up with the same backend server
> whether it has been disabled or not.
>
> Quick series of events:
>
> 1. User visits load-balancer and gets a randomly-assigned backend
> server/route. We'll call this route "X". The JSESSIONID cookie set
> by the backend server is therefore foo.X.
>
> 2. User's requests are routed by mod_jk to route X.
>
> 3. Route X is disabled using mod_jk's status worker
>
> 4. User's session on server X expires.
>
> [Technically, 3 and 4 can happen in either order]
>
> 5. User makes a new request to the load-balancer, and mod_jk sees
> the JSESSIONID cookie still set to foo.X. mod_jk sends the request
> to route X which allows the user to login, etc.
>
> Thus, it takes more time than necessary to bleed all the traffic
> from route X for maintenance, etc.
>
> Is there a way for mod_jk to ask route X if the session is *still*
> valid? It seems that mod_jk will not re-route a request that looks
> like it's got a valid session id to a new (active) backend server
> unless the backend server X is actually down.
>
> Any ideas?
Any takers?
I was thinking that the following strategy might work:
1. Extend mod_jk's load-balancer worker to include a new directive:
rebalance_statuses, then set the value for that directive to some very
rare HTTP status code(s) like 502.
2. Add a Valve/Filter/Whatever to Tomcat that, when activated because
you want to take a node out of service) returns a 502 when a session
identifier is used that does not map to a currently-valid session and
there is no "force-new-session" header included in the request with
some unpredictable value to prevent end-users from sending their own
valid headers. This could also be handled by the same component
(Manager) that current creates sessions with a new configuration option.
These two together would then bleed-off the aforementioned users: in
step 5, the request would be routed to Tomcat X, then rejected with a
502. mod_jk would detect this and re-balance the worker. If mod_jk
can't re-balance the worker to another backend (e.g. they are all
disabled, stopped, or in a failure state), then mod_jk would send the
request (back) to Tomcat X with the force-new-session header set.
Other than the above (or something similar), or actively probing a
backend Tomcat to see if a session is valid, does anyone have any
ideas for how to get a backend Tomcat into a completely idle state
before pulling the plug?
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTu+RDAAoJEBzwKT+lPKRYWtUP/Rln9w5ou2QO78qwZZ2WX7Cz
PM13BKdnqHZmG1cyDlOPC17bhGynI8xkee+o/ivk1aIsiBeaavhF5M3iBwH9I9fO
/GX/ILKUGn11I3X8E61dGHFxziUVXpBMotOtSvb3+JMo0eMOI1rTo/5gi/GnvXSr
cFkegZ/0IR1PCHVEB++I6HayoyvXLladT5F4FG8vQjQdKHqPX2lt3RFmHdjoyC6D
tgG215w2WPn0arzLvrCvmP07mOeEXNkUf2ai/u9FEx6Dm8poDXIPD9MMS30aUAm9
pFwL0Nc7MhvtUYMj8XmbnnfEMVW/SaXd8u6du2s5d4+2fON7zc27rK0HWAL8OSJQ
0S59q0G3x17JqMrWdQ98eOxCMjYfBR7vBmXXwMiLaVpknxyGRR9AM1sEOQ1zRQCZ
MhHT+hm457LrRu6pSjfUL9L9tGY0kLxX/KwkbKacIoll9rC4gkTHMgwjdPso0YaZ
WMABwDxoymPJpAM1khGjeQSHtBhq31a/H/kfnZSpMZ2Ugl7giBgLVzmJ8Q9HcUqD
/nvLPj1d9Vra3a/bYwkeUiHdpIRNmUUNnu1hoVL0/g+pT0JWSk7jBxznBv5Z/h4o
LNclTOzTP2ijAYJSFMJUKFw4Hg7n91QfSq1BccLQEWg7M5VD5fPKnp/AvOlDGptu
j+2HZ8VdoriHBYoSKuzc
=/IRy
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org