You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by juan <jj...@gmail.com> on 2021/01/15 14:57:00 UTC
Mixed User Session
Hi,
We were running tomcat 8.5.57 on CentOS 7 and together with CAS SSO, we
have multiple servers behind an AWS load balancer setup with sticky
sessions. We encountered a weird situation where a user who logged into
their application was presented with another users profile after login. Has
anyone encountered something similar to this? Both users hit the same
tomcat server seconds apart and the user was on his personal computer and
doesn't know the first user.
We have async-supported set to true for both default and jsp servlets and
using the http11.Http11NioProtocol connector with
sslImplementationName="org.apache.tomcat.util.net.openssl.OpenSSLImplementation"
and <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />.
Unfortunately we could not find any errors in the logs.
We encountered something similar in the past when running on tomcat 8.0 and
it was related to low entropy on the environment and tomcat not able to
generate a session for the incoming user and reusing the jsessionid for a
previous user. We did see errors for this in the logs. This was fixed by
adding service to increase entropy and upgrading to a later version of
tomcat 8.5.
Any suggestions or help is much appreciated regarding logging that might
help shed some light on this. We have also patched tomcat to that lastest
release 8.5.61 to see if it helps.
Thanks!
Juan Q.
Re: Mixed User Session
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Juan,
On 1/15/21 10:58, Christopher Schultz wrote:
> Juan,
>
> On 1/15/21 09:57, juan wrote:
>> We were running tomcat 8.5.57 on CentOS 7 and together with CAS SSO, we
>> have multiple servers behind an AWS load balancer setup with sticky
>> sessions. We encountered a weird situation where a user who logged into
>> their application was presented with another users profile after
>> login. Has
>> anyone encountered something similar to this? Both users hit the same
>> tomcat server seconds apart and the user was on his personal computer and
>> doesn't know the first user.
I forgot to ask a few things about this:
1. Is this reproducible? How many cases of this have you encountered?
2. Is your application holding on to references to request/response
beyond the end of a request?
Usually this kind of thing happens when applications are misusing those
objects by retaining references.
>> We have async-supported set to true for both default and jsp servlets and
>> using the http11.Http11NioProtocol connector with
>> sslImplementationName="org.apache.tomcat.util.net.openssl.OpenSSLImplementation"
>>
>> and <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol"
>> />.
Are you actually using servlet-async features? They are very tricky use
correctly without any mistakes.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Mixed User Session
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Juan,
On 1/15/21 09:57, juan wrote:
> We were running tomcat 8.5.57 on CentOS 7 and together with CAS SSO, we
> have multiple servers behind an AWS load balancer setup with sticky
> sessions. We encountered a weird situation where a user who logged into
> their application was presented with another users profile after login. Has
> anyone encountered something similar to this? Both users hit the same
> tomcat server seconds apart and the user was on his personal computer and
> doesn't know the first user.
>
> We have async-supported set to true for both default and jsp servlets and
> using the http11.Http11NioProtocol connector with
> sslImplementationName="org.apache.tomcat.util.net.openssl.OpenSSLImplementation"
> and <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />.
> Unfortunately we could not find any errors in the logs.
Are you using h2 as a protocol between the AWS lb and Tomcat, or are you
using HTTP? Or, are you using a network load balancer which just allows
bytes to flow back and forth without actually proxying HTTP(2)?
I'm assuming you have:
HTTP(2) -> ELB/ALB -> HTTP -> Tomcat
I just want to check to see if h2 is actually being used by Tomcat.
If h2 is being used as a proxy protocol, do you know if you are using h2
or h2c (non-encrypted)?
Just trying to narrow things down.
> We encountered something similar in the past when running on tomcat 8.0 and
> it was related to low entropy on the environment and tomcat not able to
> generate a session for the incoming user and reusing the jsessionid for a
> previous user. We did see errors for this in the logs. This was fixed by
> adding service to increase entropy and upgrading to a later version of
> tomcat 8.5.
That sounds suspicious to me. If you don't have sufficient entropy, you
should get very slow response times, but no re-use of session identifiers.
> Any suggestions or help is much appreciated regarding logging that might
> help shed some light on this. We have also patched tomcat to that lastest
> release 8.5.61 to see if it helps.
Can you please post your entire <Connector> element from
conf/server.xml? Remove any sensitive information that may be present.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Mixed User Session
Posted by juan <jj...@gmail.com>.
Hi,
Below is the connector definition, we had 1 case reported that we know of
but during the reported incident couldn't find anything in the logs during
the transactions. We haven't been able to reproduce the issue. We do have
another environment that was added into the mix and handles the
authentication piece. So we receive information back from them which
provides the necessary data to CAS SSO to send to the client app that
initiated the request. We did have conversations with them and they did
mention the possibility of session information for another user being
provided back. We installed haveged to help us have sufficient entropy.
When we first encountered something like we handled authentication locally
on the environment and we saw errors in the tomcat logs, this time
authentication is handled externally and there are no errors found in the
log.
<Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol"
sslImplementationName="org.apache.tomcat.util.net.openssl.OpenSSLImplementation"
SSLEnabled="true" maxHttpHeaderSize="40960" connectionTimeout="20000"
maxThreads="450">
<SSLHostConfig
ciphers="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384"
honorCipherOrder="true" protocols="TLSv1.2"
disableSessionTickets="true">
<Certificate
certificateKeystoreFile="xxxxx"
certificateKeystorePassword="xxxxxxx"
type="RSA" />
</SSLHostConfig>
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
</Connector>
Thanks!
On Fri, Jan 15, 2021 at 11:05 AM Olaf Kock <to...@olafkock.de> wrote:
>
> On 15.01.21 15:57, juan wrote:
> > Hi,
> >
> > We were running tomcat 8.5.57 on CentOS 7 and together with CAS SSO, we
> > have multiple servers behind an AWS load balancer setup with sticky
> > sessions. We encountered a weird situation where a user who logged into
> > their application was presented with another users profile after login.
> Has
> > anyone encountered something similar to this? Both users hit the same
> > tomcat server seconds apart and the user was on his personal computer and
> > doesn't know the first user.
>
> I've seen stuff like this happening when a reverse proxy was over-eager
> in caching stuff that it saw. 100% of cases where I've seen this
> behavior had this as a root cause.
>
> "over-eager" might mean that Tomcat (e.g. your app) doesn't signal the
> upstream proxy that the content is private, or the upstream proxy
> ignoring such signals.
>
>
> It's been a looooong time since I last saw it (and back then I didn't
> fix it myself - just provided information like the above) so I barely
> remember the key points to configure or look after. Maybe it helps
> already, otherwise we'll have to wait for someone to point to the usual
> suspects in the individual configurations.
>
> Olaf
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Mixed User Session
Posted by Olaf Kock <to...@olafkock.de>.
On 15.01.21 15:57, juan wrote:
> Hi,
>
> We were running tomcat 8.5.57 on CentOS 7 and together with CAS SSO, we
> have multiple servers behind an AWS load balancer setup with sticky
> sessions. We encountered a weird situation where a user who logged into
> their application was presented with another users profile after login. Has
> anyone encountered something similar to this? Both users hit the same
> tomcat server seconds apart and the user was on his personal computer and
> doesn't know the first user.
I've seen stuff like this happening when a reverse proxy was over-eager
in caching stuff that it saw. 100% of cases where I've seen this
behavior had this as a root cause.
"over-eager" might mean that Tomcat (e.g. your app) doesn't signal the
upstream proxy that the content is private, or the upstream proxy
ignoring such signals.
It's been a looooong time since I last saw it (and back then I didn't
fix it myself - just provided information like the above) so I barely
remember the key points to configure or look after. Maybe it helps
already, otherwise we'll have to wait for someone to point to the usual
suspects in the individual configurations.
Olaf
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org