Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Hi,

I have a very rare problem regarding session handling. It is 
reproducible only on a single server environment. Of cause this is the 
productive server.

I use container authentication and for simplicity 'tomcat-user.xml'.

Login is done via HttpServletRequest.login() method, whenever I need to 
access user and role information. The HttpServletRequest is saved in a 
ThreadLocal during request processing.

While that normally does the job. There is one servlet that produces 
simple thumbnails that triggers a problem. Session handling is done via 
Cookies.

A search result page lists several thumbnail images and the browser 
tries to load them (of cause) in parallel.

After the first thumbnail is loaded, the sessionId suddenly changes. As 
the browser now submitting the "false" sessionId, new sessions are 
created and the browser gets that new sessionId in the response. The 
session from the first request is lost at that point and so are the 
login credentials.

Why are sessionIds changing during the request?


Here are the logs I produce via a Servlet Filter:

--------- REQUEST ---------

[dev-mir]  DEBUG  MCRRequestDebugFilter: REQUEST URI: 
/mir/img/pdfthumb/Document_derivate_00000049/2015-05-06_Protokoll%20der%20Sitzung%20des%20Fakult%C3%A4tsrates_%C3%B6ffentlich.pdf 

Cookies:
  {path=null, maxAge=-1, domain=null, name=JSESSIONID, comment=null, 
httpOnly=false, secure=false, class=class javax.servlet.http.Cookie, 
value=BD12BA0ED4546B5BC119727DAF97086B, version=0}
COOKIES END

REQUEST PARAMETERS:
  centerThumb: no,
REQUEST PARAMETERS END

Session is requested by cookie.
Session is not requested by URL.
Session is valid.
SESSION BD12BA0ED4546B5BC119727DAF97086B created at: 2016-01-07T15:18:51.068
SESSION ATTRIBUTES:
  mycore.session: {}
SESSION ATTRIBUTES END

Header:
accept: image/png,image/*;q=0.8,*/*;q=0.5
accept-encoding: gzip, deflate
accept-language: en-US,en;q=0.5
cache-control: max-age=0
connection: keep-alive
cookie: JSESSIONID=BD12BA0ED4546B5BC119727DAF97086B
host: cms.example.com:8291
if-modified-since: Wed, 06 Jan 2016 14:40:00 GMT
if-none-match: "4e0daa3453247bcfd985ee28de9616de"
referer: 
http://cms.example.com:8291/mir/servlets/solr/select?q=objectType:mods%20AND%20createdby:protokoll&sort=mods.dateIssued%20desc
user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 
Firefox/38.0
HEADERS END

--------- RESPONSE  ---------

[dev-mir]  DEBUG  MCRRequestDebugFilter: RESPONSE URI: 
/mir/img/pdfthumb/Document_derivate_00000049/2015-05-06_Protokoll%20der%20Sitzung%20des%20Fakult%C3%A4tsrates_%C3%B6ffentlich.pdf 

Status: 304
Header:
Cache-Control: max-age=31536000
ETag: "4e0daa3453247bcfd985ee28de9616de"
Expires: Sun, 24 Jan 2016 15:30:02 GMT
Set-Cookie: JSESSIONID=0A88AABC33D336900279B2378CD510B1; Path=/mir/; 
HttpOnly
HEADERS END


--------- NEXT REQUEST ---------

[dev-mir]  DEBUG  MCRRequestDebugFilter: REQUEST URI: 
/mir/img/pdfthumb/Document_derivate_00000047/2015-03-25_Protokoll%20der%20Sitzung%20des%20Fakult%C3%A4tsrates_%C3%B6ffentlich.pdf 

Cookies:
  {path=null, maxAge=-1, domain=null, name=JSESSIONID, comment=null, 
httpOnly=false, secure=false, class=class javax.servlet.http.Cookie, 
value=BD12BA0ED4546B5BC119727DAF97086B, version=0}
COOKIES END

REQUEST PARAMETERS:
  centerThumb: no,
REQUEST PARAMETERS END

Session is requested by cookie.
Session is not requested by URL.
Session is not valid.
Header:
accept: image/png,image/*;q=0.8,*/*;q=0.5
accept-encoding: gzip, deflate
accept-language: en-US,en;q=0.5
cache-control: max-age=0
connection: keep-alive
cookie: JSESSIONID=BD12BA0ED4546B5BC119727DAF97086B
host: cms.example.com:8291
if-modified-since: Wed, 06 Jan 2016 14:40:02 GMT
if-none-match: "9e485dcd85a2b2ee373717a67c282873"
referer: 
http://cms.example.com:8291/mir/servlets/solr/select?q=objectType:mods%20AND%20createdby:protokoll&sort=mods.dateIssued%20desc
user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 
Firefox/38.0
HEADERS END




[dev-mir]  DEBUG  MCRServlet: Setting ReqCharEncoding to: UTF-8
[dev-mir]  DEBUG  MCRHttpSessionListener: HttpSession 
20934FB3493889B9EE0FB425A002BB28 is beeing created by: 
org.apache.catalina.session.StandardSessionFacade@6a75756b

--------- RESPONSE ---------

[dev-mir]  DEBUG  MCRRequestDebugFilter: RESPONSE URI: 
/mir/img/pdfthumb/Document_derivate_00000047/2015-03-25_Protokoll%20der%20Sitzung%20des%20Fakult%C3%A4tsrates_%C3%B6ffentlich.pdf 

Status: 304
Header:
Cache-Control: max-age=31536000
ETag: "9e485dcd85a2b2ee373717a67c282873"
Expires: Sun, 24 Jan 2016 15:30:03 GMT
Set-Cookie: JSESSIONID=20934FB3493889B9EE0FB425A002BB28; Path=/mir/; 
HttpOnly
HEADERS END

kind regards,

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Konstantin Kolinko <kn...@gmail.com>]

2016-01-08 19:02 GMT+03:00 Christopher Schultz <ch...@christopherschultz.net>:
> Thomas,
>
> On 1/8/16 8:00 AM, Thomas Scheffler wrote:
>> Am 08.01.16 um 11:43 schrieb Olaf Kock:
>>> Is there any chance that the first and correctly authenticated cookies
>>> (despite the debug output "secure=false") are https-only cookies and
>>> won't get transmitted in http, thus triggering new sessions? E.g. any
>>> chance they get rewritten at another level (Apache httpd, ServletFilter,
>>> others) to be secure only - or that the debug output is slightly
>>> incorrect because it omits the secure flag?
>>
>> This is from a test installation on the productive server where it can
>> only be observed. For simplicity I use the maven cargo plugin to setup
>> the tomcat here. It shows the same behavior on the productive server,
>> where it uses HTTPS in combination with Apache HTTPD.
>>
>> I use BeanUtil.describe() to produce the cookie String. So this should
>> all be correct.
>>
>> This error comes up on every browser with at least a certain number of
>> request to that servlet. It has something to do with a race condition or
>> side effect I'm not aware off.
>>
>> If I do not use container authentication, HTTP sessions won't get lost.
>>
>> Hunting this bugs for so many weeks now and ran out of ideas.
>
> Tomcat will change the session identifier when the user authenticates.
> If you are creating a session before login, you'll see that the session
> id changes when authentication is successful. This is to protect against
> session-fixation attacks.
>
> Can you explain why the changing session id breaks your application? Are
> you storing session ids somewhere and just not updating the session id
> list when the session id changes? It should be possibly to listen for
> that event and update your session id list. Or maybe there's a better
> way to accomplish your goal rather than keeping your own session id
> list. (I'm guessing you have a session id list because it would best
> explain the behavior you are describing here.)

FYI,
http://docs.oracle.com/javaee/7/api/javax/servlet/http/HttpSessionIdListener.html

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

RE: Tomcat 8.0.30 Session lost

[Cris Berneburg - US <cb...@caci.com>]

Thomas

> -----Original Message-----
> From: Olaf Kock [mailto:tomcat@olafkock.de] 
> Sent: Monday, January 11, 2016 4:12 PM
> To: Tomcat Users List
> Subject: Re: Tomcat 8.0.30 Session lost
> 
> Well, at least you do a bit of protection instead of just disabling the session fixation security filter. However, be aware that potentially many people might come from the same IP address - either because it's a NATing home router or a big company's proxy server. Especially if you want to attack someone who's in the same network as yourself, this IP-based protection is quite useless.
> 
> I think I'm seeing the problems, and as long as you know what you're doing and you accept the unavoidable consequences of this lower grade protection, everything is fine.
> 
> It still sounds funny that the session is not available with the next requests... Might be worth to try different browsers for their timing on the receipt of the cookie

+1

Have you tried different browsers, say Internet Explorer, Chrome, and Firefox to see if the behavior is the same for all?

This is kind of a long shot, but I'll mention it anyway.  I had a similar kind of problem, where every resource request on a page was trying to create a new session.  It turns out IE had a bug processing URL's with special chars in them, in my case an underscore "_".  Using a different browser other than IE, or accessing the server by IP address instead of DNS name, hid the problem.  After researching, experimentation, and pulling my hair out, I finally realized the underscore was the issue.  Renaming the server solved the problem.  Use keyword "weirdness" to search for old posts on this topic.

Perhaps another long shot, and please forgive me if you have already made this clear, have you verified the timing and concurrency of your authentication and loading of page resources?  IOW, are they the same, or different use-case events?

> Also, as you call HttpServletRequest.login manually, double check that you're doing this before the response is committed to the client, specifically before any other (old) session id cookie is already in the response stream. It sounds weird, but might help you debug further (note that I'm not looking at tomcat's code. Apologies if I'm not making sense)
> 
> Olaf
> 
> Am 11.01.2016 um 21:52 schrieb Thomas Scheffler:
> >
> > I will file two bugs soon describing the issues I had. Hopefully they 
> > will be fixed.
> >
> > 1.) if using HttpServetRequest.login(String, String) further request 
> > in the session are loosing the users Principal.
> >
> > 2.) After changing sessionId, old sessionIds should still be valid for 
> > a short period of time of to the same client.
> >
> > Fixing one of these would cause the bug to disappear.
> >
> > To prevent session fixation attacks, I use IP address checking so that 
> > sessions are bound to the same IP address.
> >
> > Thanks to all the responses. Without you help it would have not been 
> > possible to get this fixed after two month of searching!
> >
> > kind regards,
> >
> > Thomas
> >

--
Cris Berneburg, Lead Software Engineer, CACI

Re: Tomcat 8.0.30 Session lost

[Christopher Schultz <ch...@christopherschultz.net>]

Thomas,

On 1/13/16 1:04 PM, Thomas Scheffler wrote:
> Am 13.01.16 um 15:48 schrieb Christopher Schultz:
>> Thomas,
>>
>> On 1/13/16 8:31 AM, Thomas Scheffler wrote:
>>> Am 12.01.16 um 13:24 schrieb Mark Thomas:
>>>> On 12/01/2016 11:06, Thomas Scheffler wrote:
>>>>> Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>>>>>>
>>>>>>> <Valve
>>>>>>> className="org.apache.catalina.authenticator.BasicAuthenticator"
>>>>>>>      changeSessionIdOnAuthentication="false" />
>>>>>>>
>>>>>>> Found on
>>>>>>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>>>>>>>
>>>>>>>
>>>>>>> the description how to switch the "feature" off.
>>>>>>>
>>>>>>> I will file two bugs soon describing the issues I had. Hopefully
>>>>>>> they
>>>>>>> will be fixed.
>>>>>>>
>>>>>>> 1.) if using HttpServetRequest.login(String, String) further
>>>>>>> request in
>>>>>>> the session are loosing the users Principal.
>>>>>>>
>>>>>>> 2.) After changing sessionId, old sessionIds should still be valid
>>>>>>> for a
>>>>>>> short period of time of to the same client.
>>>>>>
>>>>>> The second request will get closed as INVALID on security grounds. If
>>>>>> the old ID is valid for any period of time it makes a session
>>>>>> fixation
>>>>>> attack possible. You might as well disable changing the session ID on
>>>>>> authentication.
>>>>>>
>>>>>> For the first the description above isn't clear enough to be sure
>>>>>> exactly what you are asking for. However, based on the second request
>>>>>> and what I have read of this thread I suspect that request will get
>>>>>> closed as INVALID or WONTFIX.
>>>>>
>>>>> Hi Mark,
>>>>>
>>>>> if you choose to use login() and this modifies the session ID. Further
>>>>> calls to login() should either:
>>>>>
>>>>> 1.) are not required as every request belonging to the same session
>>>>> are
>>>>> already authenticated. After login() other request of the same session
>>>>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>>>>
>>>>> 2.) are not required, as authenticate() use the information
>>>>> provided by
>>>>> the first login() call.
>>>>>
>>>>> 3.) do not modify the session ID as the same user was authenticated
>>>>> before and the session is therefor safe to session fixation attacks
>>>>
>>>> Those 3 all boil down to essentially the same requirement.
>>>>
>>>> Requests are populated with cached authentication information from the
>>>> session at the start of the request (if the authenticator is configured
>>>> to do so - all but DIGEST are by default).
>>>
>>> Hi,
>>>
>>> "all but DIGEST are by default" was my case.
>>>
>>> As I walked through the code I found most of the features I requested
>>> are already in place. There is already the tracking of the Principal in
>>> the session. To use the values of login() later in authenticate() in
>>> Tomcat 8.0.30 I had to insert "/foo" as my login page and "/bar" as my
>>> login-error-page. I think any value will do here ;-)
>>>
>>> <login-config>
>>> <auth-method>FORM</auth-method>
>>> <realm-name>Restricted</realm-name>
>>> <form-login-config>
>>> <form-login-page>/foo</form-login-page>
>>> <form-error-page>/bar</form-error-page>
>>> </form-login-config>
>>> </login-config>
>>>
>>> This activates the FormAuthenticator which correctly does, what I was
>>> hoping for.
>>>
>>> I think every authenticator can/should use the information stored by
>>> login() if it is available.
>>
>> Which argument to login() carries the server-generated nonce used for
>> login? Which argument includes the realm name?
>>
>> https://en.wikipedia.org/wiki/Digest_access_authentication
> 
> Hi Chris,
> 
> the login() method gets the username and the password. It has nothing to
> do with configured auth-method in the web.xml.

I'm familiar with this method and how it works.

> Tomcat uses a generic
> method[1] located in a super class do perform the login. The Principal
> is stored in the session, if caching is enabled and a session is present
> or created.
> Depending on the configured auth-method the Principal is used later, if
> found or not. My suggestion was to always use it, when it is stored in
> the session.

Tomcat can't verify that the incoming credentials match those
credentials that were used in a previous login. How could Tomcat decide
if new credentials were being used if Tomcat doesn't actually try to use
them?

Then, if Tomcat uses the credentials and they are valid, the session id
must be changed for session-fixation prevention.

If there is a difference between the BasicAuthenticator and the
DigestAuthenticator, then that gap should probably be closed.

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 13.01.16 um 15:48 schrieb Christopher Schultz:
> Thomas,
>
> On 1/13/16 8:31 AM, Thomas Scheffler wrote:
>> Am 12.01.16 um 13:24 schrieb Mark Thomas:
>>> On 12/01/2016 11:06, Thomas Scheffler wrote:
>>>> Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>>>>>
>>>>>> <Valve
>>>>>> className="org.apache.catalina.authenticator.BasicAuthenticator"
>>>>>>      changeSessionIdOnAuthentication="false" />
>>>>>>
>>>>>> Found on
>>>>>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>>>>>>
>>>>>> the description how to switch the "feature" off.
>>>>>>
>>>>>> I will file two bugs soon describing the issues I had. Hopefully they
>>>>>> will be fixed.
>>>>>>
>>>>>> 1.) if using HttpServetRequest.login(String, String) further
>>>>>> request in
>>>>>> the session are loosing the users Principal.
>>>>>>
>>>>>> 2.) After changing sessionId, old sessionIds should still be valid
>>>>>> for a
>>>>>> short period of time of to the same client.
>>>>>
>>>>> The second request will get closed as INVALID on security grounds. If
>>>>> the old ID is valid for any period of time it makes a session fixation
>>>>> attack possible. You might as well disable changing the session ID on
>>>>> authentication.
>>>>>
>>>>> For the first the description above isn't clear enough to be sure
>>>>> exactly what you are asking for. However, based on the second request
>>>>> and what I have read of this thread I suspect that request will get
>>>>> closed as INVALID or WONTFIX.
>>>>
>>>> Hi Mark,
>>>>
>>>> if you choose to use login() and this modifies the session ID. Further
>>>> calls to login() should either:
>>>>
>>>> 1.) are not required as every request belonging to the same session are
>>>> already authenticated. After login() other request of the same session
>>>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>>>
>>>> 2.) are not required, as authenticate() use the information provided by
>>>> the first login() call.
>>>>
>>>> 3.) do not modify the session ID as the same user was authenticated
>>>> before and the session is therefor safe to session fixation attacks
>>>
>>> Those 3 all boil down to essentially the same requirement.
>>>
>>> Requests are populated with cached authentication information from the
>>> session at the start of the request (if the authenticator is configured
>>> to do so - all but DIGEST are by default).
>>
>> Hi,
>>
>> "all but DIGEST are by default" was my case.
>>
>> As I walked through the code I found most of the features I requested
>> are already in place. There is already the tracking of the Principal in
>> the session. To use the values of login() later in authenticate() in
>> Tomcat 8.0.30 I had to insert "/foo" as my login page and "/bar" as my
>> login-error-page. I think any value will do here ;-)
>>
>> <login-config>
>> <auth-method>FORM</auth-method>
>> <realm-name>Restricted</realm-name>
>> <form-login-config>
>> <form-login-page>/foo</form-login-page>
>> <form-error-page>/bar</form-error-page>
>> </form-login-config>
>> </login-config>
>>
>> This activates the FormAuthenticator which correctly does, what I was
>> hoping for.
>>
>> I think every authenticator can/should use the information stored by
>> login() if it is available.
>
> Which argument to login() carries the server-generated nonce used for
> login? Which argument includes the realm name?
>
> https://en.wikipedia.org/wiki/Digest_access_authentication

Hi Chris,

the login() method gets the username and the password. It has nothing to 
do with configured auth-method in the web.xml. Tomcat uses a generic 
method[1] located in a super class do perform the login. The Principal 
is stored in the session, if caching is enabled and a session is present 
or created.
Depending on the configured auth-method the Principal is used later, if 
found or not. My suggestion was to always use it, when it is stored in 
the session.
Hope this clarifies my summary.

Thomas

[1] org.apache.catalina.authenticator.AuthenticatorBase.doLogin(Request, 
String, String)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Christopher Schultz <ch...@christopherschultz.net>]

Thomas,

On 1/13/16 8:31 AM, Thomas Scheffler wrote:
> Am 12.01.16 um 13:24 schrieb Mark Thomas:
>> On 12/01/2016 11:06, Thomas Scheffler wrote:
>>> Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>>>>
>>>>> <Valve
>>>>> className="org.apache.catalina.authenticator.BasicAuthenticator"
>>>>>     changeSessionIdOnAuthentication="false" />
>>>>>
>>>>> Found on
>>>>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>>>>>
>>>>> the description how to switch the "feature" off.
>>>>>
>>>>> I will file two bugs soon describing the issues I had. Hopefully they
>>>>> will be fixed.
>>>>>
>>>>> 1.) if using HttpServetRequest.login(String, String) further
>>>>> request in
>>>>> the session are loosing the users Principal.
>>>>>
>>>>> 2.) After changing sessionId, old sessionIds should still be valid
>>>>> for a
>>>>> short period of time of to the same client.
>>>>
>>>> The second request will get closed as INVALID on security grounds. If
>>>> the old ID is valid for any period of time it makes a session fixation
>>>> attack possible. You might as well disable changing the session ID on
>>>> authentication.
>>>>
>>>> For the first the description above isn't clear enough to be sure
>>>> exactly what you are asking for. However, based on the second request
>>>> and what I have read of this thread I suspect that request will get
>>>> closed as INVALID or WONTFIX.
>>>
>>> Hi Mark,
>>>
>>> if you choose to use login() and this modifies the session ID. Further
>>> calls to login() should either:
>>>
>>> 1.) are not required as every request belonging to the same session are
>>> already authenticated. After login() other request of the same session
>>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>>
>>> 2.) are not required, as authenticate() use the information provided by
>>> the first login() call.
>>>
>>> 3.) do not modify the session ID as the same user was authenticated
>>> before and the session is therefor safe to session fixation attacks
>>
>> Those 3 all boil down to essentially the same requirement.
>>
>> Requests are populated with cached authentication information from the
>> session at the start of the request (if the authenticator is configured
>> to do so - all but DIGEST are by default).
> 
> Hi,
> 
> "all but DIGEST are by default" was my case.
> 
> As I walked through the code I found most of the features I requested
> are already in place. There is already the tracking of the Principal in
> the session. To use the values of login() later in authenticate() in
> Tomcat 8.0.30 I had to insert "/foo" as my login page and "/bar" as my
> login-error-page. I think any value will do here ;-)
> 
> <login-config>
> <auth-method>FORM</auth-method>
> <realm-name>Restricted</realm-name>
> <form-login-config>
> <form-login-page>/foo</form-login-page>
> <form-error-page>/bar</form-error-page>
> </form-login-config>
> </login-config>
> 
> This activates the FormAuthenticator which correctly does, what I was
> hoping for.
> 
> I think every authenticator can/should use the information stored by
> login() if it is available.

Which argument to login() carries the server-generated nonce used for
login? Which argument includes the realm name?

https://en.wikipedia.org/wiki/Digest_access_authentication

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 12.01.16 um 13:24 schrieb Mark Thomas:
> On 12/01/2016 11:06, Thomas Scheffler wrote:
>> Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>>>
>>>> <Valve className="org.apache.catalina.authenticator.BasicAuthenticator"
>>>>     changeSessionIdOnAuthentication="false" />
>>>>
>>>> Found on
>>>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>>>> the description how to switch the "feature" off.
>>>>
>>>> I will file two bugs soon describing the issues I had. Hopefully they
>>>> will be fixed.
>>>>
>>>> 1.) if using HttpServetRequest.login(String, String) further request in
>>>> the session are loosing the users Principal.
>>>>
>>>> 2.) After changing sessionId, old sessionIds should still be valid for a
>>>> short period of time of to the same client.
>>>
>>> The second request will get closed as INVALID on security grounds. If
>>> the old ID is valid for any period of time it makes a session fixation
>>> attack possible. You might as well disable changing the session ID on
>>> authentication.
>>>
>>> For the first the description above isn't clear enough to be sure
>>> exactly what you are asking for. However, based on the second request
>>> and what I have read of this thread I suspect that request will get
>>> closed as INVALID or WONTFIX.
>>
>> Hi Mark,
>>
>> if you choose to use login() and this modifies the session ID. Further
>> calls to login() should either:
>>
>> 1.) are not required as every request belonging to the same session are
>> already authenticated. After login() other request of the same session
>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>
>> 2.) are not required, as authenticate() use the information provided by
>> the first login() call.
>>
>> 3.) do not modify the session ID as the same user was authenticated
>> before and the session is therefor safe to session fixation attacks
>
> Those 3 all boil down to essentially the same requirement.
>
> Requests are populated with cached authentication information from the
> session at the start of the request (if the authenticator is configured
> to do so - all but DIGEST are by default).

Hi,

"all but DIGEST are by default" was my case.

As I walked through the code I found most of the features I requested 
are already in place. There is already the tracking of the Principal in 
the session. To use the values of login() later in authenticate() in 
Tomcat 8.0.30 I had to insert "/foo" as my login page and "/bar" as my 
login-error-page. I think any value will do here ;-)

<login-config>
<auth-method>FORM</auth-method>
<realm-name>Restricted</realm-name>
<form-login-config>
<form-login-page>/foo</form-login-page>
<form-error-page>/bar</form-error-page>
</form-login-config>
</login-config>

This activates the FormAuthenticator which correctly does, what I was 
hoping for.

I think every authenticator can/should use the information stored by 
login() if it is available.

kind regards

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[David kerber <dc...@verizon.net>]

On 1/12/2016 10:57 AM, Thomas Scheffler wrote:
> Am 12.01.16 um 14:41 schrieb Mark Thomas:
>>>>> 1.) are not required as every request belonging to the same session
>>>>> are
>>>>> already authenticated. After login() other request of the same session
>>>>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>>>>
>>>>> 2.) are not required, as authenticate() use the information
>>>>> provided by
>>>>> the first login() call.
>>>>>
>>>>> 3.) do not modify the session ID as the same user was authenticated
>>>>> before and the session is therefor safe to session fixation attacks
>>>>
>>>> Those 3 all boil down to essentially the same requirement.
>>>
>>> Sorry, I do not see this leed to the same requirement.
>>>
>>> 1.) The Servlet-Spec notes:
>>>
>>> 13.6.3:
>>> "The login method of the HttpServletRequest interface provides an
>>> alternative means for an application to control the look and feel of
>>> it’s login screens."
>>>
>>> 13.6.3.1:
>>> "If the user is authenticated using form login and has created an HTTP
>>> session, the timeout or invalidation of that session leads
>>> to the user being logged out in the sense that subsequent requests must
>>> cause the user to be re-authenticated."
>>>
>>> This defines a call to login() should be handled like Form-Login and not
>>> as Basic-Authentication - like Tomcat currently does. It further states
>>> the the user is logged-out on session timeout and not with every new
>>> request.
>>
>> No it doesn't. You are reading more into the specification than is
>> actually there. The final sentence of 13.6.3 is there to make the reader
>> aware that the login() method exists as an alternative to using FORM
>> authentication. Nothing more.
>
> What I read in the specification is that a *fix* could be implemented
> that would a allow the bug to disappear. The third point above, changing
> the sessionId only if the user is "new" to the session, would fix the
> problem, could be integrated easily and would help me and possibly
> others. Does it open a new security risk? No. Tracking the user
> information in a session is also explicitly allowed by spec.
>
>>>> Requests are populated with cached authentication information from the
>>>> session at the start of the request (if the authenticator is configured
>>>> to do so - all but DIGEST are by default).
>>>
>>> As stated above, if I use the authenticate() method. The user get a
>>> Basic Authentication window asking for a login. At latest there should
>>> be a usage of the already known credentials used by login().
>>
>> As per the specification, calling that method is required to trigger the
>> same authentication process and that is exactly what happens.
>>
>> The authentication process does check for cached credentials but, as
>> with the other authentication related calls on the request, it checks
>> the request not the session and the request is populated from the
>> session at the start of the request processing.
>
> You are right. Again there is nothing that prevents you to help to fix
> that issue, neither spec nor anything. If one uses login() on request
> and during the request a session is created or already exist use the
> FormAuthenticator or create a LoginAuthenticator class instead of
> BasicAuthenticator on other request, when authenticate() is executed.
> authenticate() could get the Authenticator from an AtomicReference in
> the session (if available).
>
>>>> Given that the requested change will add complexity without actually
>>>> solving the problem any enhancement request opened asking for such a
>>>> change will be resolved as WONTFIX.
>>>
>>> Oh come on.
>>
>> Taking that sort of attitude is going to do nothing to help your case or
>> encourage anyone here to spend their time helping you.
>
> You don't see the frustration you brought to me but you could read it. I
> am sorry for that. In a world of asynchronous request handling and
> protocol upgrading you are forcing users to deliver responses
> sequentially. Sorry. This is so disappointing.

Not really.  If you ensure that the authentication is done first, before 
requesting any protected resources, you will then be able to deliver 
things in parallel.  It's just the initial request that is messing you 
up, from what I understand of your description.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Mark Thomas <ma...@apache.org>]

On 12/01/2016 15:57, Thomas Scheffler wrote:
> What I read in the specification is that a *fix* could be implemented
> that would a allow the bug to disappear. The third point above, changing
> the sessionId only if the user is "new" to the session, would fix the
> problem, could be integrated easily and would help me and possibly
> others. Does it open a new security risk? No.

I'm not (yet) convinced that that is the case. I'd need to think through
the various corner cases. But, as explained below, even if there were no
additional security risks, I remain unconvinced about this approach.

> You are right. Again there is nothing that prevents you to help to fix
> that issue, neither spec nor anything.

Yes there is. Read section 13.10 of the servlet spec. I was looking for
something else but found section 13.10 and it is explicit that the
behaviour you are asking for is not specification compliant.

On top of that, the issue can't be fully fixed. And if it can't be fully
fixed then all a partial fix would do is make the issue more difficult
to reproduce and harder to track down.

Even with the fix above, it doesn't solve the following (more involved
than my previous example) race condition:

UA sends sends request1 on connection 1
Request1 creates session1 with sessionid1
Request1 returns response1 to the UA

UA sends request2 with sessionid1 on connection1
UA sends request3 with sessionid1 on connection2

request2 uses session1
reuqest2 returns response2 to the UA

request3 authenticates and sessionid1 changes to sessionid1001

UA sends request4 with sessionid1 on connection1
request fails because sessionid1 no longer exists

Fundamentally, if you want to use Tomcat's session fixation protection
then you have to ensure that authentication can not happen concurrently
with any request for the same session.

Similar issues will occur if you have two parallel requests that both
create a session.

> You don't see the frustration you brought to me but you could read it. I
> am sorry for that. In a world of asynchronous request handling and
> protocol upgrading you are forcing users to deliver responses
> sequentially. Sorry. This is so disappointing.

There is no way around the fact the a request that does any of the
following (and probably other actions as well) can not be concurrent
with any other request from the same user else something is likely to break:
- create a session
- authenticate the user if session fixation protection is enabled

Once you have done those things, you can use all the concurrency you like.

> The spec just states that the user information on the request does not
> change if one does not call login(), logout() or authenticate(). It says
> nothing on the initial state, neither. But when I call authenticate() on
> a request and used login() on an other request object, why can't
> authenticate() not use that credentials? Why does the user have to
> re-login again (basic auth)?

See reference to servlet 13.10 above.

> You are right. But I have no code to alter the session handling. It is a
> standard webpage with restricted information that refers to restricted
> images on the same fast server. A rather simple use case. I use the
> login() method that was introduced in Servlet 3. There is no standard
> conform way to have login() work for me. I cannot modify the browser to
> load the images one by one. I could do by using JavaScript to load the
> images and waiting for each response. Is that what you are suggesting?

I don't know how your application is written so I can't give you
specific advice. What I can say is any web application has to conform to
the constraints I explained above else it will likely break.

> It is just like a homepage that give a tiny information, who is logged
> in. And depending on the user returns some information not readable by
> "guests". No way to force a user to enter his credentials. You can visit
> twitter without being asked for your login. You decide when you log in
> and you never loose a session. One URL returns different results
> depending on the user being logged in.

Because Twitter is careful not to do parallel requests that might
require authentication.

> Of cause, if I require authentication, I can force Tomcat to not use
> BasicAuthentication but FormAuthentication.

That isn't going to solve the parallel request problem.

> But during the page
> generation I used the login() method to get the user information and
> check for roles.

Where is the user information coming from?

> The page is delivered to the user

At this point the UA should have the new session ID since it is after
authentication takes place.

> referencing thumbnails that are itself protected

OK. So requests to those should use the new session ID which should pick
up the authenticated user.

> and lack of authenticate() to work as expected,

The key question is why are these failing.

> calling login() with the same credentials all over again
> for every thumbnail. That's causing all the trouble.

Those requests are going to be concurrent, so yes authenticating here is
likely to break stuff. The point is, if authentication took place for
the referencing page, it shouldn't be necessary again here.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 12.01.16 um 14:41 schrieb Mark Thomas:
>>>> 1.) are not required as every request belonging to the same session are
>>>> already authenticated. After login() other request of the same session
>>>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>>>
>>>> 2.) are not required, as authenticate() use the information provided by
>>>> the first login() call.
>>>>
>>>> 3.) do not modify the session ID as the same user was authenticated
>>>> before and the session is therefor safe to session fixation attacks
>>>
>>> Those 3 all boil down to essentially the same requirement.
>>
>> Sorry, I do not see this leed to the same requirement.
>>
>> 1.) The Servlet-Spec notes:
>>
>> 13.6.3:
>> "The login method of the HttpServletRequest interface provides an
>> alternative means for an application to control the look and feel of
>> it’s login screens."
>>
>> 13.6.3.1:
>> "If the user is authenticated using form login and has created an HTTP
>> session, the timeout or invalidation of that session leads
>> to the user being logged out in the sense that subsequent requests must
>> cause the user to be re-authenticated."
>>
>> This defines a call to login() should be handled like Form-Login and not
>> as Basic-Authentication - like Tomcat currently does. It further states
>> the the user is logged-out on session timeout and not with every new
>> request.
>
> No it doesn't. You are reading more into the specification than is
> actually there. The final sentence of 13.6.3 is there to make the reader
> aware that the login() method exists as an alternative to using FORM
> authentication. Nothing more.

What I read in the specification is that a *fix* could be implemented 
that would a allow the bug to disappear. The third point above, changing 
the sessionId only if the user is "new" to the session, would fix the 
problem, could be integrated easily and would help me and possibly 
others. Does it open a new security risk? No. Tracking the user 
information in a session is also explicitly allowed by spec.

>>> Requests are populated with cached authentication information from the
>>> session at the start of the request (if the authenticator is configured
>>> to do so - all but DIGEST are by default).
>>
>> As stated above, if I use the authenticate() method. The user get a
>> Basic Authentication window asking for a login. At latest there should
>> be a usage of the already known credentials used by login().
>
> As per the specification, calling that method is required to trigger the
> same authentication process and that is exactly what happens.
>
> The authentication process does check for cached credentials but, as
> with the other authentication related calls on the request, it checks
> the request not the session and the request is populated from the
> session at the start of the request processing.

You are right. Again there is nothing that prevents you to help to fix 
that issue, neither spec nor anything. If one uses login() on request 
and during the request a session is created or already exist use the 
FormAuthenticator or create a LoginAuthenticator class instead of 
BasicAuthenticator on other request, when authenticate() is executed. 
authenticate() could get the Authenticator from an AtomicReference in 
the session (if available).

>>> Given that the requested change will add complexity without actually
>>> solving the problem any enhancement request opened asking for such a
>>> change will be resolved as WONTFIX.
>>
>> Oh come on.
>
> Taking that sort of attitude is going to do nothing to help your case or
> encourage anyone here to spend their time helping you.

You don't see the frustration you brought to me but you could read it. I 
am sorry for that. In a world of asynchronous request handling and 
protocol upgrading you are forcing users to deliver responses 
sequentially. Sorry. This is so disappointing.

>> It's the same case with the logout() method. No problem
>> there. Tomcat could handle this gracefully without any problems.
>
> As I explained above, this can't be handled gracefully even with the
> changes you are requesting to Tomcat. Tomcat is required by the Servlet
> spec to throw a ServletException if a call to login() is made after a
> user has already been authenticated. The Servlet API does not provide
> the hooks necessary for an application to handle the race condition of
> parallel requests that require authentication and prevent this exception.
>
> Yes, the application could simply catch the exception and carry on but
> such a design is a large warning that that API is not being used correctly.

The spec just states that the user information on the request does not 
change if one does not call login(), logout() or authenticate(). It says 
nothing on the initial state, neither. But when I call authenticate() on 
a request and used login() on an other request object, why can't 
authenticate() not use that credentials? Why does the user have to 
re-login again (basic auth)?

>>> Therefore, I come back to my earlier point that the correct solution is
>>> for the application accesses authenticated resources in a sane way and
>>> then these issues simply won't be possible. In the specific example for
>>> this thread, the page that generates the list of thumbnails needs to
>>> require authentication as well as (some of) the thumbnails.
>>
>> It boils down to this problem:
>>
>> It's the servlet containers duty to do the session tracking.
>
> Not correct. Responsibility for correct session tracking is split
> between the container, the browser and the application.

You are right. But I have no code to alter the session handling. It is a 
standard webpage with restricted information that refers to restricted 
images on the same fast server. A rather simple use case. I use the 
login() method that was introduced in Servlet 3. There is no standard 
conform way to have login() work for me. I cannot modify the browser to 
load the images one by one. I could do by using JavaScript to load the 
images and waiting for each response. Is that what you are suggesting?

> You could also listen to the advice you are being given. You need to
> change the way your application behaves to avoid it making requests in
> parallel that may require authentication. That has always been
> problematic and now applications and containers need to worry about
> session fixation it is even more likely to be problematic.
>
> If you look back in the archives several years ago when HTML frames were
> popular similar issues existed. The solution then was to ensure that a)
> the top-level frame required authentication and b) after any
> authentication the user was always redirected to a new top-level frame.
>
> This is broadly analogous to your page with multiple thumbnails. If you
> make the page that references the thumbnails require authentication then
> the problem will be solved.

It is just like a homepage that give a tiny information, who is logged 
in. And depending on the user returns some information not readable by 
"guests". No way to force a user to enter his credentials. You can visit 
twitter without being asked for your login. You decide when you log in 
and you never loose a session. One URL returns different results 
depending on the user being logged in.

Of cause, if I require authentication, I can force Tomcat to not use 
BasicAuthentication but FormAuthentication. But during the page 
generation I used the login() method to get the user information and 
check for roles. The page is delivered to the user referencing 
thumbnails that are itself protected and lack of authenticate() to work 
as expected, calling login() with the same credentials all over again 
for every thumbnail. That's causing all the trouble.

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Mark Thomas <ma...@apache.org>]

On 12/01/2016 13:03, Thomas Scheffler wrote:
> Am 12.01.16 um 13:24 schrieb Mark Thomas:
>> On 12/01/2016 11:06, Thomas Scheffler wrote:
>>> Am 11.01.16 um 22:05 schrieb Mark Thomas:

<snip/>

>>>> For the first the description above isn't clear enough to be sure
>>>> exactly what you are asking for. However, based on the second request
>>>> and what I have read of this thread I suspect that request will get
>>>> closed as INVALID or WONTFIX.
>>>
>>> Hi Mark,
>>>
>>> if you choose to use login() and this modifies the session ID. Further
>>> calls to login() should either:
>>>
>>> 1.) are not required as every request belonging to the same session are
>>> already authenticated. After login() other request of the same session
>>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>>
>>> 2.) are not required, as authenticate() use the information provided by
>>> the first login() call.
>>>
>>> 3.) do not modify the session ID as the same user was authenticated
>>> before and the session is therefor safe to session fixation attacks
>>
>> Those 3 all boil down to essentially the same requirement.
> 
> Sorry, I do not see this leed to the same requirement.
> 
> 1.) The Servlet-Spec notes:
> 
> 13.6.3:
> "The login method of the HttpServletRequest interface provides an
> alternative means for an application to control the look and feel of
> it’s login screens."
> 
> 13.6.3.1:
> "If the user is authenticated using form login and has created an HTTP
> session, the timeout or invalidation of that session leads
> to the user being logged out in the sense that subsequent requests must
> cause the user to be re-authenticated."
> 
> This defines a call to login() should be handled like Form-Login and not
> as Basic-Authentication - like Tomcat currently does. It further states
> the the user is logged-out on session timeout and not with every new
> request.

No it doesn't. You are reading more into the specification than is
actually there. The final sentence of 13.6.3 is there to make the reader
aware that the login() method exists as an alternative to using FORM
authentication. Nothing more.

>> Requests are populated with cached authentication information from the
>> session at the start of the request (if the authenticator is configured
>> to do so - all but DIGEST are by default).
> 
> As stated above, if I use the authenticate() method. The user get a
> Basic Authentication window asking for a login. At latest there should
> be a usage of the already known credentials used by login().

As per the specification, calling that method is required to trigger the
same authentication process and that is exactly what happens.

The authentication process does check for cached credentials but, as
with the other authentication related calls on the request, it checks
the request not the session and the request is populated from the
session at the start of the request processing.

>> This request is asking that any method call on the request relating to
>> authentication checks the session if the authenticator is configured to
>> cache authentication information in the session.
>>
>> While this is possible, it won't solve the problem. There will still be
>> a race condition in the application. Consider the following:
>> Request1 - getRemoteUser() == null;
>> Request2 - getRemoteUser() == null;
>> Request2 - login() - OK
>> Request1 - login() - ServletException as user is already logged in
>>
>> Given that the requested change will add complexity without actually
>> solving the problem any enhancement request opened asking for such a
>> change will be resolved as WONTFIX.
> 
> Oh come on.

Taking that sort of attitude is going to do nothing to help your case or
encourage anyone here to spend their time helping you.

> It's the same case with the logout() method. No problem
> there. Tomcat could handle this gracefully without any problems.

As I explained above, this can't be handled gracefully even with the
changes you are requesting to Tomcat. Tomcat is required by the Servlet
spec to throw a ServletException if a call to login() is made after a
user has already been authenticated. The Servlet API does not provide
the hooks necessary for an application to handle the race condition of
parallel requests that require authentication and prevent this exception.

Yes, the application could simply catch the exception and carry on but
such a design is a large warning that that API is not being used correctly.

>> Therefore, I come back to my earlier point that the correct solution is
>> for the application accesses authenticated resources in a sane way and
>> then these issues simply won't be possible. In the specific example for
>> this thread, the page that generates the list of thumbnails needs to
>> require authentication as well as (some of) the thumbnails.
> 
> It boils down to this problem:
> 
> It's the servlet containers duty to do the session tracking.

Not correct. Responsibility for correct session tracking is split
between the container, the browser and the application.

> I showed,
> that there is a racing condition and *Tomcat* currently fails to do so.

You have demonstrated that you are observing a problem with your
application and have correctly diagnosed why you are seeing the
behaviour you are seeing. However, what you have not done is
demonstrated that the root cause of the problem is in Tomcat.

> The client (every browser on the market) are not the problem here, they
> behave like they should (Tomcat tells to value of the session cookie!)

I agree. There is no evidence of a user-agent issue in this thread.

> I file the bug, that's all a user can do.

You could also listen to the advice you are being given. You need to
change the way your application behaves to avoid it making requests in
parallel that may require authentication. That has always been
problematic and now applications and containers need to worry about
session fixation it is even more likely to be problematic.

If you look back in the archives several years ago when HTML frames were
popular similar issues existed. The solution then was to ensure that a)
the top-level frame required authentication and b) after any
authentication the user was always redirected to a new top-level frame.

This is broadly analogous to your page with multiple thumbnails. If you
make the page that references the thumbnails require authentication then
the problem will be solved.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Christopher Schultz <ch...@christopherschultz.net>]

Thomas,

On 1/12/16 8:03 AM, Thomas Scheffler wrote:
> Am 12.01.16 um 13:24 schrieb Mark Thomas:
>> On 12/01/2016 11:06, Thomas Scheffler wrote:
>>> Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>>>>
>>>>> <Valve
>>>>> className="org.apache.catalina.authenticator.BasicAuthenticator"
>>>>>     changeSessionIdOnAuthentication="false" />
>>>>>
>>>>> Found on
>>>>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>>>>>
>>>>> the description how to switch the "feature" off.
>>>>>
>>>>> I will file two bugs soon describing the issues I had. Hopefully they
>>>>> will be fixed.
>>>>>
>>>>> 1.) if using HttpServetRequest.login(String, String) further
>>>>> request in
>>>>> the session are loosing the users Principal.
>>>>>
>>>>> 2.) After changing sessionId, old sessionIds should still be valid
>>>>> for a
>>>>> short period of time of to the same client.
>>>>
>>>> The second request will get closed as INVALID on security grounds. If
>>>> the old ID is valid for any period of time it makes a session fixation
>>>> attack possible. You might as well disable changing the session ID on
>>>> authentication.
>>>>
>>>> For the first the description above isn't clear enough to be sure
>>>> exactly what you are asking for. However, based on the second request
>>>> and what I have read of this thread I suspect that request will get
>>>> closed as INVALID or WONTFIX.
>>>
>>> Hi Mark,
>>>
>>> if you choose to use login() and this modifies the session ID. Further
>>> calls to login() should either:
>>>
>>> 1.) are not required as every request belonging to the same session are
>>> already authenticated. After login() other request of the same session
>>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>>
>>> 2.) are not required, as authenticate() use the information provided by
>>> the first login() call.
>>>
>>> 3.) do not modify the session ID as the same user was authenticated
>>> before and the session is therefor safe to session fixation attacks
>>
>> Those 3 all boil down to essentially the same requirement.
> 
> Sorry, I do not see this leed to the same requirement.
> 
> 1.) The Servlet-Spec notes:
> 
> 13.6.3:
> "The login method of the HttpServletRequest interface provides an
> alternative means for an application to control the look and feel of
> it’s login screens."
> 
> 13.6.3.1:
> "If the user is authenticated using form login and has created an HTTP
> session, the timeout or invalidation of that session leads
> to the user being logged out in the sense that subsequent requests must
> cause the user to be re-authenticated."
> 
> This defines a call to login() should be handled like Form-Login and not
> as Basic-Authentication - like Tomcat currently does. It further states
> the the user is logged-out on session timeout and not with every new
> request.
> 
>> Requests are populated with cached authentication information from the
>> session at the start of the request (if the authenticator is configured
>> to do so - all but DIGEST are by default).
> 
> As stated above, if I use the authenticate() method. The user get a
> Basic Authentication window asking for a login. At latest there should
> be a usage of the already known credentials used by login().
> 
>> This request is asking that any method call on the request relating to
>> authentication checks the session if the authenticator is configured to
>> cache authentication information in the session.
>>
>> While this is possible, it won't solve the problem. There will still be
>> a race condition in the application. Consider the following:
>> Request1 - getRemoteUser() == null;
>> Request2 - getRemoteUser() == null;
>> Request2 - login() - OK
>> Request1 - login() - ServletException as user is already logged in
>>
>> Given that the requested change will add complexity without actually
>> solving the problem any enhancement request opened asking for such a
>> change will be resolved as WONTFIX.
> 
> Oh come on. It's the same case with the logout() method. No problem
> there. Tomcat could handle this gracefully without any problems.
> 
>> Therefore, I come back to my earlier point that the correct solution is
>> for the application accesses authenticated resources in a sane way and
>> then these issues simply won't be possible. In the specific example for
>> this thread, the page that generates the list of thumbnails needs to
>> require authentication as well as (some of) the thumbnails.
> 
> It boils down to this problem:
> 
> It's the servlet containers duty to do the session tracking. I showed,
> that there is a racing condition and *Tomcat* currently fails to do so.
> The client (every browser on the market) are not the problem here, they
> behave like they should (Tomcat tells to value of the session cookie!)
> 
> I file the bug, that's all a user can do.

The race condition is with the application's behavior. Tomcat cannot
protect against the use-case you present here in any sane way. The
problem is that you are issuing many requests simultaneously, some which
do and some which do not require authentication. Those that do not
require authentication are irrelevant, since they should not have any
binding to the session (right)? Why do you allow your users to make
multiple simultaneous requests to the app server that require
authentication instead of authenticating them a single time and then
giving them access to protected resources?

Maybe I just don't understand your use case very well. It sounds like
you want to have a page with mixed-use content and the first request to
require authentication triggers an authentication dialog. Combine that
with the fact that you are issuing your own login() calls means that you
are actually fighting against the process that Tomcat would normally use
for authentication.

Even if things could be locked-down within a single server node in the
way you request, extending this "feature" to a cluster would basically
be impossible. That would mean different semantics depending upon the
deployment model, which is pretty much a deal-breaker all-around.

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 12.01.16 um 13:24 schrieb Mark Thomas:
> On 12/01/2016 11:06, Thomas Scheffler wrote:
>> Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>>>
>>>> <Valve className="org.apache.catalina.authenticator.BasicAuthenticator"
>>>>     changeSessionIdOnAuthentication="false" />
>>>>
>>>> Found on
>>>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>>>> the description how to switch the "feature" off.
>>>>
>>>> I will file two bugs soon describing the issues I had. Hopefully they
>>>> will be fixed.
>>>>
>>>> 1.) if using HttpServetRequest.login(String, String) further request in
>>>> the session are loosing the users Principal.
>>>>
>>>> 2.) After changing sessionId, old sessionIds should still be valid for a
>>>> short period of time of to the same client.
>>>
>>> The second request will get closed as INVALID on security grounds. If
>>> the old ID is valid for any period of time it makes a session fixation
>>> attack possible. You might as well disable changing the session ID on
>>> authentication.
>>>
>>> For the first the description above isn't clear enough to be sure
>>> exactly what you are asking for. However, based on the second request
>>> and what I have read of this thread I suspect that request will get
>>> closed as INVALID or WONTFIX.
>>
>> Hi Mark,
>>
>> if you choose to use login() and this modifies the session ID. Further
>> calls to login() should either:
>>
>> 1.) are not required as every request belonging to the same session are
>> already authenticated. After login() other request of the same session
>> will not return 'null' on getRemoteUser() or getUserPrincipal()
>>
>> 2.) are not required, as authenticate() use the information provided by
>> the first login() call.
>>
>> 3.) do not modify the session ID as the same user was authenticated
>> before and the session is therefor safe to session fixation attacks
>
> Those 3 all boil down to essentially the same requirement.

Sorry, I do not see this leed to the same requirement.

1.) The Servlet-Spec notes:

13.6.3:
"The login method of the HttpServletRequest interface provides an 
alternative means for an application to control the look and feel of 
it’s login screens."

13.6.3.1:
"If the user is authenticated using form login and has created an HTTP 
session, the timeout or invalidation of that session leads
to the user being logged out in the sense that subsequent requests must 
cause the user to be re-authenticated."

This defines a call to login() should be handled like Form-Login and not 
as Basic-Authentication - like Tomcat currently does. It further states 
the the user is logged-out on session timeout and not with every new 
request.

> Requests are populated with cached authentication information from the
> session at the start of the request (if the authenticator is configured
> to do so - all but DIGEST are by default).

As stated above, if I use the authenticate() method. The user get a 
Basic Authentication window asking for a login. At latest there should 
be a usage of the already known credentials used by login().

> This request is asking that any method call on the request relating to
> authentication checks the session if the authenticator is configured to
> cache authentication information in the session.
>
> While this is possible, it won't solve the problem. There will still be
> a race condition in the application. Consider the following:
> Request1 - getRemoteUser() == null;
> Request2 - getRemoteUser() == null;
> Request2 - login() - OK
> Request1 - login() - ServletException as user is already logged in
>
> Given that the requested change will add complexity without actually
> solving the problem any enhancement request opened asking for such a
> change will be resolved as WONTFIX.

Oh come on. It's the same case with the logout() method. No problem 
there. Tomcat could handle this gracefully without any problems.

> Therefore, I come back to my earlier point that the correct solution is
> for the application accesses authenticated resources in a sane way and
> then these issues simply won't be possible. In the specific example for
> this thread, the page that generates the list of thumbnails needs to
> require authentication as well as (some of) the thumbnails.

It boils down to this problem:

It's the servlet containers duty to do the session tracking. I showed, 
that there is a racing condition and *Tomcat* currently fails to do so. 
The client (every browser on the market) are not the problem here, they 
behave like they should (Tomcat tells to value of the session cookie!)

I file the bug, that's all a user can do.

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Mark Thomas <ma...@apache.org>]

On 12/01/2016 11:06, Thomas Scheffler wrote:
> Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>>
>>> <Valve className="org.apache.catalina.authenticator.BasicAuthenticator"
>>>    changeSessionIdOnAuthentication="false" />
>>>
>>> Found on
>>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>>> the description how to switch the "feature" off.
>>>
>>> I will file two bugs soon describing the issues I had. Hopefully they
>>> will be fixed.
>>>
>>> 1.) if using HttpServetRequest.login(String, String) further request in
>>> the session are loosing the users Principal.
>>>
>>> 2.) After changing sessionId, old sessionIds should still be valid for a
>>> short period of time of to the same client.
>>
>> The second request will get closed as INVALID on security grounds. If
>> the old ID is valid for any period of time it makes a session fixation
>> attack possible. You might as well disable changing the session ID on
>> authentication.
>>
>> For the first the description above isn't clear enough to be sure
>> exactly what you are asking for. However, based on the second request
>> and what I have read of this thread I suspect that request will get
>> closed as INVALID or WONTFIX.
> 
> Hi Mark,
> 
> if you choose to use login() and this modifies the session ID. Further
> calls to login() should either:
> 
> 1.) are not required as every request belonging to the same session are
> already authenticated. After login() other request of the same session
> will not return 'null' on getRemoteUser() or getUserPrincipal()
> 
> 2.) are not required, as authenticate() use the information provided by
> the first login() call.
> 
> 3.) do not modify the session ID as the same user was authenticated
> before and the session is therefor safe to session fixation attacks

Those 3 all boil down to essentially the same requirement.

Requests are populated with cached authentication information from the
session at the start of the request (if the authenticator is configured
to do so - all but DIGEST are by default).

This request is asking that any method call on the request relating to
authentication checks the session if the authenticator is configured to
cache authentication information in the session.

While this is possible, it won't solve the problem. There will still be
a race condition in the application. Consider the following:
Request1 - getRemoteUser() == null;
Request2 - getRemoteUser() == null;
Request2 - login() - OK
Request1 - login() - ServletException as user is already logged in

Given that the requested change will add complexity without actually
solving the problem any enhancement request opened asking for such a
change will be resolved as WONTFIX.

Therefore, I come back to my earlier point that the correct solution is
for the application accesses authenticated resources in a sane way and
then these issues simply won't be possible. In the specific example for
this thread, the page that generates the list of thumbnails needs to
require authentication as well as (some of) the thumbnails.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

["André Warnier (tomcat)" <aw...@ice-sa.com>]

On 12.01.2016 12:06, Thomas Scheffler wrote:
> Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>>
>>> <Valve className="org.apache.catalina.authenticator.BasicAuthenticator"
>>>    changeSessionIdOnAuthentication="false" />
>>>
>>> Found on
>>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>>> the description how to switch the "feature" off.
>>>
>>> I will file two bugs soon describing the issues I had. Hopefully they
>>> will be fixed.
>>>
>>> 1.) if using HttpServetRequest.login(String, String) further request in
>>> the session are loosing the users Principal.
>>>
>>> 2.) After changing sessionId, old sessionIds should still be valid for a
>>> short period of time of to the same client.
>>
>> The second request will get closed as INVALID on security grounds. If
>> the old ID is valid for any period of time it makes a session fixation
>> attack possible. You might as well disable changing the session ID on
>> authentication.
>>
>> For the first the description above isn't clear enough to be sure
>> exactly what you are asking for. However, based on the second request
>> and what I have read of this thread I suspect that request will get
>> closed as INVALID or WONTFIX.
>
> Hi Mark,
>
> if you choose to use login() and this modifies the session ID. Further calls to login()
> should either:
>
> 1.) are not required as every request belonging to the same session are already
> authenticated. After login() other request of the same session will not return 'null' on
> getRemoteUser() or getUserPrincipal()
>
> 2.) are not required, as authenticate() use the information provided by the first login()
> call.
>
> 3.) do not modify the session ID as the same user was authenticated before and the session
> is therefor safe to session fixation attacks
>
> If login() modifies the session and the session does not track the authentication
> information, session tracking is not deterministic due to side effects for multiple
> requests at a time.
>
> Fixing either #1, #2 or #3 does not harm security at all but will fix this bug in Tomcat.
>
> I also tried to use authenticate() on the request, but instead of using the last login()
> credentials it forced the use of Basic Authentication. So really I do not see a option
> left on the developer side, how to use login() in a reliable manner.
>

Thomas,
what I was trying to explain to you previously, is this :
Any "serious" webserver software is written so that it conforms to the rules defined in 
the HTTP protocol specification (RFC 2616/2617 and related). That is the case for Tomcat, 
for Apache httpd, and all other webservers. In addition, Tomcat being a java servlet 
engine, also adheres to the rules defined in the Java Servlet Specification.
The HTTP specification and the Java Servlet Specification 
(https://java.net/downloads/servlet-spec/Final/servlet-3_1-final.pdf) have things to say 
about /some/ authentication schemes.  When they do, the corresponding webservers / servlet 
engines respect these things, and applications should do the same.  When the specs do not 
say anything about a particular aspect of user authentication e.g., then the webserver and 
the applications are free to do what they want, as long as what they do does not violate 
another aspect of the specs.

There is no specification which dictates exactly what should happen in terms of 
authentication, for multiple simultaneous requests by a client, for webserver resources 
that require authentication. There is also no specification which dictates exactly in what 
order such simultaneous requests should be processed, nor exactly how long the processing 
of each individual request should take. That is because, at the very basic level, HTTP 1.1 
is such that multiple requests, even if they originate from the same client over a short 
period of time, are still seen as individual, separate requests, from the webserver point 
of view; and according to the protocol specs, the webserver could even "delegate" each of 
these requests to separate distinct processes in the background (think of a front-end 
proxy with a load-balancer for example), and each of these background processes would know 
nothing about any other parallel process handling one of these requests for the same 
client at the same time. (For example, one of these processes would never know that some 
other related process is "just about" at the point of issuing a new session-id to that 
same client).

That is just the way it is, and you are not going to be able to get every producer of 
webserver code, and of browser code, to change this in a way that suits your own idea of 
how things should happen. (It is not forbidden to try, but it will be a long process, much 
longer than changing your application).

So, if you want your HTTP-based application to be portable across different webservers and 
configurations, it is /you/ who must make sure, in the design and logic of your 
application, that a case such as the one you encounter, cannot happen.
One way of doing this would be to design your application in such a way as to /force/ a 
client to always go through a first call to a page which triggers the authentication and 
returns a result to the browser/client, *before* this client has any chance to make 
further calls to other resources which require authentication.

Note that neither Mark nor I are saying that your basic idea is wrong.  It may indeed be 
better, and more friendly to web application developers, if the HTTP protocol had been 
designed in such a way as to avoid this kind of issue.  But it wasn't, and considering the 
number of browsers and webservers in circulation, making any change to the protocol itself 
is a long and protracted process.
You can always try to propose a new HTTP 3.0 version though. If you do, please incorporate 
also the premise that all character exchanges should be Unicode/UTF-8 by default.







---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 11.01.16 um 22:05 schrieb Mark Thomas:
>>
>> <Valve className="org.apache.catalina.authenticator.BasicAuthenticator"
>>    changeSessionIdOnAuthentication="false" />
>>
>> Found on
>> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
>> the description how to switch the "feature" off.
>>
>> I will file two bugs soon describing the issues I had. Hopefully they
>> will be fixed.
>>
>> 1.) if using HttpServetRequest.login(String, String) further request in
>> the session are loosing the users Principal.
>>
>> 2.) After changing sessionId, old sessionIds should still be valid for a
>> short period of time of to the same client.
>
> The second request will get closed as INVALID on security grounds. If
> the old ID is valid for any period of time it makes a session fixation
> attack possible. You might as well disable changing the session ID on
> authentication.
>
> For the first the description above isn't clear enough to be sure
> exactly what you are asking for. However, based on the second request
> and what I have read of this thread I suspect that request will get
> closed as INVALID or WONTFIX.

Hi Mark,

if you choose to use login() and this modifies the session ID. Further 
calls to login() should either:

1.) are not required as every request belonging to the same session are 
already authenticated. After login() other request of the same session 
will not return 'null' on getRemoteUser() or getUserPrincipal()

2.) are not required, as authenticate() use the information provided by 
the first login() call.

3.) do not modify the session ID as the same user was authenticated 
before and the session is therefor safe to session fixation attacks

If login() modifies the session and the session does not track the 
authentication information, session tracking is not deterministic due to 
side effects for multiple requests at a time.

Fixing either #1, #2 or #3 does not harm security at all but will fix 
this bug in Tomcat.

I also tried to use authenticate() on the request, but instead of using 
the last login() credentials it forced the use of Basic Authentication. 
So really I do not see a option left on the developer side, how to use 
login() in a reliable manner.

kind regards

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Mark Thomas <ma...@apache.org>]

On 11/01/2016 20:52, Thomas Scheffler wrote:
> Am 11.01.16 um 12:21 schrieb André Warnier (tomcat):
>> So the solution in your case, is to make sure, in your application
>> logic, that the first unauthenticated request would be totally processed
>> by the server, and the response processed by the client, before the
>> client sends a second request.
>> If you do this, then the second client request /will/ contain the /same/
>> authentication token as the first request, and you will not have this
>> problem.
>>
>> How to achieve this, is left as an exercise for the reader.
> 
> This means changing all browsers to submit only one concurrent request
> if the server is Apache Tomcat? Really?

No, it simply means you have to construct your application in a sane
manner so the client won't issue parallel requests for protected resources.

> 
> <Valve className="org.apache.catalina.authenticator.BasicAuthenticator"
>   changeSessionIdOnAuthentication="false" />
> 
> Found on
> http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection
> the description how to switch the "feature" off.
> 
> I will file two bugs soon describing the issues I had. Hopefully they
> will be fixed.
> 
> 1.) if using HttpServetRequest.login(String, String) further request in
> the session are loosing the users Principal.
> 
> 2.) After changing sessionId, old sessionIds should still be valid for a
> short period of time of to the same client.

The second request will get closed as INVALID on security grounds. If
the old ID is valid for any period of time it makes a session fixation
attack possible. You might as well disable changing the session ID on
authentication.

For the first the description above isn't clear enough to be sure
exactly what you are asking for. However, based on the second request
and what I have read of this thread I suspect that request will get
closed as INVALID or WONTFIX.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Olaf Kock <to...@olafkock.de>]

Hi Christopher,

Thanks for reminding me of my extra doubt that I missed writing of in
the first post:

Picking up on AOL: If I'm on proxy1 now, with many other users - will I
stay on that proxy for a long time? Or will I be loadbalanced to many
other proxies during my visit on the site? There's nothing that enforces
me to come from the same IP every time. It's still common that home
routers might get new addresses at least every 24h, and many mobile
providers don't give out public IP addresses - I'm not sure if I keep my
IP-address if I'm roaming to another network cell.

Conclusion: Protection by IP-address-fixation calls for random problems
that are hard to reproduce.

Olaf

Am 12.01.2016 um 16:51 schrieb Christopher Schultz:
> Olaf,
>
> On 1/11/16 4:12 PM, Olaf Kock wrote:
>> Well, at least you do a bit of protection instead of just disabling the
>> session fixation security filter. However, be aware that potentially
>> many people might come from the same IP address - either because it's a
>> NATing home router or a big company's proxy server. Especially if you
>> want to attack someone who's in the same network as yourself, this
>> IP-based protection is quite useless.
> The primary example of this not working is America Online (which really
> does still exist). I haven't checked recently, but for quite a long
> time, they would proxy everybody through a small set of servers --
> millions of users all appearing to have the same IP address.
>
> Restricting a login to a single IP address (e.g. like Bugzilla can do)?
> That does actually make some sense since it doesn't harm anyone and
> merely fails to add protection to AOL users and others users in a
> similar situation.
>
> But making IP address ~= session id? That will never work properly.
>
> -chris
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Christopher Schultz <ch...@christopherschultz.net>]

Olaf,

On 1/11/16 4:12 PM, Olaf Kock wrote:
> Well, at least you do a bit of protection instead of just disabling the
> session fixation security filter. However, be aware that potentially
> many people might come from the same IP address - either because it's a
> NATing home router or a big company's proxy server. Especially if you
> want to attack someone who's in the same network as yourself, this
> IP-based protection is quite useless.

The primary example of this not working is America Online (which really
does still exist). I haven't checked recently, but for quite a long
time, they would proxy everybody through a small set of servers --
millions of users all appearing to have the same IP address.

Restricting a login to a single IP address (e.g. like Bugzilla can do)?
That does actually make some sense since it doesn't harm anyone and
merely fails to add protection to AOL users and others users in a
similar situation.

But making IP address ~= session id? That will never work properly.

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Olaf Kock <to...@olafkock.de>]

Well, at least you do a bit of protection instead of just disabling the
session fixation security filter. However, be aware that potentially
many people might come from the same IP address - either because it's a
NATing home router or a big company's proxy server. Especially if you
want to attack someone who's in the same network as yourself, this
IP-based protection is quite useless.

I think I'm seeing the problems, and as long as you know what you're
doing and you accept the unavoidable consequences of this lower grade
protection, everything is fine.

It still sounds funny that the session is not available with the next
requests... Might be worth to try different browsers for their timing on
the receipt of the cookie

Also, as you call HttpServletRequest.login manually, double check that
you're doing this before the response is committed to the client,
specifically before any other (old) session id cookie is already in the
response stream. It sounds weird, but might help you debug further (note
that I'm not looking at tomcat's code. Apologies if I'm not making sense)

Olaf

Am 11.01.2016 um 21:52 schrieb Thomas Scheffler:
>
> I will file two bugs soon describing the issues I had. Hopefully they
> will be fixed.
>
> 1.) if using HttpServetRequest.login(String, String) further request
> in the session are loosing the users Principal.
>
> 2.) After changing sessionId, old sessionIds should still be valid for
> a short period of time of to the same client.
>
> Fixing one of these would cause the bug to disappear.
>
> To prevent session fixation attacks, I use IP address checking so that
> sessions are bound to the same IP address.
>
> Thanks to all the responses. Without you help it would have not been
> possible to get this fixed after two month of searching!
>
> kind regards,
>
> Thomas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 11.01.16 um 12:21 schrieb André Warnier (tomcat):
> So the solution in your case, is to make sure, in your application
> logic, that the first unauthenticated request would be totally processed
> by the server, and the response processed by the client, before the
> client sends a second request.
> If you do this, then the second client request /will/ contain the /same/
> authentication token as the first request, and you will not have this
> problem.
>
> How to achieve this, is left as an exercise for the reader.

This means changing all browsers to submit only one concurrent request 
if the server is Apache Tomcat? Really?

<Valve className="org.apache.catalina.authenticator.BasicAuthenticator"
   changeSessionIdOnAuthentication="false" />

Found on 
http://www.tomcatexpert.com/blog/2011/04/25/session-fixation-protection 
the description how to switch the "feature" off.

I will file two bugs soon describing the issues I had. Hopefully they 
will be fixed.

1.) if using HttpServetRequest.login(String, String) further request in 
the session are loosing the users Principal.

2.) After changing sessionId, old sessionIds should still be valid for a 
short period of time of to the same client.

Fixing one of these would cause the bug to disappear.

To prevent session fixation attacks, I use IP address checking so that 
sessions are bound to the same IP address.

Thanks to all the responses. Without you help it would have not been 
possible to get this fixed after two month of searching!

kind regards,

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

["André Warnier (tomcat)" <aw...@ice-sa.com>]

Thomas,

On 11.01.2016 11:30, Thomas Scheffler wrote:
> Am 08.01.16 um 17:02 schrieb Christopher Schultz:
>> Tomcat will change the session identifier when the user authenticates.
>> If you are creating a session before login, you'll see that the session
>> id changes when authentication is successful. This is to protect against
>> session-fixation attacks.
>
> I re-login the user, if tomcat returns "null" on HttpRequest.getUserPrincipal(). I noticed
> that the sessionId changes. But if I am required to re-login the user on parallel
> requests, it depends on magic what sessionId is given after all responses are delivered to
> the browser.
>
> You can see in the logs, that requested sessions are suddenly invalid.
>
> I would not require an other call to the login method if servlet container returns a user
> once it is authenticated in a session.
>
>> Can you explain why the changing session id breaks your application? Are
>> you storing session ids somewhere and just not updating the session id
>> list when the session id changes? It should be possibly to listen for
>> that event and update your session id list. Or maybe there's a better
>> way to accomplish your goal rather than keeping your own session id
>> list. (I'm guessing you have a session id list because it would best
>> explain the behavior you are describing here.)
>
> I do not save the http session id anywhere. The browser saves it in a cookie. The cookie
> is changing rapidly because the UserPrincipaö is not returned from the request.
>
> Here is something I would prefer:
>
> 1. Every request that belongs to a given session returns a non null object when calling
> getUserPrincipal() after the login() method returns successfully.
> 2. As you have no control over the net and do not know in which order the browser receives
> its packets, you can not invalidate a sessionId in a snap. Tomcat has to gracefully
> invalidate it somehow.
>
> An example:
>
> 1. Client logs in
> 2. Server responds with SID=SID1
> 3. Client request 2 resources in parallel
> 4. Server receives first request and handles it, returning SID=SID2
> 5. Server receives second request (with SID=SID2), which belongs to invalid session. Code
> is creating a new session than. Server response with SID=SID3
> 6. Client will use SID3 in future requests which belongs to the new session and not the
> original session where the user is logged in.
>

I think that the crux of the issue, is a proper understanding of how the HTTP protocol's 
Basic Authentication works, in its basic form.  Remember, HTTP is old, and at the 
beginning, it did not envision "sessions", nor "simultaneous" requests.  The basic idea was :
- client connects to, and sends one request to the server
- server processes request, and sends response to the client
- server closes connection, and forgets everything about the request just processed
Then the next request from a client comes in, etc..
Everything else aftwerward, was built on the basic schema above, as "patches", to allow 
for authentication, "sessions" composed of multiple requests/responses, "persistent 
connections", "cookies" etc..
But the basic logic remains the same to this day.
For Basic authentication, the basic schema is as follows :

1) client establishes a connection to the server
2) client sends a request on that connection, for some server "resource"
3) the server checks if that resource is submitted to some form of 
authentication/authorization/access control.
3.1) if not, the server returns the resource to the client, and the request/response cycle 
is finished.
3.2) if yes (AAA required), the server checks if the request contains some form of 
authentication/authorization.
  3.2.1) if yes (auth provided), the server checks if this authentication/authorization 
matches the requirements for this resource.
   3.2.1.1) if yes, the server returns the resource, and the request/response is finished.
   3.2.1.2) if not, the server returns a "forbidden" response, and the request/response is 
finished.
  3.2.2) if not (no auth provided), the server returns a response "authorization required" 
to the client, and the request/response is finished.

In a case like 3.2.2, the client must repeat the request, this time /with/ an 
authentication as required by 3.2 above.

Whenever the server returns a response to the client, it can include a "set cookie" in 
that response.  The next time that the client sends a request to that same server, it will 
then include this cookie, and maybe seeing this cookie will allow the server to respond 
"yes" to the question at 3.2 above.

The above is the basic schema, as included in the HTTP protocol.
There are many other schemes that work, but you have to know that if you differ from the 
schema above, then you are no longer within the strict HTTP protocol, and the 
responsibility of making sure that your alternative scheme works in all circumstances, is 
yours.

In particular, the above scheme will fail in the following scenario :

1) the client sends a request to the server, for a protected resource
2) the server responds with a "authorization required"
3) the client repeats the request, with a valid authentication
4) the server then prepares and sends the response, including a valid "authorization 
token" (in this case, indirectly, this is the session-id cookie)
5) but, before this response has been totally processed by the client, the client already 
starts sending another request for a protected resource.
Of course this request does not (yet) include the proper authentication token, because the 
initial response (4) which contained it, has not yet been completely processed by the client.
So for the server, this (5) is a new and unauthenticated request, and the server treats it 
as such : in your case, it creates a new session, and returns the response with this new 
session.  And this one is different from the session returned for the initial request.
(And because the cookie now returned for (5) has the same name as the cookie returned for 
(4), it overwrites the previous cookie, and this previous cookie (and the session-id in 
it) is now lost.

So the solution in your case, is to make sure, in your application logic, that the first 
unauthenticated request would be totally processed by the server, and the response 
processed by the client, before the client sends a second request.
If you do this, then the second client request /will/ contain the /same/ authentication 
token as the first request, and you will not have this problem.

How to achieve this, is left as an exercise for the reader.



> IMHO it would be also secure to lock the sessionId to the IP address instead of changing
> the id but less error-prone. You can even combine it and allow old sessionIds only from
> the original IP address.
>
> If the servlet container would just use the session information for HTML pages parallel
> request are rare. But when serving cachable thumbnails it often like checking headers and
> return 304.
>
> kind regards,
>
> Thomas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 08.01.16 um 17:02 schrieb Christopher Schultz:
> Tomcat will change the session identifier when the user authenticates.
> If you are creating a session before login, you'll see that the session
> id changes when authentication is successful. This is to protect against
> session-fixation attacks.

I re-login the user, if tomcat returns "null" on 
HttpRequest.getUserPrincipal(). I noticed that the sessionId changes. 
But if I am required to re-login the user on parallel requests, it 
depends on magic what sessionId is given after all responses are 
delivered to the browser.

You can see in the logs, that requested sessions are suddenly invalid.

I would not require an other call to the login method if servlet 
container returns a user once it is authenticated in a session.

> Can you explain why the changing session id breaks your application? Are
> you storing session ids somewhere and just not updating the session id
> list when the session id changes? It should be possibly to listen for
> that event and update your session id list. Or maybe there's a better
> way to accomplish your goal rather than keeping your own session id
> list. (I'm guessing you have a session id list because it would best
> explain the behavior you are describing here.)

I do not save the http session id anywhere. The browser saves it in a 
cookie. The cookie is changing rapidly because the UserPrincipaö is not 
returned from the request.

Here is something I would prefer:

1. Every request that belongs to a given session returns a non null 
object when calling getUserPrincipal() after the login() method returns 
successfully.
2. As you have no control over the net and do not know in which order 
the browser receives its packets, you can not invalidate a sessionId in 
a snap. Tomcat has to gracefully invalidate it somehow.

An example:

1. Client logs in
2. Server responds with SID=SID1
3. Client request 2 resources in parallel
4. Server receives first request and handles it, returning SID=SID2
5. Server receives second request (with SID=SID2), which belongs to 
invalid session. Code is creating a new session than. Server response 
with SID=SID3
6. Client will use SID3 in future requests which belongs to the new 
session and not the original session where the user is logged in.

IMHO it would be also secure to lock the sessionId to the IP address 
instead of changing the id but less error-prone. You can even combine it 
and allow old sessionIds only from the original IP address.

If the servlet container would just use the session information for HTML 
pages parallel request are rare. But when serving cachable thumbnails it 
often like checking headers and return 304.

kind regards,

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Christopher Schultz <ch...@christopherschultz.net>]

Thomas,

On 1/8/16 8:00 AM, Thomas Scheffler wrote:
> Am 08.01.16 um 11:43 schrieb Olaf Kock:
>> Is there any chance that the first and correctly authenticated cookies
>> (despite the debug output "secure=false") are https-only cookies and
>> won't get transmitted in http, thus triggering new sessions? E.g. any
>> chance they get rewritten at another level (Apache httpd, ServletFilter,
>> others) to be secure only - or that the debug output is slightly
>> incorrect because it omits the secure flag?
> 
> This is from a test installation on the productive server where it can
> only be observed. For simplicity I use the maven cargo plugin to setup
> the tomcat here. It shows the same behavior on the productive server,
> where it uses HTTPS in combination with Apache HTTPD.
> 
> I use BeanUtil.describe() to produce the cookie String. So this should
> all be correct.
> 
> This error comes up on every browser with at least a certain number of
> request to that servlet. It has something to do with a race condition or
> side effect I'm not aware off.
> 
> If I do not use container authentication, HTTP sessions won't get lost.
> 
> Hunting this bugs for so many weeks now and ran out of ideas.

Tomcat will change the session identifier when the user authenticates.
If you are creating a session before login, you'll see that the session
id changes when authentication is successful. This is to protect against
session-fixation attacks.

Can you explain why the changing session id breaks your application? Are
you storing session ids somewhere and just not updating the session id
list when the session id changes? It should be possibly to listen for
that event and update your session id list. Or maybe there's a better
way to accomplish your goal rather than keeping your own session id
list. (I'm guessing you have a session id list because it would best
explain the behavior you are describing here.)

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 08.01.16 um 11:43 schrieb Olaf Kock:
> Is there any chance that the first and correctly authenticated cookies
> (despite the debug output "secure=false") are https-only cookies and
> won't get transmitted in http, thus triggering new sessions? E.g. any
> chance they get rewritten at another level (Apache httpd, ServletFilter,
> others) to be secure only - or that the debug output is slightly
> incorrect because it omits the secure flag?

This is from a test installation on the productive server where it can 
only be observed. For simplicity I use the maven cargo plugin to setup 
the tomcat here. It shows the same behavior on the productive server, 
where it uses HTTPS in combination with Apache HTTPD.

I use BeanUtil.describe() to produce the cookie String. So this should 
all be correct.

This error comes up on every browser with at least a certain number of 
request to that servlet. It has something to do with a race condition or 
side effect I'm not aware off.

If I do not use container authentication, HTTP sessions won't get lost.

Hunting this bugs for so many weeks now and ran out of ideas.

regards,

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Olaf Kock <to...@olafkock.de>]

Is there any chance that the first and correctly authenticated cookies
(despite the debug output "secure=false") are https-only cookies and
won't get transmitted in http, thus triggering new sessions? E.g. any
chance they get rewritten at another level (Apache httpd, ServletFilter,
others) to be secure only - or that the debug output is slightly
incorrect because it omits the secure flag?

Olaf

Am 08.01.2016 um 10:07 schrieb Thomas Scheffler:
> Hi,
>
> I have a very rare problem regarding session handling. It is
> reproducible only on a single server environment. Of cause this is the
> productive server.
>
> I use container authentication and for simplicity 'tomcat-user.xml'.
>
> Login is done via HttpServletRequest.login() method, whenever I need
> to access user and role information. The HttpServletRequest is saved
> in a ThreadLocal during request processing.
>
> While that normally does the job. There is one servlet that produces
> simple thumbnails that triggers a problem. Session handling is done
> via Cookies.
>
> A search result page lists several thumbnail images and the browser
> tries to load them (of cause) in parallel.
>
> After the first thumbnail is loaded, the sessionId suddenly changes.
> As the browser now submitting the "false" sessionId, new sessions are
> created and the browser gets that new sessionId in the response. The
> session from the first request is lost at that point and so are the
> login credentials.
>
> Why are sessionIds changing during the request?
>
>
> Here are the logs I produce via a Servlet Filter:
>
> --------- REQUEST ---------
>
> [dev-mir]  DEBUG  MCRRequestDebugFilter: REQUEST URI:
> /mir/img/pdfthumb/Document_derivate_00000049/2015-05-06_Protokoll%20der%20Sitzung%20des%20Fakult%C3%A4tsrates_%C3%B6ffentlich.pdf
>
> Cookies:
>  {path=null, maxAge=-1, domain=null, name=JSESSIONID, comment=null,
> httpOnly=false, secure=false, class=class javax.servlet.http.Cookie,
> value=BD12BA0ED4546B5BC119727DAF97086B, version=0}
> COOKIES END
>
> REQUEST PARAMETERS:
>  centerThumb: no,
> REQUEST PARAMETERS END
>
> Session is requested by cookie.
> Session is not requested by URL.
> Session is valid.
> SESSION BD12BA0ED4546B5BC119727DAF97086B created at:
> 2016-01-07T15:18:51.068
> SESSION ATTRIBUTES:
>  mycore.session: {}
> SESSION ATTRIBUTES END
>
> Header:
> accept: image/png,image/*;q=0.8,*/*;q=0.5
> accept-encoding: gzip, deflate
> accept-language: en-US,en;q=0.5
> cache-control: max-age=0
> connection: keep-alive
> cookie: JSESSIONID=BD12BA0ED4546B5BC119727DAF97086B
> host: cms.example.com:8291
> if-modified-since: Wed, 06 Jan 2016 14:40:00 GMT
> if-none-match: "4e0daa3453247bcfd985ee28de9616de"
> referer:
> http://cms.example.com:8291/mir/servlets/solr/select?q=objectType:mods%20AND%20createdby:protokoll&sort=mods.dateIssued%20desc
> user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
> Firefox/38.0
> HEADERS END
>
> --------- RESPONSE  ---------
>
> [dev-mir]  DEBUG  MCRRequestDebugFilter: RESPONSE URI:
> /mir/img/pdfthumb/Document_derivate_00000049/2015-05-06_Protokoll%20der%20Sitzung%20des%20Fakult%C3%A4tsrates_%C3%B6ffentlich.pdf
>
> Status: 304
> Header:
> Cache-Control: max-age=31536000
> ETag: "4e0daa3453247bcfd985ee28de9616de"
> Expires: Sun, 24 Jan 2016 15:30:02 GMT
> Set-Cookie: JSESSIONID=0A88AABC33D336900279B2378CD510B1; Path=/mir/;
> HttpOnly
> HEADERS END
>
>
> --------- NEXT REQUEST ---------
>
> [dev-mir]  DEBUG  MCRRequestDebugFilter: REQUEST URI:
> /mir/img/pdfthumb/Document_derivate_00000047/2015-03-25_Protokoll%20der%20Sitzung%20des%20Fakult%C3%A4tsrates_%C3%B6ffentlich.pdf
>
> Cookies:
>  {path=null, maxAge=-1, domain=null, name=JSESSIONID, comment=null,
> httpOnly=false, secure=false, class=class javax.servlet.http.Cookie,
> value=BD12BA0ED4546B5BC119727DAF97086B, version=0}
> COOKIES END
>
> REQUEST PARAMETERS:
>  centerThumb: no,
> REQUEST PARAMETERS END
>
> Session is requested by cookie.
> Session is not requested by URL.
> Session is not valid.
> Header:
> accept: image/png,image/*;q=0.8,*/*;q=0.5
> accept-encoding: gzip, deflate
> accept-language: en-US,en;q=0.5
> cache-control: max-age=0
> connection: keep-alive
> cookie: JSESSIONID=BD12BA0ED4546B5BC119727DAF97086B
> host: cms.example.com:8291
> if-modified-since: Wed, 06 Jan 2016 14:40:02 GMT
> if-none-match: "9e485dcd85a2b2ee373717a67c282873"
> referer:
> http://cms.example.com:8291/mir/servlets/solr/select?q=objectType:mods%20AND%20createdby:protokoll&sort=mods.dateIssued%20desc
> user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
> Firefox/38.0
> HEADERS END
>
>
>
>
> [dev-mir]  DEBUG  MCRServlet: Setting ReqCharEncoding to: UTF-8
> [dev-mir]  DEBUG  MCRHttpSessionListener: HttpSession
> 20934FB3493889B9EE0FB425A002BB28 is beeing created by:
> org.apache.catalina.session.StandardSessionFacade@6a75756b
>
> --------- RESPONSE ---------
>
> [dev-mir]  DEBUG  MCRRequestDebugFilter: RESPONSE URI:
> /mir/img/pdfthumb/Document_derivate_00000047/2015-03-25_Protokoll%20der%20Sitzung%20des%20Fakult%C3%A4tsrates_%C3%B6ffentlich.pdf
>
> Status: 304
> Header:
> Cache-Control: max-age=31536000
> ETag: "9e485dcd85a2b2ee373717a67c282873"
> Expires: Sun, 24 Jan 2016 15:30:03 GMT
> Set-Cookie: JSESSIONID=20934FB3493889B9EE0FB425A002BB28; Path=/mir/;
> HttpOnly
> HEADERS END
>
> kind regards,
>
> Thomas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

[Thomas Scheffler <th...@uni-jena.de>]

Am 08.01.16 um 14:03 schrieb André Warnier (tomcat):
> Hi Thomas.
>
> It is a bit difficult to figure out where the problem really is, without
> having the full picture of what is going on (your web.xml configuration,
> the order and precise timing in which requests really happen etc.).
> But one thing I would really focus on, is the general overall logic of
> the application, to see if the order in which things happen is really
> consistent with the authentication method that you are using.
> For example, if the initial call to the first page (the one which
> contains the document links) does not immediately trigger a session
> authentication, and in the meantime the browser starts making multiple
> quasi-simultaneous requests for the links present in that page, then
> things would get out of sync, since when the authentication does happen,
> the session-id /will/ change (check Wikipedia for "web session
> fixation"). That would overwrite the session-id cookie, perhaps in the
> middle of the calls still being made to retrieve the document links
> mentioned in the page.

Hi André,

I would like to give some insight information on how authentication is 
handled.

As the application can run in command line or as a web application we 
have a custom session handling (refer to it as 'core session'). Within 
the core session there is a UserInformation implementation attached. 
UserInformation can have different implementations and allows to ask for 
the user id and if the user in in a specific role.

In the HttpSession we only store the sessionId of our core session. When 
the http session is destroyed we also close the core session.

So there is only this one property "mycore.session" in the httpsession.

When using servlet container authentication (error happens only with 
it). I need to access the request object which is stored in a 
ThreadLocal and released at end of the request.

Here are the most important methods

private void loginIfNeeded() {
     //getCurrentRequest() returns Optional<HttpServletRequest>
     getCurrentRequest().ifPresent(currentRequest -> {
         if (currentRequest.getUserPrincipal() == null) {
             try {
                 currentRequest.login(user, pwd);
                 LOGGER.debug("Re-Logged in: " + user);
             } catch (ServletException e) {
                 throw new RuntimeException(e);
             }
         }
     });
}

public String getUserID() {
     loginIfNeeded();
     lastUser = getCurrentRequest()
         .flatMap(r -> Optional.ofNullable(r.getUserPrincipal()))
         .map(Principal::getName)
         .orElseGet(() -> Optional.ofNullable(lastUser)
         .orElseGet("guest");
     return lastUser;
}

public boolean isUserInRole(String role) {
     loginIfNeeded();
     return getCurrentRequest().map(r -> 
r.isUserInRole(role)).orElse(Boolean.FALSE);
}

When the user first authenticated with his login credentials we put the 
servlet container user information to the core session. As the HTTP 
session is linked to it, we just need the http session to keep track of it.

The error happens to be later, (possibly) many requests after the login 
form that put UserInformation to the core session. So there is no racing 
condition at that position.

You can see in the code that I need to re-login from time to time, as 
the request object does not always return the remote user.

Why does that change the sessionId? How can I track the user than?

IMHO if HttpServletRequest.login() should only be called once in a http 
session than getUserPrincipal() should always return a user after 
login() is called on one request regarding to the same session, or?

But if calling login() always invalidates the old session we have a 
problem here.

As long as request are coming in sequentially this would be OK, as the 
browser gets a new cookie value with every response. But browsers are 
out of control and want to give cpu cores a good time.

Anyway. I hope you are not lost in the details and are still willing to 
help me.

regards,

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Tomcat 8.0.30 Session lost

["André Warnier (tomcat)" <aw...@ice-sa.com>]

On 08.01.2016 10:07, Thomas Scheffler wrote:
> Hi,
>
> I have a very rare problem regarding session handling. It is reproducible only on a single
> server environment. Of cause this is the productive server.
>
> I use container authentication and for simplicity 'tomcat-user.xml'.
>
> Login is done via HttpServletRequest.login() method, whenever I need to access user and
> role information. The HttpServletRequest is saved in a ThreadLocal during request processing.
>
> While that normally does the job. There is one servlet that produces simple thumbnails
> that triggers a problem. Session handling is done via Cookies.
>
> A search result page lists several thumbnail images and the browser tries to load them (of
> cause) in parallel.
>
> After the first thumbnail is loaded, the sessionId suddenly changes. As the browser now
> submitting the "false" sessionId, new sessions are created and the browser gets that new
> sessionId in the response. The session from the first request is lost at that point and so
> are the login credentials.
>
> Why are sessionIds changing during the request?
>
>
> Here are the logs I produce via a Servlet Filter:
>
[snip]

Hi Thomas.

It is a bit difficult to figure out where the problem really is, without having the full 
picture of what is going on (your web.xml configuration, the order and precise timing in 
which requests really happen etc.).
But one thing I would really focus on, is the general overall logic of the application, to 
see if the order in which things happen is really consistent with the authentication 
method that you are using.
For example, if the initial call to the first page (the one which contains the document 
links) does not immediately trigger a session authentication, and in the meantime the 
browser starts making multiple quasi-simultaneous requests for the links present in that 
page, then things would get out of sync, since when the authentication does happen, the 
session-id /will/ change (check Wikipedia for "web session fixation"). That would 
overwrite the session-id cookie, perhaps in the middle of the calls still being made to 
retrieve the document links mentioned in the page.




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org