You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Uw...@t-systems.com on 2011/11/25 13:43:32 UTC

Mysterious request edirect with value exchange

Hello together,

I recently ran into an issue which probably has nothing to do with tomcat. But because I don't know how to get rid of it
or what the cause for this issue is I want to make sure that I have taken all chances.

Just a short architecture overview.

We have a Loadbalancer which roundrobin addresses 2 Apache 2.2. which passes their requests via mod_proxy balancer to 2
tomcat 5.5.26 servers which are configured identical and have also clustering enabled. The application is located only on the tomcats.

So what we have is a redirected response from a failed login process.
As example we have an login mask and while tryinig to login we put the username "Tester" in the username inputfield
send the form away and get a login error message and the following URL string:
http://mytestsystem/login.action?login_error=1&u=Tester
In the input field the value Tester is still set.

This result is as expected, but from time to time we saw the following URL string returned
http://mytestsystem/login.action?login_error=1&u=OtherUser although we sent the form
with Tester as value away. But now in the input field it is also OtherUser set.

We have checked Loadbalancer rewrites(possible Proxywrites) and fount nothing. The behaviour seem to appear if we
have much load on these systems. Does anybody have any idea where I should start my investigation.
I would assume the application does have a problem with session handling or a corruptable redirection method but
the developer told us everything is fine.

Mit freundlichen Grüßen / Yours sincerely

Uwe Hellmann

T-Systems International GmbH
Production CSS
GCU Large Accounts
AC Express & Logistics LT1

Gradestr. 20, 30163 Hannover
+49 511 333 - 1233 (Tel.)
+49 511 333 - 1287 (Fax)
Email: uwe.hellmann@t-systems.com<ma...@t-systems.com>
Internet: http://www.t-systems.com<http://www.t-systems.com/>

T-Systems International GmbH
Aufsichtsrat: René Obermann (Vorsitzender)
Geschäftsführung: Reinhard Clemens (Vorsitzender), Dr. Ferri Abolhassan, Ulrich Meister, Georg Pepping, Hagen Rickmann, Dietmar Wendt, Klaus Werner
Handelsregister: Amtsgericht Frankfurt am Main HRB 55933
Sitz der Gesellschaft: Frankfurt am Main


Notice: This transmittal and/or attachments may be privileged or confidential. If you are not the intended recipient, you are hereby notified that you have received this transmittal in error; any review, dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.



Re: Mysterious request edirect with value exchange

Posted by Chema <de...@gmail.com>.
OK
Just for discard what is answering Chuck in another thread: a
request-specific value kept in a servlet instance field



2011/12/20  <Uw...@t-systems.com>:
> It is a webformular.
>
> The java code should this fragment.
>
> final String username = req.getParameter("username");
>    String uParam = "";
>    if (StringUtils.isBlank(username) == false) {
>      uParam = "&u=".concat(username);
>    }
>    logonFilter.setLoginFailedUrl("/action?login_error=1".concat(uParam));
>
> -----Ursprüngliche Nachricht-----
> Von: Chema [mailto:demablogia@gmail.com]
> Gesendet: Dienstag, 20. Dezember 2011 15:04
> An: Tomcat Users List
> Betreff: Re: Mysterious request edirect with value exchange
>
>>> This result is as expected, but from time to time we saw the
>>> following URL string returned
>>> http://mytestsystem/login.action?login_error=1&u=OtherUser although
>>> we sent the form with Tester as value away. But now in the input
>>> field it is also OtherUser set.
>
> One question
> How the returned URL is built ? Where is stored username ?
>
> Thanks
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: AW: Mysterious request edirect with value exchange

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Uwe,

On 12/20/11 9:07 AM, Uwe.Hellmann@t-systems.com wrote:
> It is a webformular.
> 
> The java code should this fragment.
> 
> final String username = req.getParameter("username"); String uParam
> = ""; if (StringUtils.isBlank(username) == false) { uParam =
> "&u=".concat(username); } 
> logonFilter.setLoginFailedUrl("/action?login_error=1".concat(uParam));

Ignoring
> 
the unnecessary concatenation when no parameter will be
added, and the unnecessary comparison against "false", this looks
fairly straightforward.

Note that you have an XSS vulnerability in the above code.

I notice that the "final String username" has different indentation.
Does that come from another part of the code?

I also notice that your URL from your original message is
"/login.action" but this URL above is simply "/action". Is that due to
inconsistent obfuscation of your code, or is this incorrect?

If you are using Struts 2 (judging from the ".action" you are), where
does the object that runs the above code go into the value stack?

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7wzIMACgkQ9CaO5/Lv0PCmBACdEH5tqQ4vpxGmZvKSWOqidWi9
v4EAnA0YmO5gQivSzhL2oY8Ud9EEUkMN
=fFXq
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Mysterious request edirect with value exchange

Posted by Rainer Jung <ra...@kippdata.de>.
On 20.12.2011 15:07, Uwe.Hellmann@t-systems.com wrote:
> It is a webformular.
>
> The java code should this fragment.
>
> final String username = req.getParameter("username");
>      String uParam = "";
>      if (StringUtils.isBlank(username) == false) {
>        uParam = "&u=".concat(username);
>      }
>      logonFilter.setLoginFailedUrl("/action?login_error=1".concat(uParam));

So what about "logonFilter". Are the instances shared between requests?

In addition: as Konstantin already wrote, you can activate the Tomcat 
access log and add "%{Location}o" to the pattern. This will log your 
redirect URLs directly in the Tomcat access log. That way you can check, 
whether the redirect is already wrong there and if yes ignore all 
components in front in the root cause analysis.

If the login precedure is simple you can write a simple JMeter test 
script and stress the test system with concurrent requests. JMeter can 
also be configured to check the redirect for the correct URL.

Regards,

Rainer


> -----Ursprüngliche Nachricht-----
> Von: Chema [mailto:demablogia@gmail.com]
> Gesendet: Dienstag, 20. Dezember 2011 15:04
> An: Tomcat Users List
> Betreff: Re: Mysterious request edirect with value exchange
>
>>> This result is as expected, but from time to time we saw the
>>> following URL string returned
>>> http://mytestsystem/login.action?login_error=1&u=OtherUser although
>>> we sent the form with Tester as value away. But now in the input
>>> field it is also OtherUser set.
>
> One question
> How the returned URL is built ? Where is stored username ?
>
> Thanks
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


AW: Mysterious request edirect with value exchange

Posted by Uw...@t-systems.com.
It is a webformular.

The java code should this fragment.

final String username = req.getParameter("username");
    String uParam = "";
    if (StringUtils.isBlank(username) == false) {
      uParam = "&u=".concat(username);
    }
    logonFilter.setLoginFailedUrl("/action?login_error=1".concat(uParam));

-----Ursprüngliche Nachricht-----
Von: Chema [mailto:demablogia@gmail.com] 
Gesendet: Dienstag, 20. Dezember 2011 15:04
An: Tomcat Users List
Betreff: Re: Mysterious request edirect with value exchange

>> This result is as expected, but from time to time we saw the
>> following URL string returned
>> http://mytestsystem/login.action?login_error=1&u=OtherUser although
>> we sent the form with Tester as value away. But now in the input
>> field it is also OtherUser set.

One question
How the returned URL is built ? Where is stored username ?

Thanks

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Mysterious request edirect with value exchange

Posted by Chema <de...@gmail.com>.
>> This result is as expected, but from time to time we saw the
>> following URL string returned
>> http://mytestsystem/login.action?login_error=1&u=OtherUser although
>> we sent the form with Tester as value away. But now in the input
>> field it is also OtherUser set.

One question
How the returned URL is built ? Where is stored username ?

Thanks

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: AW: Mysterious request edirect with value exchange

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Uwe,

On 12/20/11 8:51 AM, Uwe.Hellmann@t-systems.com wrote:
> Thanks for the answer. We have a test environment but there it
> isn't reproducible.

That's unfortunate.

> Just on the production system. But everything is configured
> identical. We use Apache with mod_proxy (via AJP).
> 
> Only the connector? It is pretty standard.
> 
> <Listener className="org.apache.catalina.core.AprLifecycleListener"
> />

Are you actually using APR? Many people have the listener enabled, but
the native library isn't installed.

> <Connector port="8080" maxHttpHeaderSize="8192" maxThreads="150"
> minSpareThreads="25" maxSpareThreads="75" enableLookups="false"
> redirectPort="8443" acceptCount="100" connectionTimeout="20000"
> disableUploadTimeout="true" />

That's not an AJP connector. So, how are you connecting httpd to
Tomcat? Perhaps you are using mod_proxy_http.

Honestly, it sounds from your original description (a month ago) like
you have a component that is inappropriately sharing information
between requests. I'd take a careful look at your authentication code.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7wy2oACgkQ9CaO5/Lv0PAyYgCeNdfYx7Sk/icfAfvHs70WFOQJ
/s8AoJTPPUONR+VdBDL+3lRUP3fNvIQP
=WqYa
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


AW: Mysterious request edirect with value exchange

Posted by Uw...@t-systems.com.
Thanks for the answer.
We have a test environment but there it isn't reproducible.
Just on the production system. But everything is configured identical.
We use Apache with mod_proxy (via AJP).

Only the connector? It is pretty standard.

<Server port="8005" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.core.AprLifecycleListener" />
  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.storeconfig.StoreConfigLifecycleListener"/>

<GlobalNamingResources>
<Environment name="simpleValue" type="java.lang.Integer" value="30"/>
<Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
       description="User database that can be updated and saved"
           factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
          pathname="conf/tomcat-users.xml" />

  </GlobalNamingResources>
<Service name="Catalina">



<Connector port="8080" maxHttpHeaderSize="8192"
               maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
               enableLookups="false" redirectPort="8443" acceptCount="100"
               connectionTimeout="20000" disableUploadTimeout="true" />
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvmroute1">

<Host name="localhost" appBase="webapps"
       unpackWARs="true" autoDeploy="true"
       xmlValidation="false" xmlNamespaceAware="false">

<Cluster className="org.apache.catalina.cluster.tcp.SimpleTcpCluster"
                 managerClassName="org.apache.catalina.cluster.session.DeltaManager"
                 expireSessionsOnShutdown="false"
                 useDirtyFlag="true"
                 notifyListenersOnReplication="true">

            <Membership
                 className="org.apache.catalina.cluster.mcast.McastService"
                 mcastAddr="228.0.0.4"
                 mcastPort="45564"
                 mcastFrequency="500"
                 mcastDropTime="3000"/>

            <Receiver
                className="org.apache.catalina.cluster.tcp.ReplicationListener"
                tcpListenAddress="10.197.120.52"
                tcpListenPort="4002"
                tcpSelectorTimeout="100"
                tcpThreadCount="6"/>

            <Sender
                className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"
                replicationMode="pooled"
                ackTimeout="15000"/>

            <Valve className="org.apache.catalina.cluster.tcp.ReplicationValve"
                filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.css;.*\.txt;"/>

            <Deployer className="org.apache.catalina.cluster.deploy.FarmWarDeployer"
                tempDir="/tmp/war-temp/"
                deployDir="/tmp/war-deploy/"
                watchDir="/tmp/war-listen/"
                watchEnabled="false"/>

            <ClusterListener
                className="org.apache.catalina.cluster.session.ClusterSessionListener"/>

        </Cluster>

</Host>

    </Engine>

  </Service>

</Server>

The Apache Tomcat Native library is not installed.

Thanks 
Uwe



-----Ursprüngliche Nachricht-----
Von: Christopher Schultz [mailto:chris@christopherschultz.net] 
Gesendet: Freitag, 25. November 2011 15:33
An: Tomcat Users List
Betreff: Re: Mysterious request edirect with value exchange

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Uwe,

On 11/25/11 7:43 AM, Uwe.Hellmann@t-systems.com wrote:
> We have a Loadbalancer which roundrobin addresses 2 Apache 2.2.
> which passes their requests via mod_proxy balancer to 2 tomcat
> 5.5.26 servers which are configured identical and have also
> clustering enabled. The application is located only on the
> tomcats.

Are you using HTTP or AJP? From your description, it's probably HTTP
but it's worth asking.

Are you able to test/reproduce without the lb or a second Tomcat in
the mix? I'm sure this is a rare occurrence, so reproducing might not
be easy. If you have a test bed environment that is identical to
production (you do have one, right?), can you load it down and observe
intermittent failures like this? If so, try hitting a single Tomcat
directly and see if the problem disappears.

> So what we have is a redirected response from a failed login
> process. As example we have an login mask and while tryinig to
> login we put the username "Tester" in the username inputfield send
> the form away and get a login error message and the following URL
> string: http://mytestsystem/login.action?login_error=1&u=Tester In
> the input field the value Tester is still set.
> 
> This result is as expected, but from time to time we saw the
> following URL string returned 
> http://mytestsystem/login.action?login_error=1&u=OtherUser although
> we sent the form with Tester as value away. But now in the input
> field it is also OtherUser set.

Have you reviewed the code in your test? Sometimes the tests are
flawed. :(

> We have checked Loadbalancer rewrites(possible Proxywrites) and
> fount nothing. The behaviour seem to appear if we have much load on
> these systems. Does anybody have any idea where I should start my
> investigation. I would assume the application does have a problem
> with session handling or a corruptable redirection method but the
> developer told us everything is fine.

Of course the developer told you that the software is bulletproof. :)

Can you post your <Connector> elements from your server.xml (minus
anything that might be considered sensitive, of course), and let us
know if you are using APR (aka "native") with Tomcat.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7PpxcACgkQ9CaO5/Lv0PCuigCgl0CL4wzLoe616M4NBPBVKm6i
jnIAn0q2Y8G8D1PYqAEeSB3VojOfMgms
=Th2F
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Mysterious request edirect with value exchange

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Uwe,

On 11/25/11 7:43 AM, Uwe.Hellmann@t-systems.com wrote:
> We have a Loadbalancer which roundrobin addresses 2 Apache 2.2.
> which passes their requests via mod_proxy balancer to 2 tomcat
> 5.5.26 servers which are configured identical and have also
> clustering enabled. The application is located only on the
> tomcats.

Are you using HTTP or AJP? From your description, it's probably HTTP
but it's worth asking.

Are you able to test/reproduce without the lb or a second Tomcat in
the mix? I'm sure this is a rare occurrence, so reproducing might not
be easy. If you have a test bed environment that is identical to
production (you do have one, right?), can you load it down and observe
intermittent failures like this? If so, try hitting a single Tomcat
directly and see if the problem disappears.

> So what we have is a redirected response from a failed login
> process. As example we have an login mask and while tryinig to
> login we put the username "Tester" in the username inputfield send
> the form away and get a login error message and the following URL
> string: http://mytestsystem/login.action?login_error=1&u=Tester In
> the input field the value Tester is still set.
> 
> This result is as expected, but from time to time we saw the
> following URL string returned 
> http://mytestsystem/login.action?login_error=1&u=OtherUser although
> we sent the form with Tester as value away. But now in the input
> field it is also OtherUser set.

Have you reviewed the code in your test? Sometimes the tests are
flawed. :(

> We have checked Loadbalancer rewrites(possible Proxywrites) and
> fount nothing. The behaviour seem to appear if we have much load on
> these systems. Does anybody have any idea where I should start my
> investigation. I would assume the application does have a problem
> with session handling or a corruptable redirection method but the
> developer told us everything is fine.

Of course the developer told you that the software is bulletproof. :)

Can you post your <Connector> elements from your server.xml (minus
anything that might be considered sensitive, of course), and let us
know if you are using APR (aka "native") with Tomcat.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7PpxcACgkQ9CaO5/Lv0PCuigCgl0CL4wzLoe616M4NBPBVKm6i
jnIAn0q2Y8G8D1PYqAEeSB3VojOfMgms
=Th2F
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Mysterious request edirect with value exchange

Posted by Konstantin Kolinko <kn...@gmail.com>.
2011/11/25  <Uw...@t-systems.com>:
>
> So what we have is a redirected response from a failed login process.

Is it FORM authentication, or some custom implemented login process?

> As example we have an login mask and while tryinig to login we put the username "Tester" in the username inputfield
> send the form away and get a login error message and the following URL string:
> http://mytestsystem/login.action?login_error=1&u=Tester
> In the input field the value Tester is still set.
>
> This result is as expected, but from time to time we saw the following URL string returned
> http://mytestsystem/login.action?login_error=1&u=OtherUser although we sent the form
> with Tester as value away. But now in the input field it is also OtherUser set.

I think I would configure an AccessLogValve with "%{Location}o" in it.
That should log what redirect URL is actually sent to the client in
Location response header.

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org