You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Rainer Jung <ra...@kippdata.de> on 2008/08/25 17:16:13 UTC
5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Hi,
URIEncoding="UTF-8" seems to be broken for 5.5.trunk at the moment. The
reason is related to one of the fixes for BZ 44494. It doesn't have to
do with the fixes for URIEncoding CVE.
6.0.18 works. For 5.5.trunk when URIEncoding="UTF-8" you always get a
redirect to the ROOT context start page if the rquest is handled by a
reycled instance of I-don't-know-yet-which-object.
So the first few rquests are handled fine, but very soon you only get
redirects back and end up in a redirect loop.
The URL doesn't need to contain any non-ASCII characters, e.g.
http://myserver:8080/ is enough for reproduction, and is also the
redirected URL.
If we revert the backport of
http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
then the redirect loop is gone, and the usual content gets served, but
we know, that this change was needed to fix the "remaining garbage" part
of 44494. So reverting it without any alternative is not really an option.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Posted by Rainer Jung <ra...@kippdata.de>.
I need a small correction: when I delete the available method of the
IntermediateInputStream, I can neither reproduce the redirect problem,
nor bz44494. My test against bz44494 was wrong.
Nevertheless I would strongly prefer the removal of conv.recycle in
convertURI in the Adapter code, because I think I understand much
better, that we don't need this, than I understand the exact need for
the available() method.
BTW: The conv.recycle in convertURI is there since the method convertURI
was created. At that time recycle was an empty method though. I think we
now gave it a meaning, which is not right for both use cases of
B2CConverter.
I didn't test it, but I expect TC 4.1 trunk to be broken in the same way.
Concerning tc6.0 and trunk: we should remove conv.recycle() from
convertURI there as well and whoever knows, why we have the available()
method should comment, if we need to introduce it to 6.0 and trunk too.
Regards,
Rainer
Rainer Jung schrieb:
> Unfortunately removing the available() method agains gives us back
> 44494. But there is some hope.
>
> B2CConverter gets used for two different things:
>
> 1) It is associated with a request and used inside the adapter to decode
> the URI.
>
> In this case, it seems that during the lifetime of the ByteChunk
> underlying the B2CConverter (the URI ByteChunk) we only call
> conv.convert() once. So there is no real need for any conv.recycle()
> before conv.convert() (cleaning up some unrelated old ByteChunk; if we
> would need to do that, then we should do it at the end of convertURI for
> some request and not near the beginning of convertURI for some later
> request).
>
> 2) It is associated with the InputBuffer to transform input data.
>
> In this case we mightcall convert() multiple times and handle a
> difficult relationship between direct access to the Chunks and via
> convert. Here we need to clean up the ByteChunk at the end of the
> request handling.
>
> As far as I can see, the instances of B2CConverter used for 1) and for
> 2) are independant of each other.
>
> B2CConvertor.recycle() has only one action, it calls
> ReadConvertor.recycle(), which reads the remaining bytes in the
> ByteChunk() and throws them away. This seems to be good for 2) (e.g. BZ
> 44494), but it eats data from a ByteChunk instance associated with the
> B2CConverter during an earlier request in case 1).
>
> I would suggest to simply remove the conv.recycle in case 1). I dn't see
> any use of it. If we really think we need to eat up remaining bytes in
> the ByteChunk in 1), then we should do the conv.recycle() near the end
> of convertURI.
>
> Regards,
>
> Rainer
>
> Rainer Jung schrieb:
>> It's the available() methde of the class IntermediateInputStream
>> contained in B2CConverter. It doesn't exist in 6.0. If I comment it out
>> in 5.5 trunk. the problem is gone.
>>
>> The method was first introduced in
>>
>> http://svn.apache.org/viewvc/tomcat/connectors/trunk/util/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=481614&r2=568699&pathrev=568699&diff_format=h
>>
>> and the changed to it's final contents in
>>
>> http://svn.apache.org/viewvc/tomcat/connectors/trunk/util/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=568699&r2=569970&diff_format=h
>>
>> I didn't yet check, what negative consequences removing of the method
>> has, or if there is a better implementation for it. We should carefully
>> check the consequences of changing it w.r.t. BZ 44494 with the test
>> webapp, we got for that BZ.
>>
>> Regards,
>>
>> Rainer
>>
>>
>> Rainer Jung schrieb:
>>> OK, cancelled my appointment. More info:
>>>
>>> I backported all functional changes in o.a.tomcat.util.buf from tc6.x to
>>> tc5.5 and can't reproduce the problem any more. Those are very few
>>> changes. I'll narrow it down some more during the next hour. Stay tuned.
>>>
>>> Rainer
>>>
>>> Rainer Jung schrieb:
>>>> Filip Hanik - Dev Lists schrieb:
>>>>> Thanks Rainer, I will take a look at it tonight
>>>> Thank you!
>>>>
>>>> Last info chunk for today: in CoyoteAdapter.convertURI, before the
>>>> try/catch block that either creates or recycles the B2CConverter, the
>>>> ByteChunk bc coming from the decodedURI contains the correct URI. After
>>>> the recycle of the B2CConverter, the ByteChunk is empty and thus we end
>>>> up in a default redirect.
>>>>
>>>> Although we already read the correct URI from the request, the
>>>> B2CConverter associated with the request detroys the already read URI in
>>>> the recycle. I don't see the delta to 6.0. CoyoteAdapter seems fine,
>>>> maybe in ByteChunk?
>>>>
>>>> Regards,
>>>>
>>>> Rainer
>>>>
>>>>> Filip
>>>>>
>>>>> Rainer Jung wrote:
>>>>>> Remy Maucherat schrieb:
>>>>>>
>>>>>>> On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
>>>>>>>
>>>>>>>> If we revert the backport of
>>>>>>>>
>>>>>>>> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>>>>>>>>
>>>>>>>>
>>>>>>>> then the redirect loop is gone, and the usual content gets served, but
>>>>>>>> we know, that this change was needed to fix the "remaining garbage"
>>>>>>>> part
>>>>>>>> of 44494. So reverting it without any alternative is not really an
>>>>>>>> option.
>>>>>>>>
>>>>>>> Most likely, some backport is missing (many patches made up this input
>>>>>>> fixes). OTOH, I don't think this particular problem is that critical, so
>>>>>>> I would be in favor of dropping it if fixing the issue is complex.
>>>>>>>
>>>>>> Got some more info:
>>>>>>
>>>>>> 1) What happens inside the new ReadConvertor.recycle(): actually if I
>>>>>> print out, which "superfluous" bytes are eaten by the read() loop inside
>>>>>> ReadConvertor.recycle(), I can see, that it's all the bytes making up
>>>>>> the PATH in the URL. If I request http://myserver:8080/, recycle read
>>>>>> one character, namely "/", if I request http://myserver:8080/index.jsp,
>>>>>> then recycle reads all characters from "/index.jsp". In my understanding
>>>>>> of those patches, the time recycle in ReadConvertor gets called, those
>>>>>> should have alrady been read and only body bytes left over after request
>>>>>> processing should be eaten.
>>>>>>
>>>>>> 2) So I checked, when recycle() gets called, and I see, that during the
>>>>>> first few (here: 2) requests it doesn't get called at all (and those
>>>>>> work), and during the following broken requests, it gets called in the
>>>>>> following stack:
>>>>>>
>>>>>> at
>>>>>> org.apache.tomcat.util.buf.ReadConvertor.recycle(B2CConverter.java:222)
>>>>>> at
>>>>>> org.apache.tomcat.util.buf.B2CConverter.recycle(B2CConverter.java:64)
>>>>>> at
>>>>>> org.apache.catalina.connector.CoyoteAdapter.convertURI(CoyoteAdapter.java:475)
>>>>>>
>>>>>> at
>>>>>> org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:265)
>>>>>>
>>>>>> at
>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:172)
>>>>>>
>>>>>> at
>>>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>>>>>>
>>>>>> at
>>>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>>>>
>>>>>> at
>>>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>>>>
>>>>>> at
>>>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>>>>
>>>>>> at
>>>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>>>>
>>>>>> at java.lang.Thread.run(Thread.java:595)
>>>>>>
>>>>>> Didn't yet go further into it, but maybe something is wrong in
>>>>>> CoyoteAdapter.convertURI?
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Rainer
>>>>>>
>>>>>> P.S: I'll soon need to stop investigating this for today. If anyone can
>>>>>> take over that will be nice, because we really should have a working
>>>>>> 5.5.27 soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Posted by Rainer Jung <ra...@kippdata.de>.
Unfortunately removing the available() method agains gives us back
44494. But there is some hope.
B2CConverter gets used for two different things:
1) It is associated with a request and used inside the adapter to decode
the URI.
In this case, it seems that during the lifetime of the ByteChunk
underlying the B2CConverter (the URI ByteChunk) we only call
conv.convert() once. So there is no real need for any conv.recycle()
before conv.convert() (cleaning up some unrelated old ByteChunk; if we
would need to do that, then we should do it at the end of convertURI for
some request and not near the beginning of convertURI for some later
request).
2) It is associated with the InputBuffer to transform input data.
In this case we mightcall convert() multiple times and handle a
difficult relationship between direct access to the Chunks and via
convert. Here we need to clean up the ByteChunk at the end of the
request handling.
As far as I can see, the instances of B2CConverter used for 1) and for
2) are independant of each other.
B2CConvertor.recycle() has only one action, it calls
ReadConvertor.recycle(), which reads the remaining bytes in the
ByteChunk() and throws them away. This seems to be good for 2) (e.g. BZ
44494), but it eats data from a ByteChunk instance associated with the
B2CConverter during an earlier request in case 1).
I would suggest to simply remove the conv.recycle in case 1). I dn't see
any use of it. If we really think we need to eat up remaining bytes in
the ByteChunk in 1), then we should do the conv.recycle() near the end
of convertURI.
Regards,
Rainer
Rainer Jung schrieb:
> It's the available() methde of the class IntermediateInputStream
> contained in B2CConverter. It doesn't exist in 6.0. If I comment it out
> in 5.5 trunk. the problem is gone.
>
> The method was first introduced in
>
> http://svn.apache.org/viewvc/tomcat/connectors/trunk/util/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=481614&r2=568699&pathrev=568699&diff_format=h
>
> and the changed to it's final contents in
>
> http://svn.apache.org/viewvc/tomcat/connectors/trunk/util/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=568699&r2=569970&diff_format=h
>
> I didn't yet check, what negative consequences removing of the method
> has, or if there is a better implementation for it. We should carefully
> check the consequences of changing it w.r.t. BZ 44494 with the test
> webapp, we got for that BZ.
>
> Regards,
>
> Rainer
>
>
> Rainer Jung schrieb:
>> OK, cancelled my appointment. More info:
>>
>> I backported all functional changes in o.a.tomcat.util.buf from tc6.x to
>> tc5.5 and can't reproduce the problem any more. Those are very few
>> changes. I'll narrow it down some more during the next hour. Stay tuned.
>>
>> Rainer
>>
>> Rainer Jung schrieb:
>>> Filip Hanik - Dev Lists schrieb:
>>>> Thanks Rainer, I will take a look at it tonight
>>> Thank you!
>>>
>>> Last info chunk for today: in CoyoteAdapter.convertURI, before the
>>> try/catch block that either creates or recycles the B2CConverter, the
>>> ByteChunk bc coming from the decodedURI contains the correct URI. After
>>> the recycle of the B2CConverter, the ByteChunk is empty and thus we end
>>> up in a default redirect.
>>>
>>> Although we already read the correct URI from the request, the
>>> B2CConverter associated with the request detroys the already read URI in
>>> the recycle. I don't see the delta to 6.0. CoyoteAdapter seems fine,
>>> maybe in ByteChunk?
>>>
>>> Regards,
>>>
>>> Rainer
>>>
>>>> Filip
>>>>
>>>> Rainer Jung wrote:
>>>>> Remy Maucherat schrieb:
>>>>>
>>>>>> On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
>>>>>>
>>>>>>> If we revert the backport of
>>>>>>>
>>>>>>> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>>>>>>>
>>>>>>>
>>>>>>> then the redirect loop is gone, and the usual content gets served, but
>>>>>>> we know, that this change was needed to fix the "remaining garbage"
>>>>>>> part
>>>>>>> of 44494. So reverting it without any alternative is not really an
>>>>>>> option.
>>>>>>>
>>>>>> Most likely, some backport is missing (many patches made up this input
>>>>>> fixes). OTOH, I don't think this particular problem is that critical, so
>>>>>> I would be in favor of dropping it if fixing the issue is complex.
>>>>>>
>>>>> Got some more info:
>>>>>
>>>>> 1) What happens inside the new ReadConvertor.recycle(): actually if I
>>>>> print out, which "superfluous" bytes are eaten by the read() loop inside
>>>>> ReadConvertor.recycle(), I can see, that it's all the bytes making up
>>>>> the PATH in the URL. If I request http://myserver:8080/, recycle read
>>>>> one character, namely "/", if I request http://myserver:8080/index.jsp,
>>>>> then recycle reads all characters from "/index.jsp". In my understanding
>>>>> of those patches, the time recycle in ReadConvertor gets called, those
>>>>> should have alrady been read and only body bytes left over after request
>>>>> processing should be eaten.
>>>>>
>>>>> 2) So I checked, when recycle() gets called, and I see, that during the
>>>>> first few (here: 2) requests it doesn't get called at all (and those
>>>>> work), and during the following broken requests, it gets called in the
>>>>> following stack:
>>>>>
>>>>> at
>>>>> org.apache.tomcat.util.buf.ReadConvertor.recycle(B2CConverter.java:222)
>>>>> at
>>>>> org.apache.tomcat.util.buf.B2CConverter.recycle(B2CConverter.java:64)
>>>>> at
>>>>> org.apache.catalina.connector.CoyoteAdapter.convertURI(CoyoteAdapter.java:475)
>>>>>
>>>>> at
>>>>> org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:265)
>>>>>
>>>>> at
>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:172)
>>>>>
>>>>> at
>>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>>>>>
>>>>> at
>>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>>>
>>>>> at
>>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>>>
>>>>> at
>>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>>>
>>>>> at
>>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>>>
>>>>> at java.lang.Thread.run(Thread.java:595)
>>>>>
>>>>> Didn't yet go further into it, but maybe something is wrong in
>>>>> CoyoteAdapter.convertURI?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Rainer
>>>>>
>>>>> P.S: I'll soon need to stop investigating this for today. If anyone can
>>>>> take over that will be nice, because we really should have a working
>>>>> 5.5.27 soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Posted by Rainer Jung <ra...@kippdata.de>.
It's the available() methde of the class IntermediateInputStream
contained in B2CConverter. It doesn't exist in 6.0. If I comment it out
in 5.5 trunk. the problem is gone.
The method was first introduced in
http://svn.apache.org/viewvc/tomcat/connectors/trunk/util/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=481614&r2=568699&pathrev=568699&diff_format=h
and the changed to it's final contents in
http://svn.apache.org/viewvc/tomcat/connectors/trunk/util/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=568699&r2=569970&diff_format=h
I didn't yet check, what negative consequences removing of the method
has, or if there is a better implementation for it. We should carefully
check the consequences of changing it w.r.t. BZ 44494 with the test
webapp, we got for that BZ.
Regards,
Rainer
Rainer Jung schrieb:
> OK, cancelled my appointment. More info:
>
> I backported all functional changes in o.a.tomcat.util.buf from tc6.x to
> tc5.5 and can't reproduce the problem any more. Those are very few
> changes. I'll narrow it down some more during the next hour. Stay tuned.
>
> Rainer
>
> Rainer Jung schrieb:
>> Filip Hanik - Dev Lists schrieb:
>>> Thanks Rainer, I will take a look at it tonight
>> Thank you!
>>
>> Last info chunk for today: in CoyoteAdapter.convertURI, before the
>> try/catch block that either creates or recycles the B2CConverter, the
>> ByteChunk bc coming from the decodedURI contains the correct URI. After
>> the recycle of the B2CConverter, the ByteChunk is empty and thus we end
>> up in a default redirect.
>>
>> Although we already read the correct URI from the request, the
>> B2CConverter associated with the request detroys the already read URI in
>> the recycle. I don't see the delta to 6.0. CoyoteAdapter seems fine,
>> maybe in ByteChunk?
>>
>> Regards,
>>
>> Rainer
>>
>>> Filip
>>>
>>> Rainer Jung wrote:
>>>> Remy Maucherat schrieb:
>>>>
>>>>> On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
>>>>>
>>>>>> If we revert the backport of
>>>>>>
>>>>>> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>>>>>>
>>>>>>
>>>>>> then the redirect loop is gone, and the usual content gets served, but
>>>>>> we know, that this change was needed to fix the "remaining garbage"
>>>>>> part
>>>>>> of 44494. So reverting it without any alternative is not really an
>>>>>> option.
>>>>>>
>>>>> Most likely, some backport is missing (many patches made up this input
>>>>> fixes). OTOH, I don't think this particular problem is that critical, so
>>>>> I would be in favor of dropping it if fixing the issue is complex.
>>>>>
>>>> Got some more info:
>>>>
>>>> 1) What happens inside the new ReadConvertor.recycle(): actually if I
>>>> print out, which "superfluous" bytes are eaten by the read() loop inside
>>>> ReadConvertor.recycle(), I can see, that it's all the bytes making up
>>>> the PATH in the URL. If I request http://myserver:8080/, recycle read
>>>> one character, namely "/", if I request http://myserver:8080/index.jsp,
>>>> then recycle reads all characters from "/index.jsp". In my understanding
>>>> of those patches, the time recycle in ReadConvertor gets called, those
>>>> should have alrady been read and only body bytes left over after request
>>>> processing should be eaten.
>>>>
>>>> 2) So I checked, when recycle() gets called, and I see, that during the
>>>> first few (here: 2) requests it doesn't get called at all (and those
>>>> work), and during the following broken requests, it gets called in the
>>>> following stack:
>>>>
>>>> at
>>>> org.apache.tomcat.util.buf.ReadConvertor.recycle(B2CConverter.java:222)
>>>> at
>>>> org.apache.tomcat.util.buf.B2CConverter.recycle(B2CConverter.java:64)
>>>> at
>>>> org.apache.catalina.connector.CoyoteAdapter.convertURI(CoyoteAdapter.java:475)
>>>>
>>>> at
>>>> org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:265)
>>>>
>>>> at
>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:172)
>>>>
>>>> at
>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>>>>
>>>> at
>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>>
>>>> at
>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>>
>>>> at
>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>>
>>>> at
>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>>
>>>> at java.lang.Thread.run(Thread.java:595)
>>>>
>>>> Didn't yet go further into it, but maybe something is wrong in
>>>> CoyoteAdapter.convertURI?
>>>>
>>>> Regards,
>>>>
>>>> Rainer
>>>>
>>>> P.S: I'll soon need to stop investigating this for today. If anyone can
>>>> take over that will be nice, because we really should have a working
>>>> 5.5.27 soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Posted by Rainer Jung <ra...@kippdata.de>.
OK, cancelled my appointment. More info:
I backported all functional changes in o.a.tomcat.util.buf from tc6.x to
tc5.5 and can't reproduce the problem any more. Those are very few
changes. I'll narrow it down some more during the next hour. Stay tuned.
Rainer
Rainer Jung schrieb:
> Filip Hanik - Dev Lists schrieb:
>> Thanks Rainer, I will take a look at it tonight
>
> Thank you!
>
> Last info chunk for today: in CoyoteAdapter.convertURI, before the
> try/catch block that either creates or recycles the B2CConverter, the
> ByteChunk bc coming from the decodedURI contains the correct URI. After
> the recycle of the B2CConverter, the ByteChunk is empty and thus we end
> up in a default redirect.
>
> Although we already read the correct URI from the request, the
> B2CConverter associated with the request detroys the already read URI in
> the recycle. I don't see the delta to 6.0. CoyoteAdapter seems fine,
> maybe in ByteChunk?
>
> Regards,
>
> Rainer
>
>> Filip
>>
>> Rainer Jung wrote:
>>> Remy Maucherat schrieb:
>>>
>>>> On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
>>>>
>>>>> If we revert the backport of
>>>>>
>>>>> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>>>>>
>>>>>
>>>>> then the redirect loop is gone, and the usual content gets served, but
>>>>> we know, that this change was needed to fix the "remaining garbage"
>>>>> part
>>>>> of 44494. So reverting it without any alternative is not really an
>>>>> option.
>>>>>
>>>> Most likely, some backport is missing (many patches made up this input
>>>> fixes). OTOH, I don't think this particular problem is that critical, so
>>>> I would be in favor of dropping it if fixing the issue is complex.
>>>>
>>> Got some more info:
>>>
>>> 1) What happens inside the new ReadConvertor.recycle(): actually if I
>>> print out, which "superfluous" bytes are eaten by the read() loop inside
>>> ReadConvertor.recycle(), I can see, that it's all the bytes making up
>>> the PATH in the URL. If I request http://myserver:8080/, recycle read
>>> one character, namely "/", if I request http://myserver:8080/index.jsp,
>>> then recycle reads all characters from "/index.jsp". In my understanding
>>> of those patches, the time recycle in ReadConvertor gets called, those
>>> should have alrady been read and only body bytes left over after request
>>> processing should be eaten.
>>>
>>> 2) So I checked, when recycle() gets called, and I see, that during the
>>> first few (here: 2) requests it doesn't get called at all (and those
>>> work), and during the following broken requests, it gets called in the
>>> following stack:
>>>
>>> at
>>> org.apache.tomcat.util.buf.ReadConvertor.recycle(B2CConverter.java:222)
>>> at
>>> org.apache.tomcat.util.buf.B2CConverter.recycle(B2CConverter.java:64)
>>> at
>>> org.apache.catalina.connector.CoyoteAdapter.convertURI(CoyoteAdapter.java:475)
>>>
>>> at
>>> org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:265)
>>>
>>> at
>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:172)
>>>
>>> at
>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>>>
>>> at
>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>
>>> at
>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>
>>> at
>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>
>>> at
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>
>>> at java.lang.Thread.run(Thread.java:595)
>>>
>>> Didn't yet go further into it, but maybe something is wrong in
>>> CoyoteAdapter.convertURI?
>>>
>>> Regards,
>>>
>>> Rainer
>>>
>>> P.S: I'll soon need to stop investigating this for today. If anyone can
>>> take over that will be nice, because we really should have a working
>>> 5.5.27 soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Posted by Rainer Jung <ra...@kippdata.de>.
Filip Hanik - Dev Lists schrieb:
> Thanks Rainer, I will take a look at it tonight
Thank you!
Last info chunk for today: in CoyoteAdapter.convertURI, before the
try/catch block that either creates or recycles the B2CConverter, the
ByteChunk bc coming from the decodedURI contains the correct URI. After
the recycle of the B2CConverter, the ByteChunk is empty and thus we end
up in a default redirect.
Although we already read the correct URI from the request, the
B2CConverter associated with the request detroys the already read URI in
the recycle. I don't see the delta to 6.0. CoyoteAdapter seems fine,
maybe in ByteChunk?
Regards,
Rainer
> Filip
>
> Rainer Jung wrote:
>> Remy Maucherat schrieb:
>>
>>> On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
>>>
>>>> If we revert the backport of
>>>>
>>>> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>>>>
>>>>
>>>> then the redirect loop is gone, and the usual content gets served, but
>>>> we know, that this change was needed to fix the "remaining garbage"
>>>> part
>>>> of 44494. So reverting it without any alternative is not really an
>>>> option.
>>>>
>>> Most likely, some backport is missing (many patches made up this input
>>> fixes). OTOH, I don't think this particular problem is that critical, so
>>> I would be in favor of dropping it if fixing the issue is complex.
>>>
>>
>> Got some more info:
>>
>> 1) What happens inside the new ReadConvertor.recycle(): actually if I
>> print out, which "superfluous" bytes are eaten by the read() loop inside
>> ReadConvertor.recycle(), I can see, that it's all the bytes making up
>> the PATH in the URL. If I request http://myserver:8080/, recycle read
>> one character, namely "/", if I request http://myserver:8080/index.jsp,
>> then recycle reads all characters from "/index.jsp". In my understanding
>> of those patches, the time recycle in ReadConvertor gets called, those
>> should have alrady been read and only body bytes left over after request
>> processing should be eaten.
>>
>> 2) So I checked, when recycle() gets called, and I see, that during the
>> first few (here: 2) requests it doesn't get called at all (and those
>> work), and during the following broken requests, it gets called in the
>> following stack:
>>
>> at
>> org.apache.tomcat.util.buf.ReadConvertor.recycle(B2CConverter.java:222)
>> at
>> org.apache.tomcat.util.buf.B2CConverter.recycle(B2CConverter.java:64)
>> at
>> org.apache.catalina.connector.CoyoteAdapter.convertURI(CoyoteAdapter.java:475)
>>
>> at
>> org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:265)
>>
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:172)
>>
>> at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>>
>> at
>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>
>> at
>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>
>> at
>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>
>> at
>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>
>> at java.lang.Thread.run(Thread.java:595)
>>
>> Didn't yet go further into it, but maybe something is wrong in
>> CoyoteAdapter.convertURI?
>>
>> Regards,
>>
>> Rainer
>>
>> P.S: I'll soon need to stop investigating this for today. If anyone can
>> take over that will be nice, because we really should have a working
>> 5.5.27 soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
Thanks Rainer, I will take a look at it tonight
Filip
Rainer Jung wrote:
> Remy Maucherat schrieb:
>
>> On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
>>
>>> If we revert the backport of
>>>
>>> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>>>
>>> then the redirect loop is gone, and the usual content gets served, but
>>> we know, that this change was needed to fix the "remaining garbage" part
>>> of 44494. So reverting it without any alternative is not really an option.
>>>
>> Most likely, some backport is missing (many patches made up this input
>> fixes). OTOH, I don't think this particular problem is that critical, so
>> I would be in favor of dropping it if fixing the issue is complex.
>>
>
> Got some more info:
>
> 1) What happens inside the new ReadConvertor.recycle(): actually if I
> print out, which "superfluous" bytes are eaten by the read() loop inside
> ReadConvertor.recycle(), I can see, that it's all the bytes making up
> the PATH in the URL. If I request http://myserver:8080/, recycle read
> one character, namely "/", if I request http://myserver:8080/index.jsp,
> then recycle reads all characters from "/index.jsp". In my understanding
> of those patches, the time recycle in ReadConvertor gets called, those
> should have alrady been read and only body bytes left over after request
> processing should be eaten.
>
> 2) So I checked, when recycle() gets called, and I see, that during the
> first few (here: 2) requests it doesn't get called at all (and those
> work), and during the following broken requests, it gets called in the
> following stack:
>
> at
> org.apache.tomcat.util.buf.ReadConvertor.recycle(B2CConverter.java:222)
> at
> org.apache.tomcat.util.buf.B2CConverter.recycle(B2CConverter.java:64)
> at
> org.apache.catalina.connector.CoyoteAdapter.convertURI(CoyoteAdapter.java:475)
> at
> org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:265)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:172)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
> at java.lang.Thread.run(Thread.java:595)
>
> Didn't yet go further into it, but maybe something is wrong in
> CoyoteAdapter.convertURI?
>
> Regards,
>
> Rainer
>
> P.S: I'll soon need to stop investigating this for today. If anyone can
> take over that will be nice, because we really should have a working
> 5.5.27 soon.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Posted by Rainer Jung <ra...@kippdata.de>.
Remy Maucherat schrieb:
> On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
>> If we revert the backport of
>>
>> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>>
>> then the redirect loop is gone, and the usual content gets served, but
>> we know, that this change was needed to fix the "remaining garbage" part
>> of 44494. So reverting it without any alternative is not really an option.
>
> Most likely, some backport is missing (many patches made up this input
> fixes). OTOH, I don't think this particular problem is that critical, so
> I would be in favor of dropping it if fixing the issue is complex.
Got some more info:
1) What happens inside the new ReadConvertor.recycle(): actually if I
print out, which "superfluous" bytes are eaten by the read() loop inside
ReadConvertor.recycle(), I can see, that it's all the bytes making up
the PATH in the URL. If I request http://myserver:8080/, recycle read
one character, namely "/", if I request http://myserver:8080/index.jsp,
then recycle reads all characters from "/index.jsp". In my understanding
of those patches, the time recycle in ReadConvertor gets called, those
should have alrady been read and only body bytes left over after request
processing should be eaten.
2) So I checked, when recycle() gets called, and I see, that during the
first few (here: 2) requests it doesn't get called at all (and those
work), and during the following broken requests, it gets called in the
following stack:
at
org.apache.tomcat.util.buf.ReadConvertor.recycle(B2CConverter.java:222)
at
org.apache.tomcat.util.buf.B2CConverter.recycle(B2CConverter.java:64)
at
org.apache.catalina.connector.CoyoteAdapter.convertURI(CoyoteAdapter.java:475)
at
org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:265)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:172)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:595)
Didn't yet go further into it, but maybe something is wrong in
CoyoteAdapter.convertURI?
Regards,
Rainer
P.S: I'll soon need to stop investigating this for today. If anyone can
take over that will be nice, because we really should have a working
5.5.27 soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: 5.5.27 blocker: URIEncoding UTF-8 broken for 5.5.trunk
Posted by Remy Maucherat <re...@apache.org>.
On Mon, 2008-08-25 at 17:16 +0200, Rainer Jung wrote:
> If we revert the backport of
>
> http://svn.eu.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/buf/B2CConverter.java?r1=642819&r2=647307&diff_format=h
>
> then the redirect loop is gone, and the usual content gets served, but
> we know, that this change was needed to fix the "remaining garbage" part
> of 44494. So reverting it without any alternative is not really an option.
Most likely, some backport is missing (many patches made up this input
fixes). OTOH, I don't think this particular problem is that critical, so
I would be in favor of dropping it if fixing the issue is complex.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org