You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Chris Pettitt <cp...@gmail.com> on 2007/12/12 17:46:36 UTC
Comet processor blocks if chunk includes final CRLF
Hello,
I'm using Tomcat 6.0.14 and its CometProcessor to create a long
running connection between an HTTP client and my server. The client
posts requests to the server using chunked transfer encoding (each
chunk is a complete XML request processed by the servlet).
I'm using a servlet that is heavily derived from the
example[1] on Tomcat's website. I've found that this implementation
blocks the worker thread - which I would not expect - if the client
sends a complete chunk and then wait some time before sending the next
chunk.
I'm using the W3C specification[2] of a chunk:
chunk-size CRLF
chunk-data CRLF
Surprisingly, if I send a chunk that does not include the last CRLF,
then the CometProcessor works correctly and does not block.
My analysis of the blocking condition follows:
ChunkedInputFilter.doRead() reads everything in the chunk except the
last CRLF. It defers reading of the last CRLF until the next read()
call by setting needCRLFParse to true at line 157. Calling
ChunkedInputFilter.available() before receiving new data will
therefore return 2, because the CRLF at the end of the chunk have not
yet been read.
The implementation of
org.apache.catalina.connector.InputBuffer.available() always delegates
to coyoteRequest.getAvailable() (in my environment, state is not
BYTE_STATE or CHAR_STATE during this call).
// org.apache.catalina.connector.InputBuffer, ~ line 260
public int available() {
int available = 0;
if (state == BYTE_STATE) {
available = bb.getLength();
} else if (state == CHAR_STATE) {
available = cb.getLength();
}
if (available == 0) {
coyoteRequest.action(ActionCode.ACTION_AVAILABLE, null);
available = (coyoteRequest.getAvailable() > 0) ? 1 : 0; //
<--- delegates here
}
return available;
}
The value that is returned by coyoteRequest.getAvailable() is set by
Http11NioProcessor (see processing of ActionCode.ACTION_AVAILABLE at
line 1218) to the value returned by its inputBuffer.available(). This
method delegates to the ChunkedInputFilter, which returns 2,
indicating two available bytes (CR, LF). Ultimately the
InputBuffer.available() method above returns 1, because
coyoteRequest.getAvailable() returned a non-zero value.
Because in.available() in the attached servlet is non-zero, the
servlet calls read() again. This time the call blocks, because when it
calls read ChunkedInputFilter.doRead() finishes reading the CRLF it
has in its buffer, it calls parseChunkHeader which makes a blocking
call to readBytes().
Is this misuse of CometProcessor? Is there best practice guidance
for managing long running connections with multiple requests with
Tomcat?
Thanks,
Chris
[1]: http://tomcat.apache.org/tomcat-6.0-doc/aio.html
[2]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Comet processor blocks if chunk includes final CRLF
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
fyi, we had to rework the patch, as there is a nasty data corruption bug
with the old way of doing things.
the new patch is at
http://people.apache.org/~fhanik/patches/fix-bz11117-alt-1.patch
and is up for voting
Filip
Filip Hanik - Dev Lists wrote:
> the patch has been applied and will go into 6.0.16
>
> Filip
>
> Chris Pettitt wrote:
>> Filip,
>>
>> Do you know which release this patch will go into? Is there a bug
>> number I can watch?
>>
>> Thanks,
>> Chris
>>
>> On Dec 19, 2007 9:13 AM, Chris Pettitt <cp...@gmail.com> wrote:
>>
>>> Filip,
>>>
>>> Your patch has fixed the chunk parsing and the processor is no
>>> longer blocking.
>>>
>>> Sometimes newly connected clients get disconnected immediately, but I
>>> think that is related to this bug:
>>> http://issues.apache.org/bugzilla/show_bug.cgi?id=43846.
>>>
>>> Thanks,
>>> Chris
>>>
>>
>> ---------------------------------------------------------------------
>> To start a new topic, e-mail: users@tomcat.apache.org
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>>
>>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Comet processor blocks if chunk includes final CRLF
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
the patch has been applied and will go into 6.0.16
Filip
Chris Pettitt wrote:
> Filip,
>
> Do you know which release this patch will go into? Is there a bug
> number I can watch?
>
> Thanks,
> Chris
>
> On Dec 19, 2007 9:13 AM, Chris Pettitt <cp...@gmail.com> wrote:
>
>> Filip,
>>
>> Your patch has fixed the chunk parsing and the processor is no longer blocking.
>>
>> Sometimes newly connected clients get disconnected immediately, but I
>> think that is related to this bug:
>> http://issues.apache.org/bugzilla/show_bug.cgi?id=43846.
>>
>> Thanks,
>> Chris
>>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Comet processor blocks if chunk includes final CRLF
Posted by Chris Pettitt <cp...@gmail.com>.
Filip,
Do you know which release this patch will go into? Is there a bug
number I can watch?
Thanks,
Chris
On Dec 19, 2007 9:13 AM, Chris Pettitt <cp...@gmail.com> wrote:
> Filip,
>
> Your patch has fixed the chunk parsing and the processor is no longer blocking.
>
> Sometimes newly connected clients get disconnected immediately, but I
> think that is related to this bug:
> http://issues.apache.org/bugzilla/show_bug.cgi?id=43846.
>
> Thanks,
> Chris
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Comet processor blocks if chunk includes final CRLF
Posted by Chris Pettitt <cp...@gmail.com>.
Filip,
Your patch has fixed the chunk parsing and the processor is no longer blocking.
Sometimes newly connected clients get disconnected immediately, but I
think that is related to this bug:
http://issues.apache.org/bugzilla/show_bug.cgi?id=43846.
Thanks,
Chris
On Dec 14, 2007 11:41 AM, Filip Hanik - Dev Lists <de...@hanik.com> wrote:
> I've proposed a patch for this behavior
> http://svn.apache.org/viewvc?view=rev&revision=604274
> http://svn.apache.org/viewvc/tomcat/tc6.0.x/trunk/STATUS.txt?r1=604274&r2=604273&pathrev=604274
>
> Filip
>
>
> Filip Hanik - Dev Lists wrote:
> > hi Chris, Paul Dumais just posted the same issue. I've been bit by it
> > too, I'm gonna review the spec again, and see if and how we can make
> > adjustments to the code
> >
> > Filip
> >
> > Chris Pettitt wrote:
> >> Hello,
> >>
> >> I'm using Tomcat 6.0.14 and its CometProcessor to create a long
> >> running connection between an HTTP client and my server. The client
> >> posts requests to the server using chunked transfer encoding (each
> >> chunk is a complete XML request processed by the servlet).
> >>
> >> I'm using a servlet that is heavily derived from the
> >> example[1] on Tomcat's website. I've found that this implementation
> >> blocks the worker thread - which I would not expect - if the client
> >> sends a complete chunk and then wait some time before sending the next
> >> chunk.
> >>
> >> I'm using the W3C specification[2] of a chunk:
> >>
> >> chunk-size CRLF
> >> chunk-data CRLF
> >>
> >> Surprisingly, if I send a chunk that does not include the last CRLF,
> >> then the CometProcessor works correctly and does not block.
> >>
> >> My analysis of the blocking condition follows:
> >>
> >> ChunkedInputFilter.doRead() reads everything in the chunk except the
> >> last CRLF. It defers reading of the last CRLF until the next read()
> >> call by setting needCRLFParse to true at line 157. Calling
> >> ChunkedInputFilter.available() before receiving new data will
> >> therefore return 2, because the CRLF at the end of the chunk have not
> >> yet been read.
> >>
> >> The implementation of
> >> org.apache.catalina.connector.InputBuffer.available() always delegates
> >> to coyoteRequest.getAvailable() (in my environment, state is not
> >> BYTE_STATE or CHAR_STATE during this call).
> >>
> >> // org.apache.catalina.connector.InputBuffer, ~ line 260
> >> public int available() {
> >> int available = 0;
> >> if (state == BYTE_STATE) {
> >> available = bb.getLength();
> >> } else if (state == CHAR_STATE) {
> >> available = cb.getLength();
> >> }
> >> if (available == 0) {
> >> coyoteRequest.action(ActionCode.ACTION_AVAILABLE, null);
> >> available = (coyoteRequest.getAvailable() > 0) ? 1 : 0; //
> >> <--- delegates here
> >> }
> >> return available;
> >> }
> >>
> >> The value that is returned by coyoteRequest.getAvailable() is set by
> >> Http11NioProcessor (see processing of ActionCode.ACTION_AVAILABLE at
> >> line 1218) to the value returned by its inputBuffer.available(). This
> >> method delegates to the ChunkedInputFilter, which returns 2,
> >> indicating two available bytes (CR, LF). Ultimately the
> >> InputBuffer.available() method above returns 1, because
> >> coyoteRequest.getAvailable() returned a non-zero value.
> >>
> >> Because in.available() in the attached servlet is non-zero, the
> >> servlet calls read() again. This time the call blocks, because when it
> >> calls read ChunkedInputFilter.doRead() finishes reading the CRLF it
> >> has in its buffer, it calls parseChunkHeader which makes a blocking
> >> call to readBytes().
> >>
> >> Is this misuse of CometProcessor? Is there best practice guidance
> >> for managing long running connections with multiple requests with
> >> Tomcat?
> >>
> >> Thanks,
> >> Chris
> >>
> >> [1]: http://tomcat.apache.org/tomcat-6.0-doc/aio.html
> >> [2]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
> >>
> >> ---------------------------------------------------------------------
> >> To start a new topic, e-mail: users@tomcat.apache.org
> >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: users-help@tomcat.apache.org
> >>
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To start a new topic, e-mail: users@tomcat.apache.org
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Comet processor blocks if chunk includes final CRLF
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
I've proposed a patch for this behavior
http://svn.apache.org/viewvc?view=rev&revision=604274
http://svn.apache.org/viewvc/tomcat/tc6.0.x/trunk/STATUS.txt?r1=604274&r2=604273&pathrev=604274
Filip
Filip Hanik - Dev Lists wrote:
> hi Chris, Paul Dumais just posted the same issue. I've been bit by it
> too, I'm gonna review the spec again, and see if and how we can make
> adjustments to the code
>
> Filip
>
> Chris Pettitt wrote:
>> Hello,
>>
>> I'm using Tomcat 6.0.14 and its CometProcessor to create a long
>> running connection between an HTTP client and my server. The client
>> posts requests to the server using chunked transfer encoding (each
>> chunk is a complete XML request processed by the servlet).
>>
>> I'm using a servlet that is heavily derived from the
>> example[1] on Tomcat's website. I've found that this implementation
>> blocks the worker thread - which I would not expect - if the client
>> sends a complete chunk and then wait some time before sending the next
>> chunk.
>>
>> I'm using the W3C specification[2] of a chunk:
>>
>> chunk-size CRLF
>> chunk-data CRLF
>>
>> Surprisingly, if I send a chunk that does not include the last CRLF,
>> then the CometProcessor works correctly and does not block.
>>
>> My analysis of the blocking condition follows:
>>
>> ChunkedInputFilter.doRead() reads everything in the chunk except the
>> last CRLF. It defers reading of the last CRLF until the next read()
>> call by setting needCRLFParse to true at line 157. Calling
>> ChunkedInputFilter.available() before receiving new data will
>> therefore return 2, because the CRLF at the end of the chunk have not
>> yet been read.
>>
>> The implementation of
>> org.apache.catalina.connector.InputBuffer.available() always delegates
>> to coyoteRequest.getAvailable() (in my environment, state is not
>> BYTE_STATE or CHAR_STATE during this call).
>>
>> // org.apache.catalina.connector.InputBuffer, ~ line 260
>> public int available() {
>> int available = 0;
>> if (state == BYTE_STATE) {
>> available = bb.getLength();
>> } else if (state == CHAR_STATE) {
>> available = cb.getLength();
>> }
>> if (available == 0) {
>> coyoteRequest.action(ActionCode.ACTION_AVAILABLE, null);
>> available = (coyoteRequest.getAvailable() > 0) ? 1 : 0; //
>> <--- delegates here
>> }
>> return available;
>> }
>>
>> The value that is returned by coyoteRequest.getAvailable() is set by
>> Http11NioProcessor (see processing of ActionCode.ACTION_AVAILABLE at
>> line 1218) to the value returned by its inputBuffer.available(). This
>> method delegates to the ChunkedInputFilter, which returns 2,
>> indicating two available bytes (CR, LF). Ultimately the
>> InputBuffer.available() method above returns 1, because
>> coyoteRequest.getAvailable() returned a non-zero value.
>>
>> Because in.available() in the attached servlet is non-zero, the
>> servlet calls read() again. This time the call blocks, because when it
>> calls read ChunkedInputFilter.doRead() finishes reading the CRLF it
>> has in its buffer, it calls parseChunkHeader which makes a blocking
>> call to readBytes().
>>
>> Is this misuse of CometProcessor? Is there best practice guidance
>> for managing long running connections with multiple requests with
>> Tomcat?
>>
>> Thanks,
>> Chris
>>
>> [1]: http://tomcat.apache.org/tomcat-6.0-doc/aio.html
>> [2]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
>>
>> ---------------------------------------------------------------------
>> To start a new topic, e-mail: users@tomcat.apache.org
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Comet processor blocks if chunk includes final CRLF
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
hi Chris, Paul Dumais just posted the same issue. I've been bit by it
too, I'm gonna review the spec again, and see if and how we can make
adjustments to the code
Filip
Chris Pettitt wrote:
> Hello,
>
> I'm using Tomcat 6.0.14 and its CometProcessor to create a long
> running connection between an HTTP client and my server. The client
> posts requests to the server using chunked transfer encoding (each
> chunk is a complete XML request processed by the servlet).
>
> I'm using a servlet that is heavily derived from the
> example[1] on Tomcat's website. I've found that this implementation
> blocks the worker thread - which I would not expect - if the client
> sends a complete chunk and then wait some time before sending the next
> chunk.
>
> I'm using the W3C specification[2] of a chunk:
>
> chunk-size CRLF
> chunk-data CRLF
>
> Surprisingly, if I send a chunk that does not include the last CRLF,
> then the CometProcessor works correctly and does not block.
>
> My analysis of the blocking condition follows:
>
> ChunkedInputFilter.doRead() reads everything in the chunk except the
> last CRLF. It defers reading of the last CRLF until the next read()
> call by setting needCRLFParse to true at line 157. Calling
> ChunkedInputFilter.available() before receiving new data will
> therefore return 2, because the CRLF at the end of the chunk have not
> yet been read.
>
> The implementation of
> org.apache.catalina.connector.InputBuffer.available() always delegates
> to coyoteRequest.getAvailable() (in my environment, state is not
> BYTE_STATE or CHAR_STATE during this call).
>
> // org.apache.catalina.connector.InputBuffer, ~ line 260
> public int available() {
> int available = 0;
> if (state == BYTE_STATE) {
> available = bb.getLength();
> } else if (state == CHAR_STATE) {
> available = cb.getLength();
> }
> if (available == 0) {
> coyoteRequest.action(ActionCode.ACTION_AVAILABLE, null);
> available = (coyoteRequest.getAvailable() > 0) ? 1 : 0; //
> <--- delegates here
> }
> return available;
> }
>
> The value that is returned by coyoteRequest.getAvailable() is set by
> Http11NioProcessor (see processing of ActionCode.ACTION_AVAILABLE at
> line 1218) to the value returned by its inputBuffer.available(). This
> method delegates to the ChunkedInputFilter, which returns 2,
> indicating two available bytes (CR, LF). Ultimately the
> InputBuffer.available() method above returns 1, because
> coyoteRequest.getAvailable() returned a non-zero value.
>
> Because in.available() in the attached servlet is non-zero, the
> servlet calls read() again. This time the call blocks, because when it
> calls read ChunkedInputFilter.doRead() finishes reading the CRLF it
> has in its buffer, it calls parseChunkHeader which makes a blocking
> call to readBytes().
>
> Is this misuse of CometProcessor? Is there best practice guidance
> for managing long running connections with multiple requests with
> Tomcat?
>
> Thanks,
> Chris
>
> [1]: http://tomcat.apache.org/tomcat-6.0-doc/aio.html
> [2]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org