You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Sebastiaan van Erk <se...@sebster.com> on 2007/04/26 14:18:58 UTC
comet: missing initial READ event?
Hi,
The CometProcessor seems to sometimes miss the initial READ event.
THE OBSERVED BEHAVIOR:
The client sends a request to the server. This request (a POST request)
is sent in the initial TCP packet. The initial packet also contains some
data (a boundary, Content-Disposition: form-data header, etc). The
server gets a BEGIN request in which I do some setup; there is already
data available for a READ event, but I don't get one. Meanwhile the
client sends 10 more bytes of data to the server (which I see passing by
in a TCP packet immediately after the request packet). However, I still
get no READ event, instead I get a ERROR/TIMEOUT event a bunch later. If
the client, instead of doing nothing, waits a couple of seconds and then
sends some data, it does get a READ event server side and all the data
is read (the boundary, the Content-Disposition header, etc).
Note this happens relatively infrequently, most of the time I do get the
initial READ event.
The http://tomcat.apache.org/tomcat-6.0-doc/aio.html documentation clear
states that I am not allowed to read from the input stream outside of
the READ event; thus I'm not allowed to read in the BEGIN event. So the
behavior I expect would be that I immediately get a READ event after the
BEGIN event since there is data available.
I figure this is a bug?
Regards,
Sebastiaan
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: comet: missing initial READ event?
Posted by Sebastiaan van Erk <se...@sebster.com>.
Rémy Maucherat wrote:
> On 4/26/07, Sebastiaan van Erk <se...@sebster.com> wrote:
>> Note that there really IS a bug here:
>> request.getInputStream().available() returns 0 in the BEGIN event, so I
>> cannot test if the data is already available there. However, the API
>> seems cleaner to me anyway if you only allow reads within the READ
>> event, as the documentation suggests it should be.
>
> I am fully aware of all of what you have written, and maybe
> available() will become more accurate in the future.
>
> However, I think you are able to deal with it for now, since you know
> if there's going to be data sent along with the request header or not.
>
> Rémy
I find this suggestion a terrible workaround and the use of the word
"maybe" a bit disturbing.
First of all, I do not know when the client is going to send the data.
If it sends it immediately then sure, a read in the BEGIN event will be
fine, otherwise it will block. The whole point of using Comet is to not
have your http threads block. Second of all, how do I know that the
client is going to send any data at all? Third of all, assuming the
client does NOT send data, the CometProcessor will be blocked in the
event() method trying to read data. Depending on the synchronization and
how Tomcat works internally, this is a recipe for deadlock. Will the
event method ever be called concurrently from within Tomcat? If not,
I'll never get an ERROR event, causing the thread to block forever.
I generally don't like to depend on client behavior for the proper
functioning of the server.
It seems to me that this really is a bug, and the proper fix still seems
a call to the READ event. It makes more sense than reading in the BEGIN
event.
If you want I can file a bug report in bugzilla.
Regards,
Sebastiaan
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: comet: missing initial READ event?
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/26/07, Sebastiaan van Erk <se...@sebster.com> wrote:
> Note that there really IS a bug here:
> request.getInputStream().available() returns 0 in the BEGIN event, so I
> cannot test if the data is already available there. However, the API
> seems cleaner to me anyway if you only allow reads within the READ
> event, as the documentation suggests it should be.
I am fully aware of all of what you have written, and maybe
available() will become more accurate in the future.
However, I think you are able to deal with it for now, since you know
if there's going to be data sent along with the request header or not.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: comet: missing initial READ event?
Posted by Sebastiaan van Erk <se...@sebster.com>.
Hi,
I found the cause of the missing READ event, and seems to me that it is
not too difficult to fix.
The problem is as follows. When Tomcat receives a request, the
Http11NioProcessor reads the headers. It does this by filling its
internal buffer using a read call on the socket. However, if there is
already more data available than just the headers, e.g., some request
body data, this is also read into the internal buffer. If the total data
is less than the size of the internal buffer, this means that there is
no more data available on the real socket channel, and after finishing
the BEGIN event there will not be another READ event until new data
arrives (and it may never arrive) on the socket channel to wake up the
Poller.
An easy way to reproduce this consistently is to have a client do a
request with a few bytes of data in the request body and placing a
breakpoint on line 839 of Http11NioProcessor so that all client data
arrives before Tomcat starts processing the headers.
An way to fix this is to check if there is request data remaining after
the parsing of the headers and if so, calling the event method of the
processor with a READ event directly after having called the BEGIN event.
Note that there really IS a bug here:
request.getInputStream().available() returns 0 in the BEGIN event, so I
cannot test if the data is already available there. However, the API
seems cleaner to me anyway if you only allow reads within the READ
event, as the documentation suggests it should be.
Regards,
Sebastiaan
Sebastiaan van Erk wrote:
> Rémy Maucherat wrote:
>> On 4/26/07, Sebastiaan van Erk <se...@sebster.com> wrote:
>>> The http://tomcat.apache.org/tomcat-6.0-doc/aio.html documentation
>>> clear
>>> states that I am not allowed to read from the input stream outside of
>>> the READ event; thus I'm not allowed to read in the BEGIN event. So the
>>> behavior I expect would be that I immediately get a READ event after
>>> the
>>> BEGIN event since there is data available.
>>>
>>> I figure this is a bug?
>>
>> The API is designed mostly for biderectional transfer. If somehow you
>> don't want to do that, then there's a lot less value in it (a thread
>> isn't that expensive, if all you want is wait a little bit before
>> generating a response).
>>
>> If there's data available in the begin event (if the requests you use
>> send a request body), you should read it and if this "bug" is fixed,
>> it won't make a difference for you.
>>
>> Rémy
> I'm sorry but I don't understand your answer.
>
> First of all, I use bidirectional communication: the client says A and
> the server says B, the client says C, etc.. However, since the server
> does not get a READ event for the initial A, the server just sits
> there and does not respond at all. It needs a READ event to be able to
> read from the input stream.
>
> Secondly, I'm trying to understand the API. Either it IS or it IS NOT
> allowed to read from the input stream in the BEGIN event. If I rely on
> a certain behavior while reading in the BEGIN event now, it might very
> well break in the future because I might be relying on the side effect
> of an implementation detail. The answer seems to be to me a simple yes
> or no; the documentation implies no.
>
> Finally, I expect to get a READ event if there is data available, or
> so the API seems to suggest. Then it seems to me that it is a bug if I
> don't. To be helpful I am reporting it, so that if it is indeed a bug,
> it may be fixed and the entire Tomcat community can profit from the
> bug fix. Even a simple acknowledgement that it is a bug without a fix
> is fine too, then I will look into the source myself and see if I can
> locate the problem and suggest a patch/fix.
>
> Regards,
> Sebastiaan
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: comet: missing initial READ event?
Posted by Sebastiaan van Erk <se...@sebster.com>.
Rémy Maucherat wrote:
> On 4/26/07, Sebastiaan van Erk <se...@sebster.com> wrote:
>> The http://tomcat.apache.org/tomcat-6.0-doc/aio.html documentation clear
>> states that I am not allowed to read from the input stream outside of
>> the READ event; thus I'm not allowed to read in the BEGIN event. So the
>> behavior I expect would be that I immediately get a READ event after the
>> BEGIN event since there is data available.
>>
>> I figure this is a bug?
>
> The API is designed mostly for biderectional transfer. If somehow you
> don't want to do that, then there's a lot less value in it (a thread
> isn't that expensive, if all you want is wait a little bit before
> generating a response).
>
> If there's data available in the begin event (if the requests you use
> send a request body), you should read it and if this "bug" is fixed,
> it won't make a difference for you.
>
> Rémy
I'm sorry but I don't understand your answer.
First of all, I use bidirectional communication: the client says A and
the server says B, the client says C, etc.. However, since the server
does not get a READ event for the initial A, the server just sits there
and does not respond at all. It needs a READ event to be able to read
from the input stream.
Secondly, I'm trying to understand the API. Either it IS or it IS NOT
allowed to read from the input stream in the BEGIN event. If I rely on a
certain behavior while reading in the BEGIN event now, it might very
well break in the future because I might be relying on the side effect
of an implementation detail. The answer seems to be to me a simple yes
or no; the documentation implies no.
Finally, I expect to get a READ event if there is data available, or so
the API seems to suggest. Then it seems to me that it is a bug if I
don't. To be helpful I am reporting it, so that if it is indeed a bug,
it may be fixed and the entire Tomcat community can profit from the bug
fix. Even a simple acknowledgement that it is a bug without a fix is
fine too, then I will look into the source myself and see if I can
locate the problem and suggest a patch/fix.
Regards,
Sebastiaan
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: comet: missing initial READ event?
Posted by Rémy Maucherat <re...@gmail.com>.
On 4/26/07, Sebastiaan van Erk <se...@sebster.com> wrote:
> The http://tomcat.apache.org/tomcat-6.0-doc/aio.html documentation clear
> states that I am not allowed to read from the input stream outside of
> the READ event; thus I'm not allowed to read in the BEGIN event. So the
> behavior I expect would be that I immediately get a READ event after the
> BEGIN event since there is data available.
>
> I figure this is a bug?
The API is designed mostly for biderectional transfer. If somehow you
don't want to do that, then there's a lot less value in it (a thread
isn't that expensive, if all you want is wait a little bit before
generating a response).
If there's data available in the begin event (if the requests you use
send a request body), you should read it and if this "bug" is fixed,
it won't make a difference for you.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: comet: missing initial READ event?
Posted by Sebastiaan van Erk <se...@sebster.com>.
Hi,
Thanks for the answer.
The problem is that available() returns 0 and I cannot be sure that I
can do a read without blocking. I would prefer the internals of Tomcat
handling this special case and calling the event method with a READ
event on the Comet processor if there is already read data in the input
buffer left after headers. This would be in line with the API. If
available returns a positive number in BEGIN that would be fine as well,
as long as I can be sure that I will never miss any data.
Another quick related question: is it possible for the event method to
be called concurrently or does Tomcat prevent this? That is, do I need
to synchronized access myself, or can I be sure that only one http
thread is in the event method at a time?
Regards,
Sebastiaan
Filip Hanik - Dev Lists wrote:
> Yes, this is an interesting use case, I usually issue a read on the
> BEGIN event.
> These events are IO related, ie, triggered by IO events and that is
> what they are. Since the data already has been read in with the
> request headers,
> there will be no following IO event.
> You could issue a read() in the BEGIN event, there is absolutely
> nothing wrong with doing that, as it is the same as a servlet doing a
> read during the service method.
>
> Filip
>
> Sebastiaan van Erk wrote:
>> Hi,
>>
>> The CometProcessor seems to sometimes miss the initial READ event.
>>
>> THE OBSERVED BEHAVIOR:
>> The client sends a request to the server. This request (a POST
>> request) is sent in the initial TCP packet. The initial packet also
>> contains some data (a boundary, Content-Disposition: form-data
>> header, etc). The server gets a BEGIN request in which I do some
>> setup; there is already data available for a READ event, but I don't
>> get one. Meanwhile the client sends 10 more bytes of data to the
>> server (which I see passing by in a TCP packet immediately after the
>> request packet). However, I still get no READ event, instead I get a
>> ERROR/TIMEOUT event a bunch later. If the client, instead of doing
>> nothing, waits a couple of seconds and then sends some data, it does
>> get a READ event server side and all the data is read (the boundary,
>> the Content-Disposition header, etc).
>>
>> Note this happens relatively infrequently, most of the time I do get
>> the initial READ event.
>>
>> The http://tomcat.apache.org/tomcat-6.0-doc/aio.html documentation
>> clear states that I am not allowed to read from the input stream
>> outside of the READ event; thus I'm not allowed to read in the BEGIN
>> event. So the behavior I expect would be that I immediately get a
>> READ event after the BEGIN event since there is data available.
>>
>> I figure this is a bug?
>>
>> Regards,
>> Sebastiaan
>>
>> ---------------------------------------------------------------------
>> To start a new topic, e-mail: users@tomcat.apache.org
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: comet: missing initial READ event?
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
Yes, this is an interesting use case, I usually issue a read on the
BEGIN event.
These events are IO related, ie, triggered by IO events and that is what
they are. Since the data already has been read in with the request headers,
there will be no following IO event.
You could issue a read() in the BEGIN event, there is absolutely nothing
wrong with doing that, as it is the same as a servlet doing a read
during the service method.
Filip
Sebastiaan van Erk wrote:
> Hi,
>
> The CometProcessor seems to sometimes miss the initial READ event.
>
> THE OBSERVED BEHAVIOR:
> The client sends a request to the server. This request (a POST
> request) is sent in the initial TCP packet. The initial packet also
> contains some data (a boundary, Content-Disposition: form-data header,
> etc). The server gets a BEGIN request in which I do some setup; there
> is already data available for a READ event, but I don't get one.
> Meanwhile the client sends 10 more bytes of data to the server (which
> I see passing by in a TCP packet immediately after the request
> packet). However, I still get no READ event, instead I get a
> ERROR/TIMEOUT event a bunch later. If the client, instead of doing
> nothing, waits a couple of seconds and then sends some data, it does
> get a READ event server side and all the data is read (the boundary,
> the Content-Disposition header, etc).
>
> Note this happens relatively infrequently, most of the time I do get
> the initial READ event.
>
> The http://tomcat.apache.org/tomcat-6.0-doc/aio.html documentation
> clear states that I am not allowed to read from the input stream
> outside of the READ event; thus I'm not allowed to read in the BEGIN
> event. So the behavior I expect would be that I immediately get a READ
> event after the BEGIN event since there is data available.
>
> I figure this is a bug?
>
> Regards,
> Sebastiaan
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org