You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Peter Kennard <pe...@livingwork.com> on 2007/03/11 03:40:58 UTC
BUG? - Mysterious "chunked" behavior
I have some real mysterious behavior, it seems the first chunk just doesn't
make it into the output. Doesn't matter how long or short it is.
Seems like a BUG unles I'm doing something wrong.
/* in my real tiny test servlet */
public void service( ServletRequest req,
ServletResponse res)
throws IOException, ServletException
{
java.io.OutputStream os = res.getOutputStream();
for(int i = 0; i < 5; ++i)
{
String txt = "Some Text " + i + "\n";
os.write(txt.getBytes());
os.flush();
}
}
/**** the results in a purely socket based client (ie exactly what is sent
back - all appropriate except the "Some Text 0" is plain missing !!! */
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked
Date: Sun, 11 Mar 2007 02:22:52 GMT
c
Some Text 1
c
Some Text 2
c
Some Text 3
c
Some Text 4
0
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Per,
Per Jonsson wrote:
> You have a little error, you are using ++i instead of i++ on:
>
> for(int i = 0; i < 5; ++i)
>
> The first iteration is a 1 because the increament is done before instead
> of after the looping.
I disagree. The loop's update expression is evaluated only after the
body, so ++i and i++ have the same effect. This is true in any language
that has unary increment operators and for-loops of this form.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFF9BRj9CaO5/Lv0PARAlWMAKCt/d0QOPHxXkauLSHZvKIpzoNN7gCff9rA
+/0SOGsTvVIkZ+oflI76jYE=
=j5YA
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Peter Kennard <pe...@livingwork.com>.
No problem :)
I found if you flush before any data is written, it will immediatly
commit the headers, and write a "terminal" chunk of length 0. Which
is a reasonable behavior.
In the case of a fat upload or page where one doesn't know the exact
length before starting and wants to avoid allocating a huge buffer,
one would want to use the chunked response.
PK
At 15:13 3/11/2007, you wrote:
>Hi!
>
>Yes ofcourse, you are right, I was to hasty.
>Does it work if you add an os.flush() before the loop?
>
>/Per Jonsson
>
>Peter Kennard skrev:
>>That is not an error, the last item in the (for(;;here)) is
>>executed after the loop code is executed - the "side effect" only
>>has effect within the for statement. (;;(side effect only visible
>>inside statment in here))
>>
>>ie: for(;;++i) ad for(;;i++) are equivalent
>>ie: for(;;val = ++i) ad for(;;val = i++) are *NOT* equivalent
>>
>>
>>At 05:36 3/11/2007, you wrote:
>>
>>>Hi!
>>>
>>>You have a little error, you are using ++i instead of i++ on:
>>>
>>> for(int i = 0; i < 5; ++i)
>>>
>>>The first iteration is a 1 because the increament is done before
>>>instead of after the looping.
>>
>>
>>
>>
>>---------------------------------------------------------------------
>>To start a new topic, e-mail: users@tomcat.apache.org
>>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>---------------------------------------------------------------------
>To start a new topic, e-mail: users@tomcat.apache.org
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Per Jonsson <pe...@laetus.se>.
Hi!
Yes ofcourse, you are right, I was to hasty.
Does it work if you add an os.flush() before the loop?
/Per Jonsson
Peter Kennard skrev:
> That is not an error, the last item in the (for(;;here)) is executed
> after the loop code is executed - the "side effect" only has effect
> within the for statement. (;;(side effect only visible inside statment
> in here))
>
> ie: for(;;++i) ad for(;;i++) are equivalent
>
> ie: for(;;val = ++i) ad for(;;val = i++) are *NOT* equivalent
>
>
> At 05:36 3/11/2007, you wrote:
>
>> Hi!
>>
>> You have a little error, you are using ++i instead of i++ on:
>>
>> for(int i = 0; i < 5; ++i)
>>
>> The first iteration is a 1 because the increament is done before
>> instead of after the looping.
>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Peter Kennard <pe...@livingwork.com>.
That is not an error, the last item in the (for(;;here)) is executed
after the loop code is executed - the "side effect" only has effect
within the for statement. (;;(side effect only visible inside
statment in here))
ie: for(;;++i) ad for(;;i++) are equivalent
ie: for(;;val = ++i) ad for(;;val = i++) are *NOT* equivalent
At 05:36 3/11/2007, you wrote:
>Hi!
>
>You have a little error, you are using ++i instead of i++ on:
>
> for(int i = 0; i < 5; ++i)
>
>The first iteration is a 1 because the increament is done before
>instead of after the looping.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Per Jonsson <pe...@laetus.se>.
Hi!
You have a little error, you are using ++i instead of i++ on:
for(int i = 0; i < 5; ++i)
The first iteration is a 1 because the increament is done before instead
of after the looping.
/per jonsson
Peter Kennard skrev:
> I have some real mysterious behavior, it seems the first chunk just
> doesn't
> make it into the output. Doesn't matter how long or short it is.
> Seems like a BUG unles I'm doing something wrong.
>
> /* in my real tiny test servlet */
>
> public void service( ServletRequest req,
> ServletResponse res)
> throws IOException, ServletException
> {
> java.io.OutputStream os = res.getOutputStream();
> for(int i = 0; i < 5; ++i)
> {
> String txt = "Some Text " + i + "\n";
> os.write(txt.getBytes());
> os.flush();
> }
> }
>
> /**** the results in a purely socket based client (ie exactly what is
> sent
> back - all appropriate except the "Some Text 0" is plain missing
> !!! */
>
> HTTP/1.1 200 OK
> Server: Apache-Coyote/1.1
> Transfer-Encoding: chunked
> Date: Sun, 11 Mar 2007 02:22:52 GMT
>
> c
> Some Text 1
>
> c
> Some Text 2
>
> c
> Some Text 3
>
> c
> Some Text 4
>
> 0
>
>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Peter Kennard <pe...@livingwork.com>.
Aaack - I really REALLY have to oppolgize. One of the people I
manange gave me the dumper client, and I went through it and the
problem is there :|
I'm really really sorry if I caused any flamage and wasted anyone's time.
I respect you people for hammering on me about it. I'll try to audit
things a bt more before posting in the future.
I stuck my neck out and am now under some pressure to prove the
tomcat Apache infrastructure is an appropriate choice for a big
project. I am still conviced it is appropriate.
Peter K.
At 22:29 3/11/2007, you wrote:
> > - Are the results I get from those 3 examples
> > "expected" ?
>
>I actually tried to reproduce your experience.
>
>wget, FireFox etc. - all these HTTP-clients don't have any problems with
>tomcat.
>
>
>What HTTP-client are you using? Is it self-written maybe?
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Sven Köhler <sk...@upb.de>.
> - Are the results I get from those 3 examples
> "expected" ?
I actually tried to reproduce your experience.
wget, FireFox etc. - all these HTTP-clients don't have any problems with
tomcat.
What HTTP-client are you using? Is it self-written maybe?
Re: BUG? - Mysterious "chunked" behavior
Posted by Peter Kennard <pe...@livingwork.com>.
>to the actual behavior of Tomcat. The 3 examples you have given here
>do work as expected.
On what version?
- I have a plain vanilla install of Tomcat 6.0 on a windows XP Pro machine
using the sun jdk (build 1.5.0_11-b03) I am connecting through an
HTTP1.1 connector with the default initiall configuration
via "localhost". No other servlets are deployed on this instance.
- Are the results I get from those 3 examples
"expected" ?
- If not, why do you think this might be happening?
The client is sending this minimal request:
POST /Test/ HTTP/1.1\n
Host: localhost\n
Transfer-Encoding: chunked\n
\n
... (I can read all the data uploaded just fine from the
servlet, except the read hangs and times out if I don't stop
before the "terminal" 0 chunk length chunk is encountered )
I'm really not trying to be a pain in the butt, just figure it out.
Peter K.
At 21:16 3/11/2007, you wrote:
>On 3/12/07, Peter Kennard <pe...@livingwork.com> wrote:
>>Unless dropping the first data filled chunk is defined as proper and
>>documented, and supported, behavior, I would consider this a bug.
>
>As I said in my previous post, so far, none of your claims correspond
>to the actual behavior of Tomcat. The 3 examples you have given here
>do work as expected.
>
>Rémy
>
>---------------------------------------------------------------------
>To start a new topic, e-mail: users@tomcat.apache.org
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Rémy Maucherat <re...@gmail.com>.
On 3/12/07, Peter Kennard <pe...@livingwork.com> wrote:
> Unless dropping the first data filled chunk is defined as proper and
> documented, and supported, behavior, I would consider this a bug.
As I said in my previous post, so far, none of your claims correspond
to the actual behavior of Tomcat. The 3 examples you have given here
do work as expected.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Peter Kennard <pe...@livingwork.com>.
>>Curious you mean by this? What is obvious?
>It's quite simple. You've made a number of claims in your two threads,
>every time supposedly verified by yourself, and for every one of them,
>I can tell you that the opposite is actually true. So there's really
>nothing to talk about ...
I have to oppolgise here. I thought I had tested the case of flush()
after writing no data and returning - pardon. It only returns a 0
length chunk if I flush once after writing some data. If I flush
once with no data it writes nothing and the client times out
expecting a chunk length (which I would consider another bug) I would
think it should ignore all zero length flushes, and wait to write the
terminal chunk when service() returns.
A complete simple test (which I actually ran just now) and it's
results are below - this is easily reproducable Hopefully this will
make the cut :)
Unless dropping the first data filled chunk is defined as proper and
documented, and supported, behavior, I would consider this a bug.
The servlet:
public void service( ServletRequest req,
ServletResponse res)
throws IOException
{
try
{
req.getInputStream();
java.io.OutputStream os = res.getOutputStream();
os.flush();
os.write("Line Of Text 0\n".getBytes());
os.flush();
os.write("Line Of Text 1\n".getBytes());
os.flush();
}
catch (Exception e)
{
System.out.println("servlet: exception! " + e);
}
}
The above when hit by client returns:
****
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked
Date: Sun, 11 Mar 2007 22:44:02 GMT
f
Line Of Text 1
0
********
The same servlet with:
os.flush();
os.write("Line Of Text 0\n".getBytes());
os.flush();
// os.write("Line Of Text 1\n".getBytes());
// os.flush();
When hit by client returns:
****
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked
Date: Sun, 11 Mar 2007 22:44:54 GMT
0
****
The same servlet again with:
os.flush();
// os.write("Line Of Text 0\n".getBytes());
// os.flush();
// os.write("Line Of Text 1\n".getBytes());
// os.flush();
When hit by client returns:
****
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked
Date: Sun, 11 Mar 2007 22:29:12 GMT
client: exception java.net.SocketTimeoutException: Read timed out
****
Peter K
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Rémy Maucherat <re...@gmail.com>.
On 3/11/07, Peter Kennard <pe...@livingwork.com> wrote:
> Curious you mean by this? What is obvious?
It's quite simple. You've made a number of claims in your two threads,
every time supposedly verified by yourself, and for every one of them,
I can tell you that the opposite is actually true. So there's really
nothing to talk about ...
For example for the latest one:
<claim>I found if you flush before any data is written, it will immediatly
commit the headers, and write a "terminal" chunk of length 0.</claim>
Actually, Tomcat will not write any chunk from the body on the socket,
and especially not the end chunk.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Peter Kennard <pe...@livingwork.com>.
>If you file a bug for this, or for the "issue" you describe in your
>other thread, I will immediately resolve them as invalid, obviously.
Curious you mean by this? What is obvious?
Tomcat 5.5 and 6 do make "chunked" replys to
HTTP1.1 requests that do not explicitly set
content-length if one flushes the buffer. I want
to know what the proper manner is to use this
feature and make sure it works from a servlet by
testing it. I'm trying to be nice and comform to
standards, proxy protocols, connectors etc.
Are you saying there is no proper manner ??
Is this a partially implemented "alpha" feature
intended to be completed in a future version?
PK
At 12:39 3/11/2007, you wrote:
>Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Rémy Maucherat <re...@gmail.com>.
On 3/11/07, Peter Kennard <pe...@livingwork.com> wrote:
> or bug report for it
If you file a bug for this, or for the "issue" you describe in your
other thread, I will immediately resolve them as invalid, obviously.
Rémy
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Peter Kennard <pe...@livingwork.com>.
>Have you checked the result with a hex editor or anything like that? Is
>it possible that txt.getBytes() in the first loop iteration is giving
>you some text with a CR in it? It so, your terminal could be overwriting
I am reading a socket directly in a test client. No data of any kind
comes in representing the first chunk flushed. Even if it was
"invisible" it would still need to be preceeded by the "hex
digits\r\n" for the chunk's length. (I originally spotted this
because I was parsing a binary upload and it was missing the first
chunk - the text version is an emailable reproducable illustration.
I can understand that this could be a specified behavior
Like: "to force output into chunked mode, flush a one byte (or more)
record as the first write, it will be discarded, and subsequent
chunks will be written in chuked mode"
But I have found no references to this behavior (or bug report for
it) anywhere.
PK
At 10:42 3/11/2007, you wrote:
>Have you checked the result with a hex editor or anything like that? Is
>it possible that txt.getBytes() in the first loop iteration is giving
>you some text with a CR in it? It so, your terminal could be overwriting
>the text with the next line of output, even though it's really there.
>Just a shot in the dark.
>
>Try piping wget through od and see what falls out.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: BUG? - Mysterious "chunked" behavior
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Peter,
Peter Kennard wrote:
> I have some real mysterious behavior, it seems the first chunk just doesn't
> make it into the output. Doesn't matter how long or short it is.
> Seems like a BUG unless I'm doing something wrong.
This code looks fine, except you didn't provide all of the code (where
does txt come from)?
> public void service( ServletRequest req,
> ServletResponse res)
> throws IOException, ServletException
> {
> java.io.OutputStream os = res.getOutputStream();
> for(int i = 0; i < 5; ++i)
> {
> String txt = "Some Text " + i + "\n";
> os.write(txt.getBytes());
> os.flush();
> }
> }
>
> /**** the results in a purely socket based client (ie exactly what is sent
> back - all appropriate except the "Some Text 0" is plain missing
> !!! */
Have you checked the result with a hex editor or anything like that? Is
it possible that txt.getBytes() in the first loop iteration is giving
you some text with a CR in it? It so, your terminal could be overwriting
the text with the next line of output, even though it's really there.
Just a shot in the dark.
Try piping wget through od and see what falls out.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFF9BVr9CaO5/Lv0PARAjTkAJ9XYVRmi5tEj28dpiOoFeoTFf67IQCgiGgV
aj3bDsRs0jHTo3wdXeRp0DU=
=ydBR
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org