You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by rapponcape <ra...@gmail.com> on 2011/09/29 18:28:19 UTC

Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Tomcat version 7.0.6 listening on local machine 10.3.4.7 and accepting
incoming POST connections from 4 remote machines, 10.12.5.10[2-5].
Configured in standalone mode, using HTTP connector.

Remote machines are sending constant stream of POST messages...~100 kbit/sec.

All POST traffic is accepted and acknowledged, but, RESET packets
cause remotes machines to re-establish new connections, causing errors
on sender's side and slowing down rate of transfer.

Traffic headers show local machine is sending "Connection: close" on
post acknowledgement messages.  Despite fact that remote machines'
POST messages contain "Connection: Keep-Alive" header.

======================
post traffic header ...
======================

POST /myapp HTTP/1.0
HOST: 10.3.4.7:8080
Connection: Keep-Alive
Authorization: Basic Y2VybmVyOnlpbiZ5YW5n
Content-type: multipart/form-data; boundary=----------1803119234
Content-Length: 59970

------------1803119234
Content-Disposition: form-data; name="xml"
<?xml version="1.0" encoding="ISO-
---------------------------------------------------------------------------------------------------------------------------------------

======================
response header ...
======================

------------1803119234--

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Date: Fri, 23 Sep 2011 18:27:25 GMT
Connection: close

======================

># netstat -anp | grep 8080

tcp        0      0 :::8081                     :::*
    LISTEN      23107/java
tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.103:39809
FIN_WAIT2   -
tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.104:33033
FIN_WAIT2   -
tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.102:57077
FIN_WAIT2   -
tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.105:39624
FIN_WAIT2   -

I've tried various connector configurations and protocols; HTTP, NIO,
connectionTimeout, etc.

Here's what my connector config looks like currently:

 <Connector port=8080 protocol="org.apache.coyote.http11.Http11Protocol"
              connectionTimeout="30000"
              connectionLinger="100"
              keepAliveTimeout="3000"
              maxKeepAliveRequests="2000"
              disableUploadTimeout="true"
              maxThreads="500"
              minSpareThreads="150"
              maxSpareThreads="300"
              acceptCount="200"
              socket.soKeepAlive="yes"
              enableLookups="false" />

===========================================

Why fin_wait2 state?  I would expect to see ESTABLISHED connections
and and Keep-Alive in response header.

Why are RESET packets being sent back to remote machines?


Thanks for any feedback you can provide.

-wr

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

wr,

On 9/29/2011 4:49 PM, rapponcape wrote:
>> Aha, the client is using HTTP 1.0 and not 1.1. This could be one
>> reason for Tomcat closing the connection. See below.
> 
> When I reconfigure and put Apache in front of Tomcat and use a
> jkmount with AJP connector, response is normal and without error.
> Appears to only happen when tomcat is standalone.

That's probably because Apache httpd is proxying the HTTP/1.0 request
and converting it into an HTTP/1.1 request and managing the chunked
response itself. Or, possibly, httpd is buffering the entire response
before sending the headers to the client and can provide a
content-length header. Have you looked at the conversation between
httpd and Tomcat?

>>> You see: no Content-Length header. I
> 
> I do see Content-Length in the POST header

Rainer was talking about the response, not the request.

Seems pretty clear that HTTP/1.0 + no content-length = problem.

You have a few options:

1. Use Apache httpd in the middle. That may not always work even
   though it worked in your tests.

2. Buffer the response on the server in order to determine the
   content-length, and send that in your response.

3. Have your clients use HTTP 1.1. Honestly, I'm unclear as to why there
   are so many clients out there that only use HTTP 1.0. I guess there
   are some features that aren't dumb-client-friendly (chunking, for
   instance) and it's easier to implement. Boo.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6E3qgACgkQ9CaO5/Lv0PByWACaAoJqRQjc2by/e70lrXK51fb+
Y4UAn0rDFDND3DBvozYrXlrzQ3M05YRN
=MWT/
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by André Warnier <aw...@ice-sa.com>.
Caldarale, Charles R wrote:
>> From: rapponcape [mailto:rapponcape@gmail.com] 
>> Subject: Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive
> 
>> It appears that I'll have to go back to using Apache
> 
> Note that Tomcat is also Apache.
> 
>> to proxy these requests to get keep-alive behavior to
>> function with these clients.
> 
> Or fix your webapp to be spec-compliant and set the Content-Length header as it should be doing.
> 

That's probably the crux of it.

rapponcape,

Suppose your client sends a request, requesting a persistent (keep-alive) connection, and 
the webserver (Tomcat) is willing to grant it.

The client is reading on the connection back from Tomcat, where it is expecting the 
response being written by Tomcat (or rather, by your web application running under Tomcat).

Your web application writes a response on that connection, but does not provide a header 
with the length of the response which it is sending back.
How is the client supposed to know when the response is finished ?

Answer: It can't, so it will keep waiting forever on the connection, for more bytes to arrive.

The /only/ way for Tomcat in such a case, to let the client know that the response is 
finished, is by closing the connection.  For the client reading, this acts as an 
end-of-file, so now it knows that the response is finished.

When you think of it, that's pretty clever from Tomcat : it sees that the webapp is not 
sending a content-length, and that the client is a HTTP/1.0 client who requested a 
keep-alive connection. So it knows that the client may be waiting forever for more bytes, 
and to avoid that, it closes the connection when the webapp exits, so as to let the client 
know that it's finished, no more bytes are coming.

If you want to avoid this, have your web application send a content-length along with the 
response.  Then Tomcat will know that since there is a content-length, the HTTP/1.0 client 
should be able to tell by itself when this response is finished, and it does not need to 
close the connection.

Got it ?

By /not/ sending a content-length in the response, your webapp is giving Tomcat no choice.
It has to close the connection, and cannot leave it open (and persistent).

That is the meaning of that second phrase in the passage which you quoted from Apache httpd.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


RE: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: rapponcape [mailto:rapponcape@gmail.com] 
> Subject: Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

> It appears that I'll have to go back to using Apache

Note that Tomcat is also Apache.

> to proxy these requests to get keep-alive behavior to
> function with these clients.

Or fix your webapp to be spec-compliant and set the Content-Length header as it should be doing.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by rapponcape <ra...@gmail.com>.
On Fri, Sep 30, 2011 at 11:34 AM, André Warnier <aw...@ice-sa.com> wrote:
> rapponcape wrote:
>>
>> On Fri, Sep 30, 2011 at 4:51 AM, André Warnier <aw...@ice-sa.com> wrote:
>>>
>>> Rainer Jung wrote:
>>>>
>>>> On 29.09.2011 22:49, rapponcape wrote:
>>>>>>
>>>>>> Aha, the client is using HTTP 1.0 and not 1.1. This could be one
>>>>>> reason
>>>>>> for Tomcat closing the connection. See below.
>>>>>>
>>>>> When I reconfigure and put Apache in front of Tomcat and use a jkmount
>>>>> with
>>>>> AJP connector, response is normal and without error.  Appears to only
>>>>> happen
>>>>> when tomcat is standalone.
>>>>
>>>> Could be the answer is small enough, that Apache can buffer it and send
>>>> without chunked encoding. Don't know.
>>>>
>>>>>>> You see: no Content-Length header. I
>>>>>
>>>>> I do see Content-Length in the POST header
>>>>
>>>> You cut out the relevant part. I was talkingabout the Tomcat answer, you
>>>> saw it in the client request. That's now the same ;)
>>>>
>>>> Rainer
>>>>
>>> rapponcape,
>>>
>>> to clarify maybe the discussion and answers which you have received so
>>> far :
>>>
>>> 1) Your clients, in their requests, indicate a protocol level HTTP/1.0.
>>> If you read RFC2616 (HTTP 1.1), you will see that for HTTP 1.0, the
>>> persistent ("keep-alive") kind of connection is not very well defined.
>>> Keep-alive connections are only well-defined and supported in HTTP/1.1.
>>>
>>> Consequently, a HTTP/1.1-capable server (like Apache httpd and Tomcat),
>>> upon
>>> receiving a request specifying HTTP/1.0, cannot assume that the client
>>> really knows how to handle this properly, and /can/ decide to not honor
>>> the
>>> desire for keep-alive connections, and close the connection after every
>>> response.
>>> In this case, it appears that Apache httpd may follow one behaviour, and
>>> Tomcat the other.
>>> Other webservers may choose to do either one or the other.
>>> (Both behaviours are allowed by the RFC).
>>>
>>> The point is : if your client wants to increase its chances to get a
>>> keep-alive connection, then it should use HTTP/1.1, not HTTP/1.0.
>>>
>>> 2) When your client connects to Apache httpd, and httpd serves as a proxy
>>> to
>>> Tomcat, there are two independent connections :
>>>
>>> client <-- (1) --> Apache httpd + mod_jk <-- (2) ---> Tomcat
>>>
>>> It is allowed for (Apache httpd + mod_jk) to make a proxy connection (2)
>>> to
>>> Tomcat using a HTTP 1.1 protocol level, allowing for persistent
>>> connections,
>>> even if on the other side (1) the connection is HTTP/1.0 and not
>>> persistent.
>>> (In fact, here this is pretty much irrelevant, since the connection (2)
>>> is
>>> made under the AJP protocol, not HTTP.  And mod_jk will always try to use
>>> persistent connections to Tomcat, for efficiency).
>>>
>>> So it is very possible that your client sees a different behaviour when
>>> it
>>> talk to Apache httpd, compared to when it talks to Tomcat directly.
>>> But that is because your client indicates that it wants a protocol level
>>> HTTP/1.0.
>>>
>>> So, in summary :
>>> - Apache httpd and Tomcat are right, because they follow the RFC
>>> - your client is wrong, inasmuch as it has an unfounded expectation,
>>> using
>>> HTTP/1.0, to actually get a persistent connection when it asks for it.
>>>
>>> It should be added that, even under HTTP/1.1, there is no guarantee for
>>> the
>>> client that when it asks for a persistent connection, it will actually
>>> get
>>> one.  The HTTP 1.1 server can still decide to close the connection after
>>> the
>>> response, for a variety of reasons.
>>> But let's say that it would have a much better chance.
>>>
>>> And let's leave the question of chunked responses for the next step.
>>
>> Very helpful detail.
>>
>> It would be nice to see Tomcat react similarly to how Apache reacts to
>> HTTP/1.0 messages that specify Keep-Alive and Content Length.  From
>> Apache docs...
>> "For HTTP/1.0 clients, Keep-Alive connections will only be used if
>> they are specifically requested by a client. In addition, a Keep-Alive
>> connection with an HTTP/1.0 client can only be used when the length of
>> the content is known in advance."
>>
> Sorry to insist, but do you really understand the implications of the 2d
> phrase above ?
> Is your Tomcat application setting a content-length /in its response/ ?
>

No, my app does not set/send a content length header.  The remote hosts
posting to my app send content length header details, but, my app does not.

.................................remote host's
post.................................
POST /myapp HTTP/1.0
HOST: 10.3.4.7:8080
Connection: Keep-Alive
Authorization: Basic Y2VybmVyOnlpbiZ5YW5n
Content-type: multipart/form-data; boundary=----------1509627285
Content-Length: 66632

------------1509627285
Content-Disposition: form-data; name="xml"

.................................my app's
response.................................
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Date: Mon, 26 Sep 2011 12:39:44 GMT
Connection: close

<?xml version="1.0"?><log><info><stat>OK</stat></info></log>

=========================================================

It appears that I'll have to go back to using Apache to proxy these
requests to get
keep-alive behavior to function with these clients.

-wr

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by André Warnier <aw...@ice-sa.com>.
rapponcape wrote:
> On Fri, Sep 30, 2011 at 4:51 AM, André Warnier <aw...@ice-sa.com> wrote:
>> Rainer Jung wrote:
>>> On 29.09.2011 22:49, rapponcape wrote:
>>>>> Aha, the client is using HTTP 1.0 and not 1.1. This could be one reason
>>>>> for Tomcat closing the connection. See below.
>>>>>
>>>> When I reconfigure and put Apache in front of Tomcat and use a jkmount
>>>> with
>>>> AJP connector, response is normal and without error.  Appears to only
>>>> happen
>>>> when tomcat is standalone.
>>> Could be the answer is small enough, that Apache can buffer it and send
>>> without chunked encoding. Don't know.
>>>
>>>>>> You see: no Content-Length header. I
>>>> I do see Content-Length in the POST header
>>> You cut out the relevant part. I was talkingabout the Tomcat answer, you
>>> saw it in the client request. That's now the same ;)
>>>
>>> Rainer
>>>
>> rapponcape,
>>
>> to clarify maybe the discussion and answers which you have received so far :
>>
>> 1) Your clients, in their requests, indicate a protocol level HTTP/1.0.
>> If you read RFC2616 (HTTP 1.1), you will see that for HTTP 1.0, the
>> persistent ("keep-alive") kind of connection is not very well defined.
>> Keep-alive connections are only well-defined and supported in HTTP/1.1.
>>
>> Consequently, a HTTP/1.1-capable server (like Apache httpd and Tomcat), upon
>> receiving a request specifying HTTP/1.0, cannot assume that the client
>> really knows how to handle this properly, and /can/ decide to not honor the
>> desire for keep-alive connections, and close the connection after every
>> response.
>> In this case, it appears that Apache httpd may follow one behaviour, and
>> Tomcat the other.
>> Other webservers may choose to do either one or the other.
>> (Both behaviours are allowed by the RFC).
>>
>> The point is : if your client wants to increase its chances to get a
>> keep-alive connection, then it should use HTTP/1.1, not HTTP/1.0.
>>
>> 2) When your client connects to Apache httpd, and httpd serves as a proxy to
>> Tomcat, there are two independent connections :
>>
>> client <-- (1) --> Apache httpd + mod_jk <-- (2) ---> Tomcat
>>
>> It is allowed for (Apache httpd + mod_jk) to make a proxy connection (2) to
>> Tomcat using a HTTP 1.1 protocol level, allowing for persistent connections,
>> even if on the other side (1) the connection is HTTP/1.0 and not persistent.
>> (In fact, here this is pretty much irrelevant, since the connection (2) is
>> made under the AJP protocol, not HTTP.  And mod_jk will always try to use
>> persistent connections to Tomcat, for efficiency).
>>
>> So it is very possible that your client sees a different behaviour when it
>> talk to Apache httpd, compared to when it talks to Tomcat directly.
>> But that is because your client indicates that it wants a protocol level
>> HTTP/1.0.
>>
>> So, in summary :
>> - Apache httpd and Tomcat are right, because they follow the RFC
>> - your client is wrong, inasmuch as it has an unfounded expectation, using
>> HTTP/1.0, to actually get a persistent connection when it asks for it.
>>
>> It should be added that, even under HTTP/1.1, there is no guarantee for the
>> client that when it asks for a persistent connection, it will actually get
>> one.  The HTTP 1.1 server can still decide to close the connection after the
>> response, for a variety of reasons.
>> But let's say that it would have a much better chance.
>>
>> And let's leave the question of chunked responses for the next step.
> 
> Very helpful detail.
> 
> It would be nice to see Tomcat react similarly to how Apache reacts to
> HTTP/1.0 messages that specify Keep-Alive and Content Length.  From
> Apache docs...
> "For HTTP/1.0 clients, Keep-Alive connections will only be used if
> they are specifically requested by a client. In addition, a Keep-Alive
> connection with an HTTP/1.0 client can only be used when the length of
> the content is known in advance."
> 
Sorry to insist, but do you really understand the implications of the 2d phrase above ?
Is your Tomcat application setting a content-length /in its response/ ?


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by rapponcape <ra...@gmail.com>.
On Fri, Sep 30, 2011 at 4:51 AM, André Warnier <aw...@ice-sa.com> wrote:
> Rainer Jung wrote:
>>
>> On 29.09.2011 22:49, rapponcape wrote:
>>>>
>>>> Aha, the client is using HTTP 1.0 and not 1.1. This could be one reason
>>>> for Tomcat closing the connection. See below.
>>>>
>>> When I reconfigure and put Apache in front of Tomcat and use a jkmount
>>> with
>>> AJP connector, response is normal and without error.  Appears to only
>>> happen
>>> when tomcat is standalone.
>>
>> Could be the answer is small enough, that Apache can buffer it and send
>> without chunked encoding. Don't know.
>>
>>>>> You see: no Content-Length header. I
>>>
>>> I do see Content-Length in the POST header
>>
>> You cut out the relevant part. I was talkingabout the Tomcat answer, you
>> saw it in the client request. That's now the same ;)
>>
>> Rainer
>>
> rapponcape,
>
> to clarify maybe the discussion and answers which you have received so far :
>
> 1) Your clients, in their requests, indicate a protocol level HTTP/1.0.
> If you read RFC2616 (HTTP 1.1), you will see that for HTTP 1.0, the
> persistent ("keep-alive") kind of connection is not very well defined.
> Keep-alive connections are only well-defined and supported in HTTP/1.1.
>
> Consequently, a HTTP/1.1-capable server (like Apache httpd and Tomcat), upon
> receiving a request specifying HTTP/1.0, cannot assume that the client
> really knows how to handle this properly, and /can/ decide to not honor the
> desire for keep-alive connections, and close the connection after every
> response.
> In this case, it appears that Apache httpd may follow one behaviour, and
> Tomcat the other.
> Other webservers may choose to do either one or the other.
> (Both behaviours are allowed by the RFC).
>
> The point is : if your client wants to increase its chances to get a
> keep-alive connection, then it should use HTTP/1.1, not HTTP/1.0.
>
> 2) When your client connects to Apache httpd, and httpd serves as a proxy to
> Tomcat, there are two independent connections :
>
> client <-- (1) --> Apache httpd + mod_jk <-- (2) ---> Tomcat
>
> It is allowed for (Apache httpd + mod_jk) to make a proxy connection (2) to
> Tomcat using a HTTP 1.1 protocol level, allowing for persistent connections,
> even if on the other side (1) the connection is HTTP/1.0 and not persistent.
> (In fact, here this is pretty much irrelevant, since the connection (2) is
> made under the AJP protocol, not HTTP.  And mod_jk will always try to use
> persistent connections to Tomcat, for efficiency).
>
> So it is very possible that your client sees a different behaviour when it
> talk to Apache httpd, compared to when it talks to Tomcat directly.
> But that is because your client indicates that it wants a protocol level
> HTTP/1.0.
>
> So, in summary :
> - Apache httpd and Tomcat are right, because they follow the RFC
> - your client is wrong, inasmuch as it has an unfounded expectation, using
> HTTP/1.0, to actually get a persistent connection when it asks for it.
>
> It should be added that, even under HTTP/1.1, there is no guarantee for the
> client that when it asks for a persistent connection, it will actually get
> one.  The HTTP 1.1 server can still decide to close the connection after the
> response, for a variety of reasons.
> But let's say that it would have a much better chance.
>
> And let's leave the question of chunked responses for the next step.

Very helpful detail.

It would be nice to see Tomcat react similarly to how Apache reacts to
HTTP/1.0 messages that specify Keep-Alive and Content Length.  From
Apache docs...
"For HTTP/1.0 clients, Keep-Alive connections will only be used if
they are specifically requested by a client. In addition, a Keep-Alive
connection with an HTTP/1.0 client can only be used when the length of
the content is known in advance."

Thanks all.
-wr

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by André Warnier <aw...@ice-sa.com>.
Rainer Jung wrote:
> On 29.09.2011 22:49, rapponcape wrote:
>>> Aha, the client is using HTTP 1.0 and not 1.1. This could be one reason
>>> for Tomcat closing the connection. See below.
>>>
>> When I reconfigure and put Apache in front of Tomcat and use a jkmount with
>> AJP connector, response is normal and without error.  Appears to only happen
>> when tomcat is standalone.
> 
> Could be the answer is small enough, that Apache can buffer it and send
> without chunked encoding. Don't know.
> 
>>>> You see: no Content-Length header. I
>> I do see Content-Length in the POST header
> 
> You cut out the relevant part. I was talkingabout the Tomcat answer, you
> saw it in the client request. That's now the same ;)
> 
> Rainer
> 
rapponcape,

to clarify maybe the discussion and answers which you have received so far :

1) Your clients, in their requests, indicate a protocol level HTTP/1.0.
If you read RFC2616 (HTTP 1.1), you will see that for HTTP 1.0, the persistent 
("keep-alive") kind of connection is not very well defined.
Keep-alive connections are only well-defined and supported in HTTP/1.1.

Consequently, a HTTP/1.1-capable server (like Apache httpd and Tomcat), upon receiving a 
request specifying HTTP/1.0, cannot assume that the client really knows how to handle this 
properly, and /can/ decide to not honor the desire for keep-alive connections, and close 
the connection after every response.
In this case, it appears that Apache httpd may follow one behaviour, and Tomcat the other.
Other webservers may choose to do either one or the other.
(Both behaviours are allowed by the RFC).

The point is : if your client wants to increase its chances to get a keep-alive 
connection, then it should use HTTP/1.1, not HTTP/1.0.

2) When your client connects to Apache httpd, and httpd serves as a proxy to Tomcat, there 
are two independent connections :

client <-- (1) --> Apache httpd + mod_jk <-- (2) ---> Tomcat

It is allowed for (Apache httpd + mod_jk) to make a proxy connection (2) to Tomcat using a 
HTTP 1.1 protocol level, allowing for persistent connections, even if on the other side 
(1) the connection is HTTP/1.0 and not persistent.
(In fact, here this is pretty much irrelevant, since the connection (2) is made under the 
AJP protocol, not HTTP.  And mod_jk will always try to use persistent connections to 
Tomcat, for efficiency).

So it is very possible that your client sees a different behaviour when it talk to Apache 
httpd, compared to when it talks to Tomcat directly.
But that is because your client indicates that it wants a protocol level HTTP/1.0.

So, in summary :
- Apache httpd and Tomcat are right, because they follow the RFC
- your client is wrong, inasmuch as it has an unfounded expectation, using HTTP/1.0, to 
actually get a persistent connection when it asks for it.

It should be added that, even under HTTP/1.1, there is no guarantee for the client that 
when it asks for a persistent connection, it will actually get one.  The HTTP 1.1 server 
can still decide to close the connection after the response, for a variety of reasons.
But let's say that it would have a much better chance.

And let's leave the question of chunked responses for the next step.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by Rainer Jung <ra...@kippdata.de>.
On 29.09.2011 22:49, rapponcape wrote:
>> Aha, the client is using HTTP 1.0 and not 1.1. This could be one reason
>> for Tomcat closing the connection. See below.
>>
> 
> When I reconfigure and put Apache in front of Tomcat and use a jkmount with
> AJP connector, response is normal and without error.  Appears to only happen
> when tomcat is standalone.

Could be the answer is small enough, that Apache can buffer it and send
without chunked encoding. Don't know.

>>> You see: no Content-Length header. I
> 
> I do see Content-Length in the POST header

You cut out the relevant part. I was talkingabout the Tomcat answer, you
saw it in the client request. That's now the same ;)

Rainer

> On Thu, Sep 29, 2011 at 1:49 PM, Rainer Jung <ra...@kippdata.de> wrote:
>> On 29.09.2011 18:28, rapponcape wrote:
>>> Tomcat version 7.0.6 listening on local machine 10.3.4.7 and accepting
>>> incoming POST connections from 4 remote machines, 10.12.5.10[2-5].
>>> Configured in standalone mode, using HTTP connector.
>>
>> 7.0.6 was not ment for production use. Start with the latest 7.0 release.
>>
>>> Remote machines are sending constant stream of POST messages...~100 kbit/sec.
>>>
>>> All POST traffic is accepted and acknowledged, but, RESET packets
>>> cause remotes machines to re-establish new connections, causing errors
>>> on sender's side and slowing down rate of transfer.
>>>
>>> Traffic headers show local machine is sending "Connection: close" on
>>> post acknowledgement messages.  Despite fact that remote machines'
>>> POST messages contain "Connection: Keep-Alive" header.
>>>
>>> ======================
>>> post traffic header ...
>>> ======================
>>>
>>> POST /myapp HTTP/1.0
>>
>> Aha, the client is using HTTP 1.0 and not 1.1. This could be one reason
>> for Tomcat closing the connection. See below.
>>
>>> HOST: 10.3.4.7:8080
>>> Connection: Keep-Alive
>>> Authorization: Basic Y2VybmVyOnlpbiZ5YW5n
>>> Content-type: multipart/form-data; boundary=----------1803119234
>>> Content-Length: 59970
>>>
>>> ------------1803119234
>>> Content-Disposition: form-data; name="xml"
>>> <?xml version="1.0" encoding="ISO-
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>>
>>> ======================
>>> response header ...
>>> ======================
>>>
>>> ------------1803119234--
>>>
>>> HTTP/1.1 200 OK
>>> Server: Apache-Coyote/1.1
>>> Date: Fri, 23 Sep 2011 18:27:25 GMT
>>> Connection: close
>>
>> You see: no Content-Length header. It's likely dynamic content, so it is
>> streamed, the length of the content is not known in advance. This means
>> Tomcat has either to use chunked encoding, which needs a HTTP/1.1
>> client, or it must simply close the connection at the end of the
>> transfer, which it does.
>>
>>> ======================
>>>
>>>> # netstat -anp | grep 8080
>>>
>>> tcp        0      0 :::8081                     :::*
>>>     LISTEN      23107/java
>>> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.103:39809
>>> FIN_WAIT2   -
>>> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.104:33033
>>> FIN_WAIT2   -
>>> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.102:57077
>>> FIN_WAIT2   -
>>> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.105:39624
>>> FIN_WAIT2   -
>>>
>>> I've tried various connector configurations and protocols; HTTP, NIO,
>>> connectionTimeout, etc.
>>>
>>> Here's what my connector config looks like currently:
>>>
>>>  <Connector port=8080 protocol="org.apache.coyote.http11.Http11Protocol"
>>>               connectionTimeout="30000"
>>>               connectionLinger="100"
>>>               keepAliveTimeout="3000"
>>>               maxKeepAliveRequests="2000"
>>>               disableUploadTimeout="true"
>>>               maxThreads="500"
>>>               minSpareThreads="150"
>>>               maxSpareThreads="300"
>>>               acceptCount="200"
>>>               socket.soKeepAlive="yes"
>>>               enableLookups="false" />
>>>
>>> ===========================================
>>>
>>> Why fin_wait2 state?  I would expect to see ESTABLISHED connections
>>> and and Keep-Alive in response header.
>>>
>>> Why are RESET packets being sent back to remote machines?
>>
>> Because Tomcat said "Connection: close", which means the client is not
>> allowed to reuse the connection and Tomcat closes its side. Likely the
>> client is nevertheless trying to reuse the connection, which will be
>> answered by RST by the OS. Since your client doesn't send the FIN
>> package, the state is FIN_WAIT2.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by rapponcape <ra...@gmail.com>.
Thanks for your reply.  Followup....

>
> 7.0.6 was not ment for production use. Start with the latest 7.0 release.
>

Will attempt to test with latest stable 7.0.21

>
> Aha, the client is using HTTP 1.0 and not 1.1. This could be one reason
> for Tomcat closing the connection. See below.
>

When I reconfigure and put Apache in front of Tomcat and use a jkmount with
AJP connector, response is normal and without error.  Appears to only happen
when tomcat is standalone.

>> You see: no Content-Length header. I

I do see Content-Length in the POST header


On Thu, Sep 29, 2011 at 1:49 PM, Rainer Jung <ra...@kippdata.de> wrote:
> On 29.09.2011 18:28, rapponcape wrote:
>> Tomcat version 7.0.6 listening on local machine 10.3.4.7 and accepting
>> incoming POST connections from 4 remote machines, 10.12.5.10[2-5].
>> Configured in standalone mode, using HTTP connector.
>
> 7.0.6 was not ment for production use. Start with the latest 7.0 release.
>
>> Remote machines are sending constant stream of POST messages...~100 kbit/sec.
>>
>> All POST traffic is accepted and acknowledged, but, RESET packets
>> cause remotes machines to re-establish new connections, causing errors
>> on sender's side and slowing down rate of transfer.
>>
>> Traffic headers show local machine is sending "Connection: close" on
>> post acknowledgement messages.  Despite fact that remote machines'
>> POST messages contain "Connection: Keep-Alive" header.
>>
>> ======================
>> post traffic header ...
>> ======================
>>
>> POST /myapp HTTP/1.0
>
> Aha, the client is using HTTP 1.0 and not 1.1. This could be one reason
> for Tomcat closing the connection. See below.
>
>> HOST: 10.3.4.7:8080
>> Connection: Keep-Alive
>> Authorization: Basic Y2VybmVyOnlpbiZ5YW5n
>> Content-type: multipart/form-data; boundary=----------1803119234
>> Content-Length: 59970
>>
>> ------------1803119234
>> Content-Disposition: form-data; name="xml"
>> <?xml version="1.0" encoding="ISO-
>> ---------------------------------------------------------------------------------------------------------------------------------------
>>
>> ======================
>> response header ...
>> ======================
>>
>> ------------1803119234--
>>
>> HTTP/1.1 200 OK
>> Server: Apache-Coyote/1.1
>> Date: Fri, 23 Sep 2011 18:27:25 GMT
>> Connection: close
>
> You see: no Content-Length header. It's likely dynamic content, so it is
> streamed, the length of the content is not known in advance. This means
> Tomcat has either to use chunked encoding, which needs a HTTP/1.1
> client, or it must simply close the connection at the end of the
> transfer, which it does.
>
>> ======================
>>
>>> # netstat -anp | grep 8080
>>
>> tcp        0      0 :::8081                     :::*
>>     LISTEN      23107/java
>> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.103:39809
>> FIN_WAIT2   -
>> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.104:33033
>> FIN_WAIT2   -
>> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.102:57077
>> FIN_WAIT2   -
>> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.105:39624
>> FIN_WAIT2   -
>>
>> I've tried various connector configurations and protocols; HTTP, NIO,
>> connectionTimeout, etc.
>>
>> Here's what my connector config looks like currently:
>>
>>  <Connector port=8080 protocol="org.apache.coyote.http11.Http11Protocol"
>>               connectionTimeout="30000"
>>               connectionLinger="100"
>>               keepAliveTimeout="3000"
>>               maxKeepAliveRequests="2000"
>>               disableUploadTimeout="true"
>>               maxThreads="500"
>>               minSpareThreads="150"
>>               maxSpareThreads="300"
>>               acceptCount="200"
>>               socket.soKeepAlive="yes"
>>               enableLookups="false" />
>>
>> ===========================================
>>
>> Why fin_wait2 state?  I would expect to see ESTABLISHED connections
>> and and Keep-Alive in response header.
>>
>> Why are RESET packets being sent back to remote machines?
>
> Because Tomcat said "Connection: close", which means the client is not
> allowed to reuse the connection and Tomcat closes its side. Likely the
> client is nevertheless trying to reuse the connection, which will be
> answered by RST by the OS. Since your client doesn't send the FIN
> package, the state is FIN_WAIT2.
>
> Regards,
>
> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat 7.0.6 FIN_WAIT2 Connections and Connection Keep-Alive

Posted by Rainer Jung <ra...@kippdata.de>.
On 29.09.2011 18:28, rapponcape wrote:
> Tomcat version 7.0.6 listening on local machine 10.3.4.7 and accepting
> incoming POST connections from 4 remote machines, 10.12.5.10[2-5].
> Configured in standalone mode, using HTTP connector.

7.0.6 was not ment for production use. Start with the latest 7.0 release.

> Remote machines are sending constant stream of POST messages...~100 kbit/sec.
> 
> All POST traffic is accepted and acknowledged, but, RESET packets
> cause remotes machines to re-establish new connections, causing errors
> on sender's side and slowing down rate of transfer.
> 
> Traffic headers show local machine is sending "Connection: close" on
> post acknowledgement messages.  Despite fact that remote machines'
> POST messages contain "Connection: Keep-Alive" header.
> 
> ======================
> post traffic header ...
> ======================
> 
> POST /myapp HTTP/1.0

Aha, the client is using HTTP 1.0 and not 1.1. This could be one reason
for Tomcat closing the connection. See below.

> HOST: 10.3.4.7:8080
> Connection: Keep-Alive
> Authorization: Basic Y2VybmVyOnlpbiZ5YW5n
> Content-type: multipart/form-data; boundary=----------1803119234
> Content-Length: 59970
> 
> ------------1803119234
> Content-Disposition: form-data; name="xml"
> <?xml version="1.0" encoding="ISO-
> ---------------------------------------------------------------------------------------------------------------------------------------
> 
> ======================
> response header ...
> ======================
> 
> ------------1803119234--
> 
> HTTP/1.1 200 OK
> Server: Apache-Coyote/1.1
> Date: Fri, 23 Sep 2011 18:27:25 GMT
> Connection: close

You see: no Content-Length header. It's likely dynamic content, so it is
streamed, the length of the content is not known in advance. This means
Tomcat has either to use chunked encoding, which needs a HTTP/1.1
client, or it must simply close the connection at the end of the
transfer, which it does.

> ======================
> 
>> # netstat -anp | grep 8080
> 
> tcp        0      0 :::8081                     :::*
>     LISTEN      23107/java
> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.103:39809
> FIN_WAIT2   -
> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.104:33033
> FIN_WAIT2   -
> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.102:57077
> FIN_WAIT2   -
> tcp        0      0 ::ffff:10.3.4.7:8080    ::ffff:10.12.5.105:39624
> FIN_WAIT2   -
> 
> I've tried various connector configurations and protocols; HTTP, NIO,
> connectionTimeout, etc.
> 
> Here's what my connector config looks like currently:
> 
>  <Connector port=8080 protocol="org.apache.coyote.http11.Http11Protocol"
>               connectionTimeout="30000"
>               connectionLinger="100"
>               keepAliveTimeout="3000"
>               maxKeepAliveRequests="2000"
>               disableUploadTimeout="true"
>               maxThreads="500"
>               minSpareThreads="150"
>               maxSpareThreads="300"
>               acceptCount="200"
>               socket.soKeepAlive="yes"
>               enableLookups="false" />
> 
> ===========================================
> 
> Why fin_wait2 state?  I would expect to see ESTABLISHED connections
> and and Keep-Alive in response header.
> 
> Why are RESET packets being sent back to remote machines?

Because Tomcat said "Connection: close", which means the client is not
allowed to reuse the connection and Tomcat closes its side. Likely the
client is nevertheless trying to reuse the connection, which will be
answered by RST by the OS. Since your client doesn't send the FIN
package, the state is FIN_WAIT2.

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org