You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2015/10/29 15:29:35 UTC
[Bug 58565] New: Connection will be closed on slow file download
with non blocking http protocol
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
Bug ID: 58565
Summary: Connection will be closed on slow file download with
non blocking http protocol
Product: Tomcat 8
Version: 8.0.28
Hardware: PC
OS: All
Status: NEW
Severity: major
Priority: P2
Component: Connectors
Assignee: dev@tomcat.apache.org
Reporter: david.pentzlin@commscope.com
Problem:
Setup Tomcat (8.0.28) with default settings. Copy a file in a webapp e.g.
webapps/root and download the file with limited speed to simulate a slow modem
download.
wget http://xxx.xxx.xxx.xxx:8080/xxxxxxx/7.zip --limit-rate=1k
Connection is closed always after 6minutes and 24seconds
e.g:
Saving to: ‘7.zip’
7.zip 0%[ ] 767.99K 1.00KB/s in 6m 24s
2015-10-28 17:27:39 (1024 B/s) - Connection closed at byte 786426. Retrying.
if i change the protocol to an old blocking protocol:
protocol="org.apache.coyote.http11.Http11Protocol"
everything is fine and the download will complete with slow speed.
If the change the speed (with protocol HTTP/1.1) the time after the connection
is closed is different (e.g. for 7kb/sec it will always close after 3minutes
and 21sec).
If the speed is 8kb/sec or higher the download is stable with both protocols.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
--- Comment #5 from Christopher Schultz <ch...@christopherschultz.net> ---
Bufferbloat strikes again. :(
Does NIOx have enough control over the underlying socket/connection to reduce
buffer sizes so that this problem is minimized?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
--- Comment #3 from Remy Maucherat <re...@apache.org> ---
After debugging, it looks like it's a combination of factors.
What happens is that the network stack buffers things a lot without blocking.
At some point it will block until everything gets flushed. Unfortunately, the
flushing takes forever due to the client speed limit, and a legitimate write
connection timeout occurs in Tomcat. Then the network stack will continue
trickling bytes to the client until the connection is closed due to the error
(which occurred a long time before).
I would recommend to *not* increase the timeout for the connection [that would
be a workaround], and I doubt this can be fixed effectively. So this could end
up as a WONTFIX.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
--- Comment #8 from Remy Maucherat <re...@apache.org> ---
It works for me with the right configuration, but not the default one, so I
chose WORKSFORME but I don't mind if someone wants to change it.
It would seem with the default buffer on Linux + localhost and the default 20s
timeout, the client read speed would have to be over 64KB/s to avoid a timeout.
Increasing the timeout from 20s to 60s would allow slower speeds.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
--- Comment #9 from Konstantin Kolinko <kn...@gmail.com> ---
(In reply to comment #8)
> It would seem with the default buffer on Linux + localhost ...
1. If I understand correctly, real networks have some quality-of-service
mechanisms. A well known one is MTU size. For a localhost connection the MTU
size is enormous.
2. I do not know how wget implements its --limit-rate feature. Maybe all those
MBs of data have already arrived and are in it's own (client's) buffer.
3. Maybe in some configurations it makes sense to specify timeouts measured by
throughput, instead of a time unit.
(E.g. it may be easier to configure requirement of minimum throughput of X kb/s
instead of a read timeout of Y sec.) It means that the actual timeout (that
will be used as an argument to APIs) needs to be calculated dynamically based
on amount of data that have been transfered earlier over this connection.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
Remy Maucherat <re...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |WORKSFORME
Status|NEW |RESOLVED
--- Comment #6 from Remy Maucherat <re...@apache.org> ---
Well, it's a network stack thing so it is transparent. I suppose it is dynamic
depending on the memory condition. The network stack has an incentive to make
buffers huge since it will vastly improve performance as everything is non
blocking.
Luckily, it is exposed in NIOx (NIO2 has fewer socket options compared to NIO1,
but it has that one) and I could find socket.txBufSize allows configuration in
Tomcat, and probably there's something at the OS level as well. So a low value
for socket.txBufSize allows this to run, I just verified it. By default I found
out it is 1313280 for me [AKA memory is cheap :) ].
Anyway, thanks for the hint to get me to look at the socket options, it is now
determined it is only a configuration issue. Given the low speeds needed for
this to happen and the possible performance impact, I would be -1 for trying to
override the network stack default without explicit user configuration.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
Remy Maucherat <re...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Severity|major |normal
--- Comment #1 from Remy Maucherat <re...@apache.org> ---
Ok. Internally things are very different with NIOx: lots of buffering in the
network stack and non blocking calls. It is possible this needs OS level
configuration if available, and Tomcat itself doesn't do anything wrong. This
will be investigated.
Downgrading severity since in a way it's an anti DoS feature ...
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
--- Comment #7 from Christopher Schultz <ch...@christopherschultz.net> ---
It seems like less buffer for NIO makes sense in general, though I agree that
changing the default is probably not necessary, here. It might be worth
mentioning in the documentation that NIO + slow speed = increased change of
end-to-end timeouts.
It seems like there is also the possibility of maybe having some kind of QoS
setting that causes flushes to occur with more regularity, even if the buffer
is large enough to handle more data. Something like once every few seconds
maybe? That way, the (slow) writes won't pile-up and cause a large flush that
takes too much time.
I don't know enough about the underlying NIO and threading strategy to know
whether this would be easy or difficult, or yet another unnecessary
complication in an already complicated machine.
But theoretically, there are always some values for which connections will be
dropped due to this problem. Unless buffers are eliminated, which degrades NIO
-> BIO which is of course not what we want. How does a user balance throughput
and performance against fault-tolerance?
(also, should this resolution be INVALID/NOTABUG or WORKSFORME?)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
--- Comment #4 from Dave <da...@commscope.com> ---
so what do you recommend? our application is accessed by many of our customers
via gsm modem.
In some cases the speed is lower than 8kb/sec (gprs).
Which timeout do i have to change?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
--- Comment #10 from Remy Maucherat <re...@apache.org> ---
IMO a direct "speed" configuration would be quite convoluted and users would
have a hard time understanding it. It sounds better to me to keep buffer size
and timeout instead.
Are there really many who still want to support "large" files and dialup
[without retry, since for example wget just happily resumes] ? In that
situation, an admin should first increase the timeout, since 20s sounds too
short for dialup even if not counting this issue. I'd say 60s in that situation
at least, or maybe longer and using a separate shorter keepalive timeout.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 58565] Connection will be closed on slow file download with non
blocking http protocol
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58565
--- Comment #2 from Dave <da...@commscope.com> ---
just as a note. i tried it with windows and linux and got the same
results/problems.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org