You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Wittmann Armin <aw...@ethz.ch> on 2007/09/04 13:22:53 UTC
after 60000 requests more than 700 sockets in CLOSE_WAIT
Hi httpclient-Team
over the last 4 days I made a long term test for
our new software release.
This software ist integrated into an Apache Tomcat 5.5
(as a servlet receiving requests, transform them and
sending out other GET-requests through http-client) and should
run very reliable and without any resource leaks.
After finishing this test cycle I noticed that there
remained over 700 sockets in CLOSE_WAIT state
(Linux -> netstat -a -p).
Due to the identified PID and the destination ip number it is
obvious that these sockets have been caused/used
by the program part using http-client-3.0.1.
I am not a real network crack so I don't know if I need
to bother about this. Since the software/tomcat is itended
to run months (7x24) without necessarily rebooting it
I am not shure if I will run out of network resources.
Can somebody help in this subject?
By the way: all 60000 http-GET-requests worked well and there
were no other problems at all.
Regards
Armin
--------------------------------------------------------------------
My Code (simplified):
this code ist executed for every single request
HttpConnectionManager connectionManager = new
SimpleHttpConnectionManager();
HttpClient client = new HttpClient(clientParams, connectionManager);
client.setHostConfiguration(hostConfiguration);
HttpMethod method = = new GetMethod();
method.setQueryString(pairs);
method.setPath(pUrl.getPath());
method.setParams(methodParams);
try {
client.executeMethod(method);
} catch (Exception e) {
failed = true;
throw new Exception(...);
} finally {
if (failed) method.abort();
method.releaseConnection();
client.setHttpConnectionManager(null);
client = null;
}
try {
responseString = method.getResponseBodyAsString();
} catch (Exception e) {
throw new Exception(...);
} finally {
method.releaseConnection();
method = null;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2007-09-06 at 08:40 +0200, Wittmann Armin wrote:
> Hi Oleg
>
> > HttpClient keeps connections alive (open) infinitely unless told
> > otherwise, as it has no way of knowing when those connections may be
> > needed again. It is a responsibility of the application to make sure
> > open connections get closed if they are no longer needed. The
> > recommended way of doing that is by calling #closeIdleConnections()
> > method on the connection manager or by shutting it down.
>
> Thanks für your answer.
> I will try to make 'my conclusions'.
> Since I am using the SimpleHttpConnectionManager that is not thread safe
> (in a multi thread environment as a servlet container as tomcat is) I
> correctly do use the instance of the simple http-connection manager only as a local
> variable that will be garbage collected after my http request has been made.
>
One should _always_ shut down the connection manager if it is no longer
needed / used.
Actually by creating a new connection manager on a per request you
effectively disable connection persistence, which can significantly
degrade performance. Consider reusing the connection manager between
requests if for some reason you are not willing to use the
multi-threaded (pooling) version.
> But, to release ALL used internal resources it is not sufficient to allocate the instance
> of the simple http-connection manager just locally, I also do need to use
> #closeIdleConnection (after a #releaseConnection) to free these network
> resources as well. (I checked this also looking into the code of the
> simple http-connection manager.)
> Correct?
>
If you simply dereference an instance of a connection manager without
shutting it down, you leave it up to the garbage collector to close open
connections and reclaim allocated sockets, which can really take a
while. This can especially be a problem if you allocate connection
managers in a tight loop / on a per request basis.
Hope this helps
Oleg
> Regards
>
> Armin
>
> ------------------------------------
> > > > My Code (simplified):
> > > > this code ist executed for every single request
> > > >
> > > > HttpConnectionManager connectionManager = new SimpleHttpConnectionManager();
> > > > HttpClient client = new HttpClient(clientParams, connectionManager);
> > > > client.setHostConfiguration(hostConfiguration);
> > > > HttpMethod method = = new GetMethod();
> > > > method.setQueryString(pairs);
> > > > method.setPath(pUrl.getPath());
> > > > method.setParams(methodParams);
> > > > try {
> > > > client.executeMethod(method);
> > > > } catch (Exception e) {
> > > > failed = true;
> > > > throw new Exception(...);
> > > > } finally {
> > > > if (failed) {
> > > > method.abort();
> > > > method.releaseConnection();
> > > > }
> > > >
> > > > client.setHttpConnectionManager(null);
> > > > client = null;
> > > > }
> > > >
> > > > try {
> > > > responseString = method.getResponseBodyAsString();
> > > > } catch (Exception e) {
> > > > throw new Exception(...);
> > > > } finally {
> > > > method.releaseConnection();
> > > > method = null;
> > > > }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
Posted by Wittmann Armin <aw...@ethz.ch>.
Hi Oleg
> HttpClient keeps connections alive (open) infinitely unless told
> otherwise, as it has no way of knowing when those connections may be
> needed again. It is a responsibility of the application to make sure
> open connections get closed if they are no longer needed. The
> recommended way of doing that is by calling #closeIdleConnections()
> method on the connection manager or by shutting it down.
Thanks für your answer.
I will try to make 'my conclusions'.
Since I am using the SimpleHttpConnectionManager that is not thread safe
(in a multi thread environment as a servlet container as tomcat is) I
correctly do use the instance of the simple http-connection manager only as a local
variable that will be garbage collected after my http request has been made.
But, to release ALL used internal resources it is not sufficient to allocate the instance
of the simple http-connection manager just locally, I also do need to use
#closeIdleConnection (after a #releaseConnection) to free these network
resources as well. (I checked this also looking into the code of the
simple http-connection manager.)
Correct?
Regards
Armin
------------------------------------
> > > My Code (simplified):
> > > this code ist executed for every single request
> > >
> > > HttpConnectionManager connectionManager = new SimpleHttpConnectionManager();
> > > HttpClient client = new HttpClient(clientParams, connectionManager);
> > > client.setHostConfiguration(hostConfiguration);
> > > HttpMethod method = = new GetMethod();
> > > method.setQueryString(pairs);
> > > method.setPath(pUrl.getPath());
> > > method.setParams(methodParams);
> > > try {
> > > client.executeMethod(method);
> > > } catch (Exception e) {
> > > failed = true;
> > > throw new Exception(...);
> > > } finally {
> > > if (failed) {
> > > method.abort();
> > > method.releaseConnection();
> > > }
> > >
> > > client.setHttpConnectionManager(null);
> > > client = null;
> > > }
> > >
> > > try {
> > > responseString = method.getResponseBodyAsString();
> > > } catch (Exception e) {
> > > throw new Exception(...);
> > > } finally {
> > > method.releaseConnection();
> > > method = null;
> > > }
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2007-09-04 at 17:16 +0200, Wittmann Armin wrote:
> Hi Pete
>
> under Linux it is found at a different place (in proc-fs). Look at the
> output below. I guess most settings are in seconds.
> I liked that one for newbies:
> http://www.utdallas.edu/~cantrell/ee6345/pocketguide.pdf
> http://ipsysctl-tutorial.frozentux.net/chunkyhtml/tcpvariables.html
>
> Any suggestions?
> 5 hours after the test stopped (tomcat is still running) none
> of the sockets has been freed.
>
> Thanks, Armin
>
Armin
HttpClient keeps connections alive (open) infinitely unless told
otherwise, as it has no way of knowing when those connections may be
needed again. It is a responsibility of the application to make sure
open connections get closed if they are no longer needed. The
recommended way of doing that is by calling #closeIdleConnections()
method on the connection manager or by shutting it down.
Hope this helps
Oleg
> linux:>$ ls | grep tcp_
> tcp_abc
> tcp_abort_on_overflow
> tcp_adv_win_scale
> tcp_app_win
> tcp_congestion_control
> tcp_dsack
> tcp_ecn
> tcp_fack
> tcp_fin_timeout
> tcp_frto
> tcp_keepalive_intvl
> tcp_keepalive_probes
> tcp_keepalive_time
> tcp_low_latency
> tcp_max_orphans
> tcp_max_syn_backlog
> tcp_max_tw_buckets
> tcp_mem
> tcp_moderate_rcvbuf
> tcp_no_metrics_save
> tcp_orphan_retries
> tcp_reordering
> tcp_retrans_collapse
> tcp_retries1
> tcp_retries2
> tcp_rfc1337
> tcp_rmem
> tcp_sack
> tcp_stdurg
> tcp_synack_retries
> tcp_syncookies
> tcp_syn_retries
> tcp_timestamps
> tcp_tso_win_divisor
> tcp_tw_recycle
> tcp_tw_reuse
> tcp_window_scaling
> tcp_wmem
> linux:>$ cat tcp_fin_timeout
> 60
> linux:>$ cat tcp_keepalive_time
> 7200
> linux:>$ cat tcp_keepalive_probes
> 9
> linux:>$ cat tcp_keepalive_intvl
> 75
> linux:>$ cat tcp_fin_timeout
> 60
>
> > -----Original Message-----
> > From: Pete Keyes [mailto:PKeyes@starbucks.com]
> > Sent: Tuesday, September 04, 2007 4:21 PM
> > To: HttpClient User Discussion
> > Subject: RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
> >
> > I believe you problem has to do with a UNIX network (tcp)
> > configuration
> > setting. We've seen this often in volume testing. The socket is left
> > in this state for re-use...
> >
> > Not sure how the configuration is handled on Linux, but on Solaris you
> > can see all the configuration options with the following:
> > ndd -get /dev/tcp ?
> > tcp_time_wait_interval (read and write)
> > tcp_conn_req_max_q (read and write)
> > tcp_conn_req_max_q0 (read and write)
> > tcp_conn_req_min (read and write)
> > tcp_conn_grace_period (read and write)
> > tcp_cwnd_max (read and write)
> > tcp_debug (read and write)
> > tcp_smallest_nonpriv_port (read and write)
> > tcp_ip_abort_cinterval (read and write)
> > tcp_ip_abort_linterval (read and write)
> > tcp_ip_abort_interval (read and write)
> > tcp_ip_notify_cinterval (read and write)
> > tcp_ip_notify_interval (read and write)
> > tcp_ipv4_ttl (read and write)
> > tcp_keepalive_interval (read and write)
> > tcp_maxpsz_multiplier (read and write)
> > tcp_mss_def_ipv4 (read and write)
> > tcp_mss_max_ipv4 (read and write)
> > tcp_mss_min (read and write)
> > tcp_naglim_def (read and write)
> > tcp_rexmit_interval_initial (read and write)
> > tcp_rexmit_interval_max (read and write)
> > tcp_rexmit_interval_min (read and write)
> > tcp_deferred_ack_interval (read and write)
> > tcp_snd_lowat_fraction (read and write)
> > tcp_sth_rcv_hiwat (read and write)
> > tcp_sth_rcv_lowat (read and write)
> > tcp_dupack_fast_retransmit (read and write)
> > tcp_ignore_path_mtu (read and write)
> > tcp_rcv_push_wait (read and write)
> > tcp_smallest_anon_port (read and write)
> > tcp_largest_anon_port (read and write)
> > tcp_xmit_hiwat (read and write)
> > tcp_xmit_lowat (read and write)
> > tcp_recv_hiwat (read and write)
> > tcp_recv_hiwat_minmss (read and write)
> > tcp_fin_wait_2_flush_interval (read and write)
> > tcp_co_min (read and write)
> > tcp_max_buf (read and write)
> > tcp_strong_iss (read and write)
> > tcp_rtt_updates (read and write)
> > tcp_wscale_always (read and write)
> > tcp_tstamp_always (read and write)
> > tcp_tstamp_if_wscale (read and write)
> > tcp_rexmit_interval_extra (read and write)
> > tcp_deferred_acks_max (read and write)
> > tcp_slow_start_after_idle (read and write)
> > tcp_slow_start_initial (read and write)
> > tcp_co_timer_interval (read and write)
> > tcp_sack_permitted (read and write)
> > tcp_trace (read and write)
> > tcp_compression_enabled (read and write)
> > tcp_ipv6_hoplimit (read and write)
> > tcp_mss_def_ipv6 (read and write)
> > tcp_mss_max_ipv6 (read and write)
> > tcp_rev_src_routes (read and write)
> > tcp_ndd_get_info_interval (read and write)
> > tcp_rst_sent_rate_enabled (read and write)
> > tcp_rst_sent_rate (read and write)
> > tcp_use_smss_as_mss_opt (read and write)
> > tcp_wroff_xtra (read and write)
> > tcp_extra_priv_ports (read only)
> > tcp_extra_priv_ports_add (write only)
> > tcp_extra_priv_ports_del (write only)
> > tcp_status (read only)
> > tcp_bind_hash (read only)
> > tcp_listen_hash (read only)
> > tcp_conn_hash (read only)
> > tcp_acceptor_hash (read only)
> > tcp_host_param (read and write)
> > tcp_time_wait_stats (read only)
> > tcp_host_param_ipv6 (read and write)
> > tcp_1948_phrase (write only)
> > tcp_reserved_port_list (read only)
> > tcp_close_wait_interval(obsoleted- use
> > tcp_time_wait_interval) (no read
> > or write)
> >
> > ...Pete
> > Starbucks Coffee Co. - MS IT-5
> > 2401 Utah Ave S
> > Seattle, WA. 98134
> > (w) 206-318-5933
> > -----Original Message-----
> > From: Wittmann Armin [mailto:awittmann@ethz.ch]
> > Sent: Tuesday, September 04, 2007 4:23 AM
> > To: HttpClient User Discussion
> > Subject: after 60000 requests more than 700 sockets in CLOSE_WAIT
> >
> >
> > Hi httpclient-Team
> >
> > over the last 4 days I made a long term test for
> > our new software release.
> >
> > This software ist integrated into an Apache Tomcat 5.5
> > (as a servlet receiving requests, transform them and
> > sending out other GET-requests through http-client) and should
> > run very reliable and without any resource leaks.
> >
> > After finishing this test cycle I noticed that there
> > remained over 700 sockets in CLOSE_WAIT state
> > (Linux -> netstat -a -p).
> > Due to the identified PID and the destination ip number it is
> > obvious that these sockets have been caused/used
> > by the program part using http-client-3.0.1.
> >
> > I am not a real network crack so I don't know if I need
> > to bother about this. Since the software/tomcat is itended
> > to run months (7x24) without necessarily rebooting it
> > I am not shure if I will run out of network resources.
> >
> > Can somebody help in this subject?
> >
> > By the way: all 60000 http-GET-requests worked well and there
> > were no other problems at all.
> >
> > Regards
> >
> > Armin
> >
> > --------------------------------------------------------------------
> > My Code (simplified):
> > this code ist executed for every single request
> >
> > HttpConnectionManager connectionManager = new
> > SimpleHttpConnectionManager();
> > HttpClient client = new HttpClient(clientParams, connectionManager);
> > client.setHostConfiguration(hostConfiguration);
> > HttpMethod method = = new GetMethod();
> > method.setQueryString(pairs);
> > method.setPath(pUrl.getPath());
> > method.setParams(methodParams);
> > try {
> > client.executeMethod(method);
> > } catch (Exception e) {
> > failed = true;
> > throw new Exception(...);
> > } finally {
> > if (failed) method.abort();
> >
> > method.releaseConnection();
> >
> > client.setHttpConnectionManager(null);
> > client = null;
> > }
> >
> > try {
> > responseString = method.getResponseBodyAsString();
> > } catch (Exception e) {
> > throw new Exception(...);
> > } finally {
> > method.releaseConnection();
> > method = null;
> > }
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > httpclient-user-help@jakarta.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > httpclient-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: after 60000 requests more than 700 sockets in CLOSE_WAIT
Posted by Raymond Kroeker <ra...@thinkparity.com>.
Hi Armin,
If your requests are all coming from the same source to the same target,
and the target server supports the option; you might want to take a look at
using a keep alive header.
http://wiki.apache.org/jakarta-httpclient/ConnectionManagementDesign#head-9bd8b93d0d47d5391cff0db414e2593f4cd2f0a3
Raymond
On 9/4/07, Pete Keyes <PK...@starbucks.com> wrote:
>
> I think these 2 are the ones you want to look at tuning:
>
> # Solaris TCP_TIME_WAIT_INTERVAL
>
> * Description: Notifies TCP/IP on how long to keep the connection
> control blocks closed. After the applications complete the TCP/IP
> connection, the control blocks are kept for the specified time. When
> high connection rates occur, a large backlog of the TCP/IP connections
> accumulates and can slow server performance. The server can stall during
> certain peak periods. If the server stalls, the netstat command shows
> that many of the sockets that are opened to the HTTP server are in the
> CLOSE_WAIT or FIN_WAIT_2 state. Visible delays can occur for up to four
> minutes, during which time the server does not send any responses, but
> CPU utilization stays high, with all of the activities in system
> processes.
> * How to view or set: Use the get command to determine the current
> interval and the set command to specify an interval of 30 seconds. For
> example:
>
> ndd -get /dev/tcp tcp_time_wait_interval
> ndd -set /dev/tcp tcp_time_wait_interval 30000
>
> * Default value: The default time wait interval for a Solaris
> operating system is 240000 milliseconds, which is equal to 4 minutes.
> * Recommended value: 60000 milliseconds
>
> # Solaris TCP_FIN_WAIT_2_FLUSH_INTERVAL
>
> * Description: Specifies the timer interval prohibiting a connection
> in the FIN_WAIT_2 state to remain in that state. When high connection
> rates occur, a large backlog of TCP/IP connections accumulates and can
> slow server performance. The server can stall during peak periods. If
> the server stalls, using the netstat command shows that many of the
> sockets opened to the HTTP server are in the CLOSE_WAIT or FIN_WAIT_2
> state. Visible delays can occur for up to four minutes, during which
> time the server does not send any responses, but CPU utilization stays
> high, with all of the activity in system processes.
> * How to view and set: Use the get command to determine the current
> interval and the set command to specify an interval of 67.5 seconds. For
> example,
>
> ndd -get /dev/tcp tcp_fin_wait_2_flush_interval
> ndd -set /dev/tcp tcp_fin_wait_2_flush_interval 67500
>
> * Default value: 675000 milliseconds
> * Recommended value: 67500 milliseconds
>
>
> ...Pete
> Starbucks Coffee Co. - MS IT-5
> 2401 Utah Ave S
> Seattle, WA. 98134
> (w) 206-318-5933
>
> -----Original Message-----
> From: Wittmann Armin [mailto:awittmann@ethz.ch]
> Sent: Tuesday, September 04, 2007 8:16 AM
> To: HttpClient User Discussion
> Subject: RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
>
> Hi Pete
>
> under Linux it is found at a different place (in proc-fs). Look at the
> output below. I guess most settings are in seconds.
> I liked that one for newbies:
> http://www.utdallas.edu/~cantrell/ee6345/pocketguide.pdf
> http://ipsysctl-tutorial.frozentux.net/chunkyhtml/tcpvariables.html
>
> Any suggestions?
> 5 hours after the test stopped (tomcat is still running) none
> of the sockets has been freed.
>
> Thanks, Armin
>
> linux:>$ ls | grep tcp_
> tcp_abc
> tcp_abort_on_overflow
> tcp_adv_win_scale
> tcp_app_win
> tcp_congestion_control
> tcp_dsack
> tcp_ecn
> tcp_fack
> tcp_fin_timeout
> tcp_frto
> tcp_keepalive_intvl
> tcp_keepalive_probes
> tcp_keepalive_time
> tcp_low_latency
> tcp_max_orphans
> tcp_max_syn_backlog
> tcp_max_tw_buckets
> tcp_mem
> tcp_moderate_rcvbuf
> tcp_no_metrics_save
> tcp_orphan_retries
> tcp_reordering
> tcp_retrans_collapse
> tcp_retries1
> tcp_retries2
> tcp_rfc1337
> tcp_rmem
> tcp_sack
> tcp_stdurg
> tcp_synack_retries
> tcp_syncookies
> tcp_syn_retries
> tcp_timestamps
> tcp_tso_win_divisor
> tcp_tw_recycle
> tcp_tw_reuse
> tcp_window_scaling
> tcp_wmem
> linux:>$ cat tcp_fin_timeout
> 60
> linux:>$ cat tcp_keepalive_time
> 7200
> linux:>$ cat tcp_keepalive_probes
> 9
> linux:>$ cat tcp_keepalive_intvl
> 75
> linux:>$ cat tcp_fin_timeout
> 60
>
> > -----Original Message-----
> > From: Pete Keyes [mailto:PKeyes@starbucks.com]
> > Sent: Tuesday, September 04, 2007 4:21 PM
> > To: HttpClient User Discussion
> > Subject: RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
> >
> > I believe you problem has to do with a UNIX network (tcp)
> > configuration
> > setting. We've seen this often in volume testing. The socket is left
> > in this state for re-use...
> >
> > Not sure how the configuration is handled on Linux, but on Solaris you
> > can see all the configuration options with the following:
> > ndd -get /dev/tcp ?
> > tcp_time_wait_interval (read and write)
> > tcp_conn_req_max_q (read and write)
> > tcp_conn_req_max_q0 (read and write)
> > tcp_conn_req_min (read and write)
> > tcp_conn_grace_period (read and write)
> > tcp_cwnd_max (read and write)
> > tcp_debug (read and write)
> > tcp_smallest_nonpriv_port (read and write)
> > tcp_ip_abort_cinterval (read and write)
> > tcp_ip_abort_linterval (read and write)
> > tcp_ip_abort_interval (read and write)
> > tcp_ip_notify_cinterval (read and write)
> > tcp_ip_notify_interval (read and write)
> > tcp_ipv4_ttl (read and write)
> > tcp_keepalive_interval (read and write)
> > tcp_maxpsz_multiplier (read and write)
> > tcp_mss_def_ipv4 (read and write)
> > tcp_mss_max_ipv4 (read and write)
> > tcp_mss_min (read and write)
> > tcp_naglim_def (read and write)
> > tcp_rexmit_interval_initial (read and write)
> > tcp_rexmit_interval_max (read and write)
> > tcp_rexmit_interval_min (read and write)
> > tcp_deferred_ack_interval (read and write)
> > tcp_snd_lowat_fraction (read and write)
> > tcp_sth_rcv_hiwat (read and write)
> > tcp_sth_rcv_lowat (read and write)
> > tcp_dupack_fast_retransmit (read and write)
> > tcp_ignore_path_mtu (read and write)
> > tcp_rcv_push_wait (read and write)
> > tcp_smallest_anon_port (read and write)
> > tcp_largest_anon_port (read and write)
> > tcp_xmit_hiwat (read and write)
> > tcp_xmit_lowat (read and write)
> > tcp_recv_hiwat (read and write)
> > tcp_recv_hiwat_minmss (read and write)
> > tcp_fin_wait_2_flush_interval (read and write)
> > tcp_co_min (read and write)
> > tcp_max_buf (read and write)
> > tcp_strong_iss (read and write)
> > tcp_rtt_updates (read and write)
> > tcp_wscale_always (read and write)
> > tcp_tstamp_always (read and write)
> > tcp_tstamp_if_wscale (read and write)
> > tcp_rexmit_interval_extra (read and write)
> > tcp_deferred_acks_max (read and write)
> > tcp_slow_start_after_idle (read and write)
> > tcp_slow_start_initial (read and write)
> > tcp_co_timer_interval (read and write)
> > tcp_sack_permitted (read and write)
> > tcp_trace (read and write)
> > tcp_compression_enabled (read and write)
> > tcp_ipv6_hoplimit (read and write)
> > tcp_mss_def_ipv6 (read and write)
> > tcp_mss_max_ipv6 (read and write)
> > tcp_rev_src_routes (read and write)
> > tcp_ndd_get_info_interval (read and write)
> > tcp_rst_sent_rate_enabled (read and write)
> > tcp_rst_sent_rate (read and write)
> > tcp_use_smss_as_mss_opt (read and write)
> > tcp_wroff_xtra (read and write)
> > tcp_extra_priv_ports (read only)
> > tcp_extra_priv_ports_add (write only)
> > tcp_extra_priv_ports_del (write only)
> > tcp_status (read only)
> > tcp_bind_hash (read only)
> > tcp_listen_hash (read only)
> > tcp_conn_hash (read only)
> > tcp_acceptor_hash (read only)
> > tcp_host_param (read and write)
> > tcp_time_wait_stats (read only)
> > tcp_host_param_ipv6 (read and write)
> > tcp_1948_phrase (write only)
> > tcp_reserved_port_list (read only)
> > tcp_close_wait_interval(obsoleted- use
> > tcp_time_wait_interval) (no read
> > or write)
> >
> > ...Pete
> > Starbucks Coffee Co. - MS IT-5
> > 2401 Utah Ave S
> > Seattle, WA. 98134
> > (w) 206-318-5933
> > -----Original Message-----
> > From: Wittmann Armin [mailto:awittmann@ethz.ch]
> > Sent: Tuesday, September 04, 2007 4:23 AM
> > To: HttpClient User Discussion
> > Subject: after 60000 requests more than 700 sockets in CLOSE_WAIT
> >
> >
> > Hi httpclient-Team
> >
> > over the last 4 days I made a long term test for
> > our new software release.
> >
> > This software ist integrated into an Apache Tomcat 5.5
> > (as a servlet receiving requests, transform them and
> > sending out other GET-requests through http-client) and should
> > run very reliable and without any resource leaks.
> >
> > After finishing this test cycle I noticed that there
> > remained over 700 sockets in CLOSE_WAIT state
> > (Linux -> netstat -a -p).
> > Due to the identified PID and the destination ip number it is
> > obvious that these sockets have been caused/used
> > by the program part using http-client-3.0.1.
> >
> > I am not a real network crack so I don't know if I need
> > to bother about this. Since the software/tomcat is itended
> > to run months (7x24) without necessarily rebooting it
> > I am not shure if I will run out of network resources.
> >
> > Can somebody help in this subject?
> >
> > By the way: all 60000 http-GET-requests worked well and there
> > were no other problems at all.
> >
> > Regards
> >
> > Armin
> >
> > --------------------------------------------------------------------
> > My Code (simplified):
> > this code ist executed for every single request
> >
> > HttpConnectionManager connectionManager = new
> > SimpleHttpConnectionManager();
> > HttpClient client = new HttpClient(clientParams, connectionManager);
> > client.setHostConfiguration(hostConfiguration);
> > HttpMethod method = = new GetMethod();
> > method.setQueryString(pairs);
> > method.setPath(pUrl.getPath());
> > method.setParams(methodParams);
> > try {
> > client.executeMethod(method);
> > } catch (Exception e) {
> > failed = true;
> > throw new Exception(...);
> > } finally {
> > if (failed) method.abort();
> >
> > method.releaseConnection();
> >
> > client.setHttpConnectionManager(null);
> > client = null;
> > }
> >
> > try {
> > responseString = method.getResponseBodyAsString();
> > } catch (Exception e) {
> > throw new Exception(...);
> > } finally {
> > method.releaseConnection();
> > method = null;
> > }
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > httpclient-user-help@jakarta.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > httpclient-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>
--
--------------------------------------------------------------------------------
Raymond Kroeker
thinkParity Solutions Inc.
RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
Posted by Pete Keyes <PK...@starbucks.com>.
I think these 2 are the ones you want to look at tuning:
# Solaris TCP_TIME_WAIT_INTERVAL
* Description: Notifies TCP/IP on how long to keep the connection
control blocks closed. After the applications complete the TCP/IP
connection, the control blocks are kept for the specified time. When
high connection rates occur, a large backlog of the TCP/IP connections
accumulates and can slow server performance. The server can stall during
certain peak periods. If the server stalls, the netstat command shows
that many of the sockets that are opened to the HTTP server are in the
CLOSE_WAIT or FIN_WAIT_2 state. Visible delays can occur for up to four
minutes, during which time the server does not send any responses, but
CPU utilization stays high, with all of the activities in system
processes.
* How to view or set: Use the get command to determine the current
interval and the set command to specify an interval of 30 seconds. For
example:
ndd -get /dev/tcp tcp_time_wait_interval
ndd -set /dev/tcp tcp_time_wait_interval 30000
* Default value: The default time wait interval for a Solaris
operating system is 240000 milliseconds, which is equal to 4 minutes.
* Recommended value: 60000 milliseconds
# Solaris TCP_FIN_WAIT_2_FLUSH_INTERVAL
* Description: Specifies the timer interval prohibiting a connection
in the FIN_WAIT_2 state to remain in that state. When high connection
rates occur, a large backlog of TCP/IP connections accumulates and can
slow server performance. The server can stall during peak periods. If
the server stalls, using the netstat command shows that many of the
sockets opened to the HTTP server are in the CLOSE_WAIT or FIN_WAIT_2
state. Visible delays can occur for up to four minutes, during which
time the server does not send any responses, but CPU utilization stays
high, with all of the activity in system processes.
* How to view and set: Use the get command to determine the current
interval and the set command to specify an interval of 67.5 seconds. For
example,
ndd -get /dev/tcp tcp_fin_wait_2_flush_interval
ndd -set /dev/tcp tcp_fin_wait_2_flush_interval 67500
* Default value: 675000 milliseconds
* Recommended value: 67500 milliseconds
...Pete
Starbucks Coffee Co. - MS IT-5
2401 Utah Ave S
Seattle, WA. 98134
(w) 206-318-5933
-----Original Message-----
From: Wittmann Armin [mailto:awittmann@ethz.ch]
Sent: Tuesday, September 04, 2007 8:16 AM
To: HttpClient User Discussion
Subject: RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
Hi Pete
under Linux it is found at a different place (in proc-fs). Look at the
output below. I guess most settings are in seconds.
I liked that one for newbies:
http://www.utdallas.edu/~cantrell/ee6345/pocketguide.pdf
http://ipsysctl-tutorial.frozentux.net/chunkyhtml/tcpvariables.html
Any suggestions?
5 hours after the test stopped (tomcat is still running) none
of the sockets has been freed.
Thanks, Armin
linux:>$ ls | grep tcp_
tcp_abc
tcp_abort_on_overflow
tcp_adv_win_scale
tcp_app_win
tcp_congestion_control
tcp_dsack
tcp_ecn
tcp_fack
tcp_fin_timeout
tcp_frto
tcp_keepalive_intvl
tcp_keepalive_probes
tcp_keepalive_time
tcp_low_latency
tcp_max_orphans
tcp_max_syn_backlog
tcp_max_tw_buckets
tcp_mem
tcp_moderate_rcvbuf
tcp_no_metrics_save
tcp_orphan_retries
tcp_reordering
tcp_retrans_collapse
tcp_retries1
tcp_retries2
tcp_rfc1337
tcp_rmem
tcp_sack
tcp_stdurg
tcp_synack_retries
tcp_syncookies
tcp_syn_retries
tcp_timestamps
tcp_tso_win_divisor
tcp_tw_recycle
tcp_tw_reuse
tcp_window_scaling
tcp_wmem
linux:>$ cat tcp_fin_timeout
60
linux:>$ cat tcp_keepalive_time
7200
linux:>$ cat tcp_keepalive_probes
9
linux:>$ cat tcp_keepalive_intvl
75
linux:>$ cat tcp_fin_timeout
60
> -----Original Message-----
> From: Pete Keyes [mailto:PKeyes@starbucks.com]
> Sent: Tuesday, September 04, 2007 4:21 PM
> To: HttpClient User Discussion
> Subject: RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
>
> I believe you problem has to do with a UNIX network (tcp)
> configuration
> setting. We've seen this often in volume testing. The socket is left
> in this state for re-use...
>
> Not sure how the configuration is handled on Linux, but on Solaris you
> can see all the configuration options with the following:
> ndd -get /dev/tcp ?
> tcp_time_wait_interval (read and write)
> tcp_conn_req_max_q (read and write)
> tcp_conn_req_max_q0 (read and write)
> tcp_conn_req_min (read and write)
> tcp_conn_grace_period (read and write)
> tcp_cwnd_max (read and write)
> tcp_debug (read and write)
> tcp_smallest_nonpriv_port (read and write)
> tcp_ip_abort_cinterval (read and write)
> tcp_ip_abort_linterval (read and write)
> tcp_ip_abort_interval (read and write)
> tcp_ip_notify_cinterval (read and write)
> tcp_ip_notify_interval (read and write)
> tcp_ipv4_ttl (read and write)
> tcp_keepalive_interval (read and write)
> tcp_maxpsz_multiplier (read and write)
> tcp_mss_def_ipv4 (read and write)
> tcp_mss_max_ipv4 (read and write)
> tcp_mss_min (read and write)
> tcp_naglim_def (read and write)
> tcp_rexmit_interval_initial (read and write)
> tcp_rexmit_interval_max (read and write)
> tcp_rexmit_interval_min (read and write)
> tcp_deferred_ack_interval (read and write)
> tcp_snd_lowat_fraction (read and write)
> tcp_sth_rcv_hiwat (read and write)
> tcp_sth_rcv_lowat (read and write)
> tcp_dupack_fast_retransmit (read and write)
> tcp_ignore_path_mtu (read and write)
> tcp_rcv_push_wait (read and write)
> tcp_smallest_anon_port (read and write)
> tcp_largest_anon_port (read and write)
> tcp_xmit_hiwat (read and write)
> tcp_xmit_lowat (read and write)
> tcp_recv_hiwat (read and write)
> tcp_recv_hiwat_minmss (read and write)
> tcp_fin_wait_2_flush_interval (read and write)
> tcp_co_min (read and write)
> tcp_max_buf (read and write)
> tcp_strong_iss (read and write)
> tcp_rtt_updates (read and write)
> tcp_wscale_always (read and write)
> tcp_tstamp_always (read and write)
> tcp_tstamp_if_wscale (read and write)
> tcp_rexmit_interval_extra (read and write)
> tcp_deferred_acks_max (read and write)
> tcp_slow_start_after_idle (read and write)
> tcp_slow_start_initial (read and write)
> tcp_co_timer_interval (read and write)
> tcp_sack_permitted (read and write)
> tcp_trace (read and write)
> tcp_compression_enabled (read and write)
> tcp_ipv6_hoplimit (read and write)
> tcp_mss_def_ipv6 (read and write)
> tcp_mss_max_ipv6 (read and write)
> tcp_rev_src_routes (read and write)
> tcp_ndd_get_info_interval (read and write)
> tcp_rst_sent_rate_enabled (read and write)
> tcp_rst_sent_rate (read and write)
> tcp_use_smss_as_mss_opt (read and write)
> tcp_wroff_xtra (read and write)
> tcp_extra_priv_ports (read only)
> tcp_extra_priv_ports_add (write only)
> tcp_extra_priv_ports_del (write only)
> tcp_status (read only)
> tcp_bind_hash (read only)
> tcp_listen_hash (read only)
> tcp_conn_hash (read only)
> tcp_acceptor_hash (read only)
> tcp_host_param (read and write)
> tcp_time_wait_stats (read only)
> tcp_host_param_ipv6 (read and write)
> tcp_1948_phrase (write only)
> tcp_reserved_port_list (read only)
> tcp_close_wait_interval(obsoleted- use
> tcp_time_wait_interval) (no read
> or write)
>
> ...Pete
> Starbucks Coffee Co. - MS IT-5
> 2401 Utah Ave S
> Seattle, WA. 98134
> (w) 206-318-5933
> -----Original Message-----
> From: Wittmann Armin [mailto:awittmann@ethz.ch]
> Sent: Tuesday, September 04, 2007 4:23 AM
> To: HttpClient User Discussion
> Subject: after 60000 requests more than 700 sockets in CLOSE_WAIT
>
>
> Hi httpclient-Team
>
> over the last 4 days I made a long term test for
> our new software release.
>
> This software ist integrated into an Apache Tomcat 5.5
> (as a servlet receiving requests, transform them and
> sending out other GET-requests through http-client) and should
> run very reliable and without any resource leaks.
>
> After finishing this test cycle I noticed that there
> remained over 700 sockets in CLOSE_WAIT state
> (Linux -> netstat -a -p).
> Due to the identified PID and the destination ip number it is
> obvious that these sockets have been caused/used
> by the program part using http-client-3.0.1.
>
> I am not a real network crack so I don't know if I need
> to bother about this. Since the software/tomcat is itended
> to run months (7x24) without necessarily rebooting it
> I am not shure if I will run out of network resources.
>
> Can somebody help in this subject?
>
> By the way: all 60000 http-GET-requests worked well and there
> were no other problems at all.
>
> Regards
>
> Armin
>
> --------------------------------------------------------------------
> My Code (simplified):
> this code ist executed for every single request
>
> HttpConnectionManager connectionManager = new
> SimpleHttpConnectionManager();
> HttpClient client = new HttpClient(clientParams, connectionManager);
> client.setHostConfiguration(hostConfiguration);
> HttpMethod method = = new GetMethod();
> method.setQueryString(pairs);
> method.setPath(pUrl.getPath());
> method.setParams(methodParams);
> try {
> client.executeMethod(method);
> } catch (Exception e) {
> failed = true;
> throw new Exception(...);
> } finally {
> if (failed) method.abort();
>
> method.releaseConnection();
>
> client.setHttpConnectionManager(null);
> client = null;
> }
>
> try {
> responseString = method.getResponseBodyAsString();
> } catch (Exception e) {
> throw new Exception(...);
> } finally {
> method.releaseConnection();
> method = null;
> }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> httpclient-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> httpclient-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
Posted by Wittmann Armin <aw...@ethz.ch>.
Hi Pete
under Linux it is found at a different place (in proc-fs). Look at the
output below. I guess most settings are in seconds.
I liked that one for newbies:
http://www.utdallas.edu/~cantrell/ee6345/pocketguide.pdf
http://ipsysctl-tutorial.frozentux.net/chunkyhtml/tcpvariables.html
Any suggestions?
5 hours after the test stopped (tomcat is still running) none
of the sockets has been freed.
Thanks, Armin
linux:>$ ls | grep tcp_
tcp_abc
tcp_abort_on_overflow
tcp_adv_win_scale
tcp_app_win
tcp_congestion_control
tcp_dsack
tcp_ecn
tcp_fack
tcp_fin_timeout
tcp_frto
tcp_keepalive_intvl
tcp_keepalive_probes
tcp_keepalive_time
tcp_low_latency
tcp_max_orphans
tcp_max_syn_backlog
tcp_max_tw_buckets
tcp_mem
tcp_moderate_rcvbuf
tcp_no_metrics_save
tcp_orphan_retries
tcp_reordering
tcp_retrans_collapse
tcp_retries1
tcp_retries2
tcp_rfc1337
tcp_rmem
tcp_sack
tcp_stdurg
tcp_synack_retries
tcp_syncookies
tcp_syn_retries
tcp_timestamps
tcp_tso_win_divisor
tcp_tw_recycle
tcp_tw_reuse
tcp_window_scaling
tcp_wmem
linux:>$ cat tcp_fin_timeout
60
linux:>$ cat tcp_keepalive_time
7200
linux:>$ cat tcp_keepalive_probes
9
linux:>$ cat tcp_keepalive_intvl
75
linux:>$ cat tcp_fin_timeout
60
> -----Original Message-----
> From: Pete Keyes [mailto:PKeyes@starbucks.com]
> Sent: Tuesday, September 04, 2007 4:21 PM
> To: HttpClient User Discussion
> Subject: RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
>
> I believe you problem has to do with a UNIX network (tcp)
> configuration
> setting. We've seen this often in volume testing. The socket is left
> in this state for re-use...
>
> Not sure how the configuration is handled on Linux, but on Solaris you
> can see all the configuration options with the following:
> ndd -get /dev/tcp ?
> tcp_time_wait_interval (read and write)
> tcp_conn_req_max_q (read and write)
> tcp_conn_req_max_q0 (read and write)
> tcp_conn_req_min (read and write)
> tcp_conn_grace_period (read and write)
> tcp_cwnd_max (read and write)
> tcp_debug (read and write)
> tcp_smallest_nonpriv_port (read and write)
> tcp_ip_abort_cinterval (read and write)
> tcp_ip_abort_linterval (read and write)
> tcp_ip_abort_interval (read and write)
> tcp_ip_notify_cinterval (read and write)
> tcp_ip_notify_interval (read and write)
> tcp_ipv4_ttl (read and write)
> tcp_keepalive_interval (read and write)
> tcp_maxpsz_multiplier (read and write)
> tcp_mss_def_ipv4 (read and write)
> tcp_mss_max_ipv4 (read and write)
> tcp_mss_min (read and write)
> tcp_naglim_def (read and write)
> tcp_rexmit_interval_initial (read and write)
> tcp_rexmit_interval_max (read and write)
> tcp_rexmit_interval_min (read and write)
> tcp_deferred_ack_interval (read and write)
> tcp_snd_lowat_fraction (read and write)
> tcp_sth_rcv_hiwat (read and write)
> tcp_sth_rcv_lowat (read and write)
> tcp_dupack_fast_retransmit (read and write)
> tcp_ignore_path_mtu (read and write)
> tcp_rcv_push_wait (read and write)
> tcp_smallest_anon_port (read and write)
> tcp_largest_anon_port (read and write)
> tcp_xmit_hiwat (read and write)
> tcp_xmit_lowat (read and write)
> tcp_recv_hiwat (read and write)
> tcp_recv_hiwat_minmss (read and write)
> tcp_fin_wait_2_flush_interval (read and write)
> tcp_co_min (read and write)
> tcp_max_buf (read and write)
> tcp_strong_iss (read and write)
> tcp_rtt_updates (read and write)
> tcp_wscale_always (read and write)
> tcp_tstamp_always (read and write)
> tcp_tstamp_if_wscale (read and write)
> tcp_rexmit_interval_extra (read and write)
> tcp_deferred_acks_max (read and write)
> tcp_slow_start_after_idle (read and write)
> tcp_slow_start_initial (read and write)
> tcp_co_timer_interval (read and write)
> tcp_sack_permitted (read and write)
> tcp_trace (read and write)
> tcp_compression_enabled (read and write)
> tcp_ipv6_hoplimit (read and write)
> tcp_mss_def_ipv6 (read and write)
> tcp_mss_max_ipv6 (read and write)
> tcp_rev_src_routes (read and write)
> tcp_ndd_get_info_interval (read and write)
> tcp_rst_sent_rate_enabled (read and write)
> tcp_rst_sent_rate (read and write)
> tcp_use_smss_as_mss_opt (read and write)
> tcp_wroff_xtra (read and write)
> tcp_extra_priv_ports (read only)
> tcp_extra_priv_ports_add (write only)
> tcp_extra_priv_ports_del (write only)
> tcp_status (read only)
> tcp_bind_hash (read only)
> tcp_listen_hash (read only)
> tcp_conn_hash (read only)
> tcp_acceptor_hash (read only)
> tcp_host_param (read and write)
> tcp_time_wait_stats (read only)
> tcp_host_param_ipv6 (read and write)
> tcp_1948_phrase (write only)
> tcp_reserved_port_list (read only)
> tcp_close_wait_interval(obsoleted- use
> tcp_time_wait_interval) (no read
> or write)
>
> ...Pete
> Starbucks Coffee Co. - MS IT-5
> 2401 Utah Ave S
> Seattle, WA. 98134
> (w) 206-318-5933
> -----Original Message-----
> From: Wittmann Armin [mailto:awittmann@ethz.ch]
> Sent: Tuesday, September 04, 2007 4:23 AM
> To: HttpClient User Discussion
> Subject: after 60000 requests more than 700 sockets in CLOSE_WAIT
>
>
> Hi httpclient-Team
>
> over the last 4 days I made a long term test for
> our new software release.
>
> This software ist integrated into an Apache Tomcat 5.5
> (as a servlet receiving requests, transform them and
> sending out other GET-requests through http-client) and should
> run very reliable and without any resource leaks.
>
> After finishing this test cycle I noticed that there
> remained over 700 sockets in CLOSE_WAIT state
> (Linux -> netstat -a -p).
> Due to the identified PID and the destination ip number it is
> obvious that these sockets have been caused/used
> by the program part using http-client-3.0.1.
>
> I am not a real network crack so I don't know if I need
> to bother about this. Since the software/tomcat is itended
> to run months (7x24) without necessarily rebooting it
> I am not shure if I will run out of network resources.
>
> Can somebody help in this subject?
>
> By the way: all 60000 http-GET-requests worked well and there
> were no other problems at all.
>
> Regards
>
> Armin
>
> --------------------------------------------------------------------
> My Code (simplified):
> this code ist executed for every single request
>
> HttpConnectionManager connectionManager = new
> SimpleHttpConnectionManager();
> HttpClient client = new HttpClient(clientParams, connectionManager);
> client.setHostConfiguration(hostConfiguration);
> HttpMethod method = = new GetMethod();
> method.setQueryString(pairs);
> method.setPath(pUrl.getPath());
> method.setParams(methodParams);
> try {
> client.executeMethod(method);
> } catch (Exception e) {
> failed = true;
> throw new Exception(...);
> } finally {
> if (failed) method.abort();
>
> method.releaseConnection();
>
> client.setHttpConnectionManager(null);
> client = null;
> }
>
> try {
> responseString = method.getResponseBodyAsString();
> } catch (Exception e) {
> throw new Exception(...);
> } finally {
> method.releaseConnection();
> method = null;
> }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> httpclient-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> httpclient-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
RE: after 60000 requests more than 700 sockets in CLOSE_WAIT
Posted by Pete Keyes <PK...@starbucks.com>.
I believe you problem has to do with a UNIX network (tcp) configuration
setting. We've seen this often in volume testing. The socket is left
in this state for re-use...
Not sure how the configuration is handled on Linux, but on Solaris you
can see all the configuration options with the following:
ndd -get /dev/tcp ?
tcp_time_wait_interval (read and write)
tcp_conn_req_max_q (read and write)
tcp_conn_req_max_q0 (read and write)
tcp_conn_req_min (read and write)
tcp_conn_grace_period (read and write)
tcp_cwnd_max (read and write)
tcp_debug (read and write)
tcp_smallest_nonpriv_port (read and write)
tcp_ip_abort_cinterval (read and write)
tcp_ip_abort_linterval (read and write)
tcp_ip_abort_interval (read and write)
tcp_ip_notify_cinterval (read and write)
tcp_ip_notify_interval (read and write)
tcp_ipv4_ttl (read and write)
tcp_keepalive_interval (read and write)
tcp_maxpsz_multiplier (read and write)
tcp_mss_def_ipv4 (read and write)
tcp_mss_max_ipv4 (read and write)
tcp_mss_min (read and write)
tcp_naglim_def (read and write)
tcp_rexmit_interval_initial (read and write)
tcp_rexmit_interval_max (read and write)
tcp_rexmit_interval_min (read and write)
tcp_deferred_ack_interval (read and write)
tcp_snd_lowat_fraction (read and write)
tcp_sth_rcv_hiwat (read and write)
tcp_sth_rcv_lowat (read and write)
tcp_dupack_fast_retransmit (read and write)
tcp_ignore_path_mtu (read and write)
tcp_rcv_push_wait (read and write)
tcp_smallest_anon_port (read and write)
tcp_largest_anon_port (read and write)
tcp_xmit_hiwat (read and write)
tcp_xmit_lowat (read and write)
tcp_recv_hiwat (read and write)
tcp_recv_hiwat_minmss (read and write)
tcp_fin_wait_2_flush_interval (read and write)
tcp_co_min (read and write)
tcp_max_buf (read and write)
tcp_strong_iss (read and write)
tcp_rtt_updates (read and write)
tcp_wscale_always (read and write)
tcp_tstamp_always (read and write)
tcp_tstamp_if_wscale (read and write)
tcp_rexmit_interval_extra (read and write)
tcp_deferred_acks_max (read and write)
tcp_slow_start_after_idle (read and write)
tcp_slow_start_initial (read and write)
tcp_co_timer_interval (read and write)
tcp_sack_permitted (read and write)
tcp_trace (read and write)
tcp_compression_enabled (read and write)
tcp_ipv6_hoplimit (read and write)
tcp_mss_def_ipv6 (read and write)
tcp_mss_max_ipv6 (read and write)
tcp_rev_src_routes (read and write)
tcp_ndd_get_info_interval (read and write)
tcp_rst_sent_rate_enabled (read and write)
tcp_rst_sent_rate (read and write)
tcp_use_smss_as_mss_opt (read and write)
tcp_wroff_xtra (read and write)
tcp_extra_priv_ports (read only)
tcp_extra_priv_ports_add (write only)
tcp_extra_priv_ports_del (write only)
tcp_status (read only)
tcp_bind_hash (read only)
tcp_listen_hash (read only)
tcp_conn_hash (read only)
tcp_acceptor_hash (read only)
tcp_host_param (read and write)
tcp_time_wait_stats (read only)
tcp_host_param_ipv6 (read and write)
tcp_1948_phrase (write only)
tcp_reserved_port_list (read only)
tcp_close_wait_interval(obsoleted- use tcp_time_wait_interval) (no read
or write)
...Pete
Starbucks Coffee Co. - MS IT-5
2401 Utah Ave S
Seattle, WA. 98134
(w) 206-318-5933
-----Original Message-----
From: Wittmann Armin [mailto:awittmann@ethz.ch]
Sent: Tuesday, September 04, 2007 4:23 AM
To: HttpClient User Discussion
Subject: after 60000 requests more than 700 sockets in CLOSE_WAIT
Hi httpclient-Team
over the last 4 days I made a long term test for
our new software release.
This software ist integrated into an Apache Tomcat 5.5
(as a servlet receiving requests, transform them and
sending out other GET-requests through http-client) and should
run very reliable and without any resource leaks.
After finishing this test cycle I noticed that there
remained over 700 sockets in CLOSE_WAIT state
(Linux -> netstat -a -p).
Due to the identified PID and the destination ip number it is
obvious that these sockets have been caused/used
by the program part using http-client-3.0.1.
I am not a real network crack so I don't know if I need
to bother about this. Since the software/tomcat is itended
to run months (7x24) without necessarily rebooting it
I am not shure if I will run out of network resources.
Can somebody help in this subject?
By the way: all 60000 http-GET-requests worked well and there
were no other problems at all.
Regards
Armin
--------------------------------------------------------------------
My Code (simplified):
this code ist executed for every single request
HttpConnectionManager connectionManager = new
SimpleHttpConnectionManager();
HttpClient client = new HttpClient(clientParams, connectionManager);
client.setHostConfiguration(hostConfiguration);
HttpMethod method = = new GetMethod();
method.setQueryString(pairs);
method.setPath(pUrl.getPath());
method.setParams(methodParams);
try {
client.executeMethod(method);
} catch (Exception e) {
failed = true;
throw new Exception(...);
} finally {
if (failed) method.abort();
method.releaseConnection();
client.setHttpConnectionManager(null);
client = null;
}
try {
responseString = method.getResponseBodyAsString();
} catch (Exception e) {
throw new Exception(...);
} finally {
method.releaseConnection();
method = null;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org