You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Eric Bloch <bl...@laszlosystems.com> on 2004/03/19 19:21:39 UTC
Re: Memory Leaks when web server hangs
Hey there,
I create/destroy http clients but always have them use the same
connection manager.
Will that cause thread thrashing?
Thanks,
-Eric
Roland Weber wrote:
>Hello Srini,
>
>you should *not* create a new HTTP Client for each request!
>This will also create a new connection manager, and a new
>cleanup thread. It beats the whole purpose of connection
>management.
>
>You should create *one* HTTP Client object for the lifetime
>of your application. Then, the connection manager of that
>client will reuse and free connections as appropriate.
>
>best regards,
> Roland
>
>
>
>
>
>
>Srinivas Vemula <sr...@mensamind.com>
>02.02.2004 11:56
>Please respond to "Commons HttpClient Project"
>
> To: Commons HttpClient Project
><co...@jakarta.apache.org>
> cc:
> Subject: Re: Memory Leaks when web server hangs
>
>
>Oleg,
>
> We are using JDK1.4.1 and connection time out is set to 30Ms and
>read time out to 60Ms. We are trying to connect to a IP Camera, and send
>a heart beat command to the camera to check for its availability every
>30 Secs from the time we find it is un available. If the web server is
>down, we feel that HttpClient is not cleaning up failed connections
>properly and thats resulting in Memory Leaks at the server side.
>
> The code we use is the standard way of using HttpClient and all the
>code is in a method and new HttpClient object is created for every
>request. Are there any ways to make sure the connections are all being
>purged properly? Are there any precautionary measures or flags we can
>set on HttpClient API when communicating with a web server (running on
>firm ware) with relatively less RAM and processing power (IpCamera with
>built in web server)?
>
>Thanks for your time and help.
>Srini
>
>Kalnichevski, Oleg wrote:
>
>
>
>>>Mike and Oleg, the stack trace Srini included indicates that HttpClient
>>>
>>>
>is
>
>
>>>attempting to create a new timeout thread on every connection - does
>>>
>>>
>this
>
>
>>>always occur or is it just one timeout thread per httpclient instance?
>>>
>>>
>It
>
>
>>>would be ideal if we could reduce this to one static thread as starting
>>>threads is never nice for server side apps. Not sure how feasible that
>>>
>>>
>is
>
>
>>>though.
>>>
>>>
>>>
>>>
>>Hi Srini,
>>HttpClient uses an additional controller thread to work around the
>>
>>
>limitation of older (< 1.4) JDKs which do not provide a possibility to set
>connect timeout. If you do not really need to control connect timeout (for
>instance, when communicating with an intranet site with good availability)
>simply set connect timeout to. That will prevent HttpClient from spawning
>an additional thread per request.
>
>
>>Adrian, et al
>>Another possibility to use reflection to set connect timeout using the
>>
>>
>Socket methods when running in JVM 1.4 or above
>
>
>>Oleg
>>
>>
>>
>>-----Original Message-----
>>From: Adrian Sutton [mailto:adrian@intencha.com]
>>Sent: Monday, February 02, 2004 09:03
>>To: Commons HttpClient Project
>>Subject: Re: Memory Leaks when web server hangs
>>
>>
>>On 2/2/04 2:00 PM, "Srinivas Vemula" <sr...@mensamind.com>
>>
>>
>wrote:
>
>
>>
>>
>>
>>>Hi All,
>>> We are seeing thread leaks when having client open connections to
>>>
>>>
>a web
>
>
>>>server that hangs. Has any one seen this happening?? How do we ensure
>>>
>>>
>that the
>
>
>>>library correctly closes socket connections on failures, cleaning up
>>>
>>>
>system
>
>
>>>resources, and threads actually finish in the timeout period and get
>>>
>>>
>freed
>
>
>>>up. Would using MultiThreadedHttpConnectionManager
>>><
>>>
>>>
>file:///D:/silkroad/http-commons/commons-httpclient-2.0-rc3/docs/threading.ht
>
>
>
>>>ml#MultiThreadedHttpConnectionManager> be of any help??
>>>
>>>
>>>
>>>
>>Hi Srini,
>>If you're using the same HttpClient instance across multiple threads, you
>>must use the MultiThreadedHttpConnectionManager or you'll run into
>>
>>
>strange
>
>
>>problems. If you're using separate HttpClient instances, you may as well
>>stick with the single threaded (default) connection manager.
>>
>>In terms of connections hanging - you probably want to look into the
>>setConnectionTimeout and setTimeout methods of the HttpClient class.
>>
>>
>These
>
>
>>allow you to control how long HttpClient waits when making a connection
>>
>>
>and
>
>
>>how long it waits for data once the connection is established. If you
>>
>>
>have
>
>
>>set either of these to 0 (not sure what the default is, it may be
>>
>>
>platform
>
>
>>specific) the connection will never timeout which sounds a lot like what
>>you're seeing.
>>
>>Mike and Oleg, the stack trace Srini included indicates that HttpClient
>>
>>
>is
>
>
>>attempting to create a new timeout thread on every connection - does this
>>always occur or is it just one timeout thread per httpclient instance? It
>>would be ideal if we could reduce this to one static thread as starting
>>threads is never nice for server side apps. Not sure how feasible that
>>
>>
>is
>
>
>>though.
>>
>>
>>
>>
>>
>>>Srini
>>>
>>>
>>>
>>>
>>Regards,
>>
>>Adrian Sutton.
>>
>>----------------------------------------------
>>Intencha "tomorrow's technology today"
>>Ph: 38478913 0422236329
>>Suite 8/29 Oatland Crescent
>>Holland Park West 4121
>>Australia QLD
>>www.intencha.com
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail:
>>
>>
>commons-httpclient-dev-unsubscribe@jakarta.apache.org
>
>
>>For additional commands, e-mail:
>>
>>
>commons-httpclient-dev-help@jakarta.apache.org
>
>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail:
>>
>>
>commons-httpclient-dev-unsubscribe@jakarta.apache.org
>
>
>>For additional commands, e-mail:
>>
>>
>commons-httpclient-dev-help@jakarta.apache.org
>
>
>>
>>
>>
>
>
>
--
Laszlo Systems, Inc.
1040 Mariposa Street, SF, CA, USA 94107
voice: 415.241.2721
fax: 415.865.2914
email: bloch@laszlosystems.com
--
Laszlo allows Behr to deliver a breakthrough experience with
ColorSmart by BEHR application at http://www.behr.com
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: Memory Leaks when web server hangs
Posted by Michael Becke <be...@u.washington.edu>.
Hi Eric,
Reusing the MultiThreadedHttpConnectionManager is definitely the way to
go. As Roland mentions reusing HttpClient instances might help
performance, but it is not necessary.
Mike
On Mar 22, 2004, at 2:01 AM, Eric Bloch wrote:
> I'm happy to change my code but have a version of my product out there
> that does it this way and wanted to make sure it wasn't in need of
> immediate patch :-)
>
> -E
>
>
> Roland Weber wrote:
>
>> Hello Eric,
>>
>> by using the same connection manager for each HttpClient,
>> you avoid the biggest penalty for creating new clients. But
>> still, each new client uses up some heap space, which will
>> have to be garbage collected sooner or later. There are
>> state objects, and parameter objecs, and probably some
>> more that will be created. Nothing that hurts much, but what
>> gain do you expect from creating new HttpClient objects?
>> You don't have to.
>>
>> cheers,
>> Roland
>>
>>
>>
>>
>>
>>
>> Eric Bloch <bl...@laszlosystems.com>
>> 22.03.2004 07:03
>> Please respond to "Commons HttpClient Project"
>> To: Commons HttpClient Project
>> <co...@jakarta.apache.org>
>> cc: Subject: Re: Memory Leaks when web server
>> hangs
>>
>>
>> Oh... and I just want to know if that's "not the httpclient way",
>> too? That is, should I be reusing the client? What will that get me
>> if I'm already reusing the connection manager?
>>
>> Thanks again,
>> Eric
>>
>> Eric Bloch wrote:
>>
>>
>>> I am creating one MultiThreadedHttpConnectionManager and
>>> creating/destroying HttpClient each time but always constructing
>>> with the same connection manager.
>>> Thanks,
>>> Eric
>>>
>>>
>>> Michael Becke wrote:
>>>
>>>
>>>> Hi Eric,
>>>>
>>>> What exactly do you mean by "thread thrashing"? Which connection
>>>> manager are you using?
>>>>
>>>> Mike
>>>>
>>>> On Mar 19, 2004, at 1:21 PM, Eric Bloch wrote:
>>>>
>>>>
>>>>> Hey there,
>>>>>
>>>>> I create/destroy http clients but always have them use the same
>>>>> connection manager.
>>>>>
>>>>> Will that cause thread thrashing?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Eric
>>>>>
>>>>> Roland Weber wrote:
>>>>>
>>>>>
>>>>>> Hello Srini,
>>>>>>
>>>>>> you should *not* create a new HTTP Client for each request!
>>>>>> This will also create a new connection manager, and a new
>>>>>> cleanup thread. It beats the whole purpose of connection
>>>>>> management.
>>>>>>
>>>>>> You should create *one* HTTP Client object for the lifetime
>>>>>> of your application. Then, the connection manager of that
>>>>>> client will reuse and free connections as appropriate.
>>>>>>
>>>>>> best regards,
>>>>>> Roland
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Srinivas Vemula <sr...@mensamind.com>
>>>>>> 02.02.2004 11:56
>>>>>> Please respond to "Commons HttpClient Project"
>>>>>> To: Commons HttpClient Project
>>>>>> <co...@jakarta.apache.org>
>>>>>> cc: Subject: Re: Memory Leaks when web server
>>>>>> hangs
>>>>>>
>>>>>>
>>>>>> Oleg,
>>>>>> We are using JDK1.4.1 and connection time out is set to 30Ms
>>>>>> and read time out to 60Ms. We are trying to connect to a IP
>>>>>> Camera, and send a heart beat command to the camera to check for
>>>>>> its availability every 30 Secs from the time we find it is un
>>>>>> available. If the web server is down, we feel that HttpClient is
>>>>>> not cleaning up failed connections properly and thats resulting
>>>>>> in Memory Leaks at the server side.
>>>>>>
>>>>>> The code we use is the standard way of using HttpClient and all
>>>>>> the code is in a method and new HttpClient object is created for
>>>>>> every request. Are there any ways to make sure the connections
>>>>>> are all being purged properly? Are there any precautionary
>>>>>> measures or flags we can set on HttpClient API when communicating
>>>>>> with a web server (running on firm ware) with relatively less RAM
>>>>>> and processing power (IpCamera with built in web server)?
>>>>>>
>>>>>> Thanks for your time and help.
>>>>>> Srini
>>>>>>
>>>>>> Kalnichevski, Oleg wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>>>>>> HttpClient
>>>>>>>>
>>>>>>>
>>>>>> is
>>>>>>
>>>>>>
>>>>>>>> attempting to create a new timeout thread on every connection -
>> does
>>
>>>>>>>
>>>>>> this
>>>>>>
>>>>>>
>>>>>>>> always occur or is it just one timeout thread per httpclient
>>>>>>>> instance?
>>>>>>>>
>>>>>>>
>>>>>> It
>>>>>>
>>>>>>
>>>>>>>> would be ideal if we could reduce this to one static thread as
>>>>>>>> starting
>>>>>>>> threads is never nice for server side apps. Not sure how
>>>>>>>> feasible that
>>>>>>>>
>>>>>>>
>>>>>> is
>>>>>>
>>>>>>
>>>>>>>> though.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Hi Srini,
>>>>>>> HttpClient uses an additional controller thread to work around
>>>>>>> the
>>>>>>>
>>>>>> limitation of older (< 1.4) JDKs which do not provide a
>>>>>> possibility to set connect timeout. If you do not really need to
>>>>>> control connect timeout (for instance, when communicating with
>>>>>> an intranet site with good availability) simply set connect
>>>>>> timeout to. That will prevent HttpClient from spawning an
>>>>>> additional thread per request.
>>>>>>
>>>>>>
>>>>>>> Adrian, et al
>>>>>>> Another possibility to use reflection to set connect timeout
>>>>>>> using the
>>>>>>>
>>>>>> Socket methods when running in JVM 1.4 or above
>>>>>>
>>>>>>
>>>>>>> Oleg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Adrian Sutton [mailto:adrian@intencha.com]
>>>>>>> Sent: Monday, February 02, 2004 09:03
>>>>>>> To: Commons HttpClient Project
>>>>>>> Subject: Re: Memory Leaks when web server hangs
>>>>>>>
>>>>>>>
>>>>>>> On 2/2/04 2:00 PM, "Srinivas Vemula"
>>>>>>> <sr...@mensamind.com>
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>> We are seeing thread leaks when having client open
>>>>>>>> connections to
>>>>>>>>
>>>>>>>
>>>>>> a web
>>>>>>
>>>>>>
>>>>>>>> server that hangs. Has any one seen this happening?? How do we
>>>>>>>> ensure
>>>>>>>>
>>>>>>>
>>>>>> that the
>>>>>>
>>>>>>
>>>>>>>> library correctly closes socket connections on failures,
>>>>>>>> cleaning up
>>>>>>>>
>>>>>>>
>>>>>> system
>>>>>>
>>>>>>
>>>>>>>> resources, and threads actually finish in the timeout period
>>>>>>>> and get
>>>>>>>>
>>>>>>>
>>>>>> freed
>>>>>>
>>>>>>
>>>>>>>> up. Would using MultiThreadedHttpConnectionManager
>>>>>>>> <
>>>>>>>>
>>>>>>>>
>>>>>> file:///D:/silkroad/http-commons/commons-httpclient-2.0-rc3/docs/
>>>>>> threading.ht
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> ml#MultiThreadedHttpConnectionManager> be of any help??
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Hi Srini,
>>>>>>> If you're using the same HttpClient instance across multiple
>>>>>>> threads, you
>>>>>>> must use the MultiThreadedHttpConnectionManager or you'll run
>>>>>>> into
>>>>>>>
>>>>>> strange
>>>>>>
>>>>>>
>>>>>>> problems. If you're using separate HttpClient instances, you
>>>>>>> may as well
>>>>>>> stick with the single threaded (default) connection manager.
>>>>>>>
>>>>>>> In terms of connections hanging - you probably want to look into
>>>>>>> the
>>>>>>> setConnectionTimeout and setTimeout methods of the HttpClient
>>>>>>> class.
>>>>>>>
>>>>>> These
>>>>>>
>>>>>>
>>>>>>> allow you to control how long HttpClient waits when making a
>>>>>>> connection
>>>>>>>
>>>>>> and
>>>>>>
>>>>>>
>>>>>>> how long it waits for data once the connection is established.
>>>>>>> If you
>>>>>>>
>>>>>> have
>>>>>>
>>>>>>
>>>>>>> set either of these to 0 (not sure what the default is, it may be
>>>>>>>
>>>>>> platform
>>>>>>
>>>>>>
>>>>>>> specific) the connection will never timeout which sounds a lot
>>>>>>> like what
>>>>>>> you're seeing.
>>>>>>>
>>>>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>>>>> HttpClient
>>>>>>>
>>>>>> is
>>>>>>
>>>>>>
>>>>>>> attempting to create a new timeout thread on every connection -
>>>>>>> does this
>>>>>>> always occur or is it just one timeout thread per httpclient
>>>>>>> instance? It
>>>>>>> would be ideal if we could reduce this to one static thread as
>>>>>>> starting
>>>>>>> threads is never nice for server side apps. Not sure how
>>>>>>> feasible that
>>>>>>>
>>>>>> is
>>>>>>
>>>>>>
>>>>>>> though.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Srini
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Adrian Sutton.
>>>>>>>
>>>>>>> ----------------------------------------------
>>>>>>> Intencha "tomorrow's technology today"
>>>>>>> Ph: 38478913 0422236329
>>>>>>> Suite 8/29 Oatland Crescent
>>>>>>> Holland Park West 4121
>>>>>>> Australia QLD
>>>>>>> www.intencha.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>>>> To unsubscribe, e-mail:
>>>>>>>
>>>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>>>
>>>>>>
>>>>>>> For additional commands, e-mail:
>>>>>>>
>>>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>>>
>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>>>> To unsubscribe, e-mail:
>>>>>>>
>>>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>>>
>>>>>>
>>>>>>> For additional commands, e-mail:
>>>>>>>
>>>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>>>
>>>>>>
>>>>>>>
>>>>>>
>>>>> --
>>>>> Laszlo Systems, Inc.
>>>>> 1040 Mariposa Street, SF, CA, USA 94107
>>>>>
>>>>> voice: 415.241.2721
>>>>> fax: 415.865.2914
>>>>> email: bloch@laszlosystems.com
>>>>> --
>>>>>
>>>>> Laszlo allows Behr to deliver a breakthrough experience with
>>>>> ColorSmart by BEHR application at http://www.behr.com
>>>>>
>>>>>
>>>>> -------------------------------------------------------------------
>>>>> --
>>>>> To unsubscribe, e-mail:
>>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>> For additional commands, e-mail:
>>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>>
>>>>>
>>>> --------------------------------------------------------------------
>>>> -
>>>> To unsubscribe, e-mail:
>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail:
>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail:
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> commons-httpclient-dev-help@jakarta.apache.org
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: Memory Leaks when web server hangs
Posted by Eric Bloch <bl...@laszlosystems.com>.
I'm happy to change my code but have a version of my product out there
that does it this way and wanted to make sure it wasn't in need of
immediate patch :-)
-E
Roland Weber wrote:
>Hello Eric,
>
>by using the same connection manager for each HttpClient,
>you avoid the biggest penalty for creating new clients. But
>still, each new client uses up some heap space, which will
>have to be garbage collected sooner or later. There are
>state objects, and parameter objecs, and probably some
>more that will be created. Nothing that hurts much, but what
>gain do you expect from creating new HttpClient objects?
>You don't have to.
>
>cheers,
> Roland
>
>
>
>
>
>
>Eric Bloch <bl...@laszlosystems.com>
>22.03.2004 07:03
>Please respond to "Commons HttpClient Project"
>
> To: Commons HttpClient Project
><co...@jakarta.apache.org>
> cc:
> Subject: Re: Memory Leaks when web server hangs
>
>
>Oh... and I just want to know if that's "not the httpclient way", too?
>That is, should I be reusing the client? What will that get me if I'm
>already reusing the connection manager?
>
>Thanks again,
>Eric
>
>Eric Bloch wrote:
>
>
>
>>I am creating one MultiThreadedHttpConnectionManager and
>>creating/destroying HttpClient each time but always constructing with
>>the same connection manager.
>>Thanks,
>>Eric
>>
>>
>>Michael Becke wrote:
>>
>>
>>
>>>Hi Eric,
>>>
>>>What exactly do you mean by "thread thrashing"? Which connection
>>>manager are you using?
>>>
>>>Mike
>>>
>>>On Mar 19, 2004, at 1:21 PM, Eric Bloch wrote:
>>>
>>>
>>>
>>>>Hey there,
>>>>
>>>>I create/destroy http clients but always have them use the same
>>>>connection manager.
>>>>
>>>>Will that cause thread thrashing?
>>>>
>>>>Thanks,
>>>>
>>>>-Eric
>>>>
>>>>Roland Weber wrote:
>>>>
>>>>
>>>>
>>>>>Hello Srini,
>>>>>
>>>>>you should *not* create a new HTTP Client for each request!
>>>>>This will also create a new connection manager, and a new
>>>>>cleanup thread. It beats the whole purpose of connection
>>>>>management.
>>>>>
>>>>>You should create *one* HTTP Client object for the lifetime
>>>>>of your application. Then, the connection manager of that
>>>>>client will reuse and free connections as appropriate.
>>>>>
>>>>>best regards,
>>>>> Roland
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>Srinivas Vemula <sr...@mensamind.com>
>>>>>02.02.2004 11:56
>>>>>Please respond to "Commons HttpClient Project"
>>>>> To: Commons HttpClient Project
>>>>><co...@jakarta.apache.org>
>>>>> cc: Subject: Re: Memory Leaks when web server
>>>>>hangs
>>>>>
>>>>>
>>>>>Oleg,
>>>>> We are using JDK1.4.1 and connection time out is set to 30Ms
>>>>>and read time out to 60Ms. We are trying to connect to a IP
>>>>>Camera, and send a heart beat command to the camera to check for
>>>>>its availability every 30 Secs from the time we find it is un
>>>>>available. If the web server is down, we feel that HttpClient is
>>>>>not cleaning up failed connections properly and thats resulting in
>>>>>Memory Leaks at the server side.
>>>>>
>>>>> The code we use is the standard way of using HttpClient and all
>>>>>the code is in a method and new HttpClient object is created for
>>>>>every request. Are there any ways to make sure the connections are
>>>>>all being purged properly? Are there any precautionary measures or
>>>>>flags we can set on HttpClient API when communicating with a web
>>>>>server (running on firm ware) with relatively less RAM and
>>>>>processing power (IpCamera with built in web server)?
>>>>>
>>>>>Thanks for your time and help.
>>>>>Srini
>>>>>
>>>>>Kalnichevski, Oleg wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>>Mike and Oleg, the stack trace Srini included indicates that
>>>>>>>HttpClient
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>is
>>>>>
>>>>>
>>>>>
>>>>>>>attempting to create a new timeout thread on every connection -
>>>>>>>
>>>>>>>
>does
>
>
>>>>>>
>>>>>>
>>>>>this
>>>>>
>>>>>
>>>>>
>>>>>>>always occur or is it just one timeout thread per httpclient
>>>>>>>instance?
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>It
>>>>>
>>>>>
>>>>>
>>>>>>>would be ideal if we could reduce this to one static thread as
>>>>>>>starting
>>>>>>>threads is never nice for server side apps. Not sure how
>>>>>>>feasible that
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>is
>>>>>
>>>>>
>>>>>
>>>>>>>though.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>Hi Srini,
>>>>>>HttpClient uses an additional controller thread to work around the
>>>>>>
>>>>>>
>>>>>limitation of older (< 1.4) JDKs which do not provide a
>>>>>possibility to set connect timeout. If you do not really need to
>>>>>control connect timeout (for instance, when communicating with an
>>>>>intranet site with good availability) simply set connect timeout
>>>>>to. That will prevent HttpClient from spawning an additional
>>>>>thread per request.
>>>>>
>>>>>
>>>>>
>>>>>>Adrian, et al
>>>>>>Another possibility to use reflection to set connect timeout
>>>>>>using the
>>>>>>
>>>>>>
>>>>>Socket methods when running in JVM 1.4 or above
>>>>>
>>>>>
>>>>>
>>>>>>Oleg
>>>>>>
>>>>>>
>>>>>>
>>>>>>-----Original Message-----
>>>>>>From: Adrian Sutton [mailto:adrian@intencha.com]
>>>>>>Sent: Monday, February 02, 2004 09:03
>>>>>>To: Commons HttpClient Project
>>>>>>Subject: Re: Memory Leaks when web server hangs
>>>>>>
>>>>>>
>>>>>>On 2/2/04 2:00 PM, "Srinivas Vemula" <sr...@mensamind.com>
>>>>>>
>>>>>>
>>>>>wrote:
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>>Hi All,
>>>>>>> We are seeing thread leaks when having client open
>>>>>>>connections to
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>a web
>>>>>
>>>>>
>>>>>
>>>>>>>server that hangs. Has any one seen this happening?? How do we
>>>>>>>ensure
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>that the
>>>>>
>>>>>
>>>>>
>>>>>>>library correctly closes socket connections on failures,
>>>>>>>cleaning up
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>system
>>>>>
>>>>>
>>>>>
>>>>>>>resources, and threads actually finish in the timeout period
>>>>>>>and get
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>freed
>>>>>
>>>>>
>>>>>
>>>>>>>up. Would using MultiThreadedHttpConnectionManager
>>>>>>><
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>file:///D:/silkroad/http-commons/commons-httpclient-2.0-rc3/docs/
>>>>>threading.ht
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>>ml#MultiThreadedHttpConnectionManager> be of any help??
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>Hi Srini,
>>>>>>If you're using the same HttpClient instance across multiple
>>>>>>threads, you
>>>>>>must use the MultiThreadedHttpConnectionManager or you'll run into
>>>>>>
>>>>>>
>>>>>strange
>>>>>
>>>>>
>>>>>
>>>>>>problems. If you're using separate HttpClient instances, you may
>>>>>>as well
>>>>>>stick with the single threaded (default) connection manager.
>>>>>>
>>>>>>In terms of connections hanging - you probably want to look into the
>>>>>>setConnectionTimeout and setTimeout methods of the HttpClient class.
>>>>>>
>>>>>>
>>>>>These
>>>>>
>>>>>
>>>>>
>>>>>>allow you to control how long HttpClient waits when making a
>>>>>>connection
>>>>>>
>>>>>>
>>>>>and
>>>>>
>>>>>
>>>>>
>>>>>>how long it waits for data once the connection is established.
>>>>>>If you
>>>>>>
>>>>>>
>>>>>have
>>>>>
>>>>>
>>>>>
>>>>>>set either of these to 0 (not sure what the default is, it may be
>>>>>>
>>>>>>
>>>>>platform
>>>>>
>>>>>
>>>>>
>>>>>>specific) the connection will never timeout which sounds a lot
>>>>>>like what
>>>>>>you're seeing.
>>>>>>
>>>>>>Mike and Oleg, the stack trace Srini included indicates that
>>>>>>HttpClient
>>>>>>
>>>>>>
>>>>>is
>>>>>
>>>>>
>>>>>
>>>>>>attempting to create a new timeout thread on every connection -
>>>>>>does this
>>>>>>always occur or is it just one timeout thread per httpclient
>>>>>>instance? It
>>>>>>would be ideal if we could reduce this to one static thread as
>>>>>>starting
>>>>>>threads is never nice for server side apps. Not sure how
>>>>>>feasible that
>>>>>>
>>>>>>
>>>>>is
>>>>>
>>>>>
>>>>>
>>>>>>though.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Srini
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>Regards,
>>>>>>
>>>>>>Adrian Sutton.
>>>>>>
>>>>>>----------------------------------------------
>>>>>>Intencha "tomorrow's technology today"
>>>>>>Ph: 38478913 0422236329
>>>>>>Suite 8/29 Oatland Crescent
>>>>>>Holland Park West 4121
>>>>>>Australia QLD
>>>>>>www.intencha.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>---------------------------------------------------------------------
>
>
>>>>>>To unsubscribe, e-mail:
>>>>>>
>>>>>>
>>>>>commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>>
>>>>>
>>>>>
>>>>>>For additional commands, e-mail:
>>>>>>
>>>>>>
>>>>>commons-httpclient-dev-help@jakarta.apache.org
>>>>>
>>>>>
>>>>>
>---------------------------------------------------------------------
>
>
>>>>>>To unsubscribe, e-mail:
>>>>>>
>>>>>>
>>>>>commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>>
>>>>>
>>>>>
>>>>>>For additional commands, e-mail:
>>>>>>
>>>>>>
>>>>>commons-httpclient-dev-help@jakarta.apache.org
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>--
>>>>Laszlo Systems, Inc.
>>>>1040 Mariposa Street, SF, CA, USA 94107
>>>>
>>>>voice: 415.241.2721
>>>>fax: 415.865.2914
>>>>email: bloch@laszlosystems.com
>>>>--
>>>>
>>>>Laszlo allows Behr to deliver a breakthrough experience with
>>>>ColorSmart by BEHR application at http://www.behr.com
>>>>
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe, e-mail:
>>>>commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>For additional commands, e-mail:
>>>>commons-httpclient-dev-help@jakarta.apache.org
>>>>
>>>>
>>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail:
>>>commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>For additional commands, e-mail:
>>>commons-httpclient-dev-help@jakarta.apache.org
>>>
>>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail:
>>commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail:
>>commons-httpclient-dev-help@jakarta.apache.org
>>
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail:
>commons-httpclient-dev-unsubscribe@jakarta.apache.org
>For additional commands, e-mail:
>commons-httpclient-dev-help@jakarta.apache.org
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: Memory Leaks when web server hangs
Posted by Roland Weber <RO...@de.ibm.com>.
Hello Eric,
by using the same connection manager for each HttpClient,
you avoid the biggest penalty for creating new clients. But
still, each new client uses up some heap space, which will
have to be garbage collected sooner or later. There are
state objects, and parameter objecs, and probably some
more that will be created. Nothing that hurts much, but what
gain do you expect from creating new HttpClient objects?
You don't have to.
cheers,
Roland
Eric Bloch <bl...@laszlosystems.com>
22.03.2004 07:03
Please respond to "Commons HttpClient Project"
To: Commons HttpClient Project
<co...@jakarta.apache.org>
cc:
Subject: Re: Memory Leaks when web server hangs
Oh... and I just want to know if that's "not the httpclient way", too?
That is, should I be reusing the client? What will that get me if I'm
already reusing the connection manager?
Thanks again,
Eric
Eric Bloch wrote:
> I am creating one MultiThreadedHttpConnectionManager and
> creating/destroying HttpClient each time but always constructing with
> the same connection manager.
> Thanks,
> Eric
>
>
> Michael Becke wrote:
>
>> Hi Eric,
>>
>> What exactly do you mean by "thread thrashing"? Which connection
>> manager are you using?
>>
>> Mike
>>
>> On Mar 19, 2004, at 1:21 PM, Eric Bloch wrote:
>>
>>> Hey there,
>>>
>>> I create/destroy http clients but always have them use the same
>>> connection manager.
>>>
>>> Will that cause thread thrashing?
>>>
>>> Thanks,
>>>
>>> -Eric
>>>
>>> Roland Weber wrote:
>>>
>>>> Hello Srini,
>>>>
>>>> you should *not* create a new HTTP Client for each request!
>>>> This will also create a new connection manager, and a new
>>>> cleanup thread. It beats the whole purpose of connection
>>>> management.
>>>>
>>>> You should create *one* HTTP Client object for the lifetime
>>>> of your application. Then, the connection manager of that
>>>> client will reuse and free connections as appropriate.
>>>>
>>>> best regards,
>>>> Roland
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Srinivas Vemula <sr...@mensamind.com>
>>>> 02.02.2004 11:56
>>>> Please respond to "Commons HttpClient Project"
>>>> To: Commons HttpClient Project
>>>> <co...@jakarta.apache.org>
>>>> cc: Subject: Re: Memory Leaks when web server
>>>> hangs
>>>>
>>>>
>>>> Oleg,
>>>> We are using JDK1.4.1 and connection time out is set to 30Ms
>>>> and read time out to 60Ms. We are trying to connect to a IP
>>>> Camera, and send a heart beat command to the camera to check for
>>>> its availability every 30 Secs from the time we find it is un
>>>> available. If the web server is down, we feel that HttpClient is
>>>> not cleaning up failed connections properly and thats resulting in
>>>> Memory Leaks at the server side.
>>>>
>>>> The code we use is the standard way of using HttpClient and all
>>>> the code is in a method and new HttpClient object is created for
>>>> every request. Are there any ways to make sure the connections are
>>>> all being purged properly? Are there any precautionary measures or
>>>> flags we can set on HttpClient API when communicating with a web
>>>> server (running on firm ware) with relatively less RAM and
>>>> processing power (IpCamera with built in web server)?
>>>>
>>>> Thanks for your time and help.
>>>> Srini
>>>>
>>>> Kalnichevski, Oleg wrote:
>>>>
>>>>
>>>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>>>> HttpClient
>>>>>
>>>>>
>>>> is
>>>>
>>>>>> attempting to create a new timeout thread on every connection -
does
>>>>>
>>>>>
>>>> this
>>>>
>>>>>> always occur or is it just one timeout thread per httpclient
>>>>>> instance?
>>>>>
>>>>>
>>>> It
>>>>
>>>>>> would be ideal if we could reduce this to one static thread as
>>>>>> starting
>>>>>> threads is never nice for server side apps. Not sure how
>>>>>> feasible that
>>>>>
>>>>>
>>>> is
>>>>
>>>>>> though.
>>>>>>
>>>>>>
>>>>>>
>>>>> Hi Srini,
>>>>> HttpClient uses an additional controller thread to work around the
>>>>
>>>>
>>>> limitation of older (< 1.4) JDKs which do not provide a
>>>> possibility to set connect timeout. If you do not really need to
>>>> control connect timeout (for instance, when communicating with an
>>>> intranet site with good availability) simply set connect timeout
>>>> to. That will prevent HttpClient from spawning an additional
>>>> thread per request.
>>>>
>>>>> Adrian, et al
>>>>> Another possibility to use reflection to set connect timeout
>>>>> using the
>>>>
>>>>
>>>> Socket methods when running in JVM 1.4 or above
>>>>
>>>>> Oleg
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Adrian Sutton [mailto:adrian@intencha.com]
>>>>> Sent: Monday, February 02, 2004 09:03
>>>>> To: Commons HttpClient Project
>>>>> Subject: Re: Memory Leaks when web server hangs
>>>>>
>>>>>
>>>>> On 2/2/04 2:00 PM, "Srinivas Vemula" <sr...@mensamind.com>
>>>>
>>>>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>>> Hi All,
>>>>>> We are seeing thread leaks when having client open
>>>>>> connections to
>>>>>
>>>>>
>>>> a web
>>>>
>>>>>> server that hangs. Has any one seen this happening?? How do we
>>>>>> ensure
>>>>>
>>>>>
>>>> that the
>>>>
>>>>>> library correctly closes socket connections on failures,
>>>>>> cleaning up
>>>>>
>>>>>
>>>> system
>>>>
>>>>>> resources, and threads actually finish in the timeout period
>>>>>> and get
>>>>>
>>>>>
>>>> freed
>>>>
>>>>>> up. Would using MultiThreadedHttpConnectionManager
>>>>>> <
>>>>>>
>>>> file:///D:/silkroad/http-commons/commons-httpclient-2.0-rc3/docs/
>>>> threading.ht
>>>>
>>>>
>>>>>> ml#MultiThreadedHttpConnectionManager> be of any help??
>>>>>>
>>>>>>
>>>>>>
>>>>> Hi Srini,
>>>>> If you're using the same HttpClient instance across multiple
>>>>> threads, you
>>>>> must use the MultiThreadedHttpConnectionManager or you'll run into
>>>>
>>>>
>>>> strange
>>>>
>>>>> problems. If you're using separate HttpClient instances, you may
>>>>> as well
>>>>> stick with the single threaded (default) connection manager.
>>>>>
>>>>> In terms of connections hanging - you probably want to look into the
>>>>> setConnectionTimeout and setTimeout methods of the HttpClient class.
>>>>
>>>>
>>>> These
>>>>
>>>>> allow you to control how long HttpClient waits when making a
>>>>> connection
>>>>
>>>>
>>>> and
>>>>
>>>>> how long it waits for data once the connection is established.
>>>>> If you
>>>>
>>>>
>>>> have
>>>>
>>>>> set either of these to 0 (not sure what the default is, it may be
>>>>
>>>>
>>>> platform
>>>>
>>>>> specific) the connection will never timeout which sounds a lot
>>>>> like what
>>>>> you're seeing.
>>>>>
>>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>>> HttpClient
>>>>
>>>>
>>>> is
>>>>
>>>>> attempting to create a new timeout thread on every connection -
>>>>> does this
>>>>> always occur or is it just one timeout thread per httpclient
>>>>> instance? It
>>>>> would be ideal if we could reduce this to one static thread as
>>>>> starting
>>>>> threads is never nice for server side apps. Not sure how
>>>>> feasible that
>>>>
>>>>
>>>> is
>>>>
>>>>> though.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Srini
>>>>>>
>>>>>>
>>>>>>
>>>>> Regards,
>>>>>
>>>>> Adrian Sutton.
>>>>>
>>>>> ----------------------------------------------
>>>>> Intencha "tomorrow's technology today"
>>>>> Ph: 38478913 0422236329
>>>>> Suite 8/29 Oatland Crescent
>>>>> Holland Park West 4121
>>>>> Australia QLD
>>>>> www.intencha.com
>>>>>
>>>>>
>>>>>
---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>>
>>>>
>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>
>>>>> For additional commands, e-mail:
>>>>
>>>>
>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>
>>>>>
---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>>
>>>>
>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>
>>>>> For additional commands, e-mail:
>>>>
>>>>
>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Laszlo Systems, Inc.
>>> 1040 Mariposa Street, SF, CA, USA 94107
>>>
>>> voice: 415.241.2721
>>> fax: 415.865.2914
>>> email: bloch@laszlosystems.com
>>> --
>>>
>>> Laszlo allows Behr to deliver a breakthrough experience with
>>> ColorSmart by BEHR application at http://www.behr.com
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail:
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> commons-httpclient-dev-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org
Re: Memory Leaks when web server hangs
Posted by Eric Bloch <bl...@laszlosystems.com>.
Oh... and I just want to know if that's "not the httpclient way", too?
That is, should I be reusing the client? What will that get me if I'm
already reusing the connection manager?
Thanks again,
Eric
Eric Bloch wrote:
> I am creating one MultiThreadedHttpConnectionManager and
> creating/destroying HttpClient each time but always constructing with
> the same connection manager.
> Thanks,
> Eric
>
>
> Michael Becke wrote:
>
>> Hi Eric,
>>
>> What exactly do you mean by "thread thrashing"? Which connection
>> manager are you using?
>>
>> Mike
>>
>> On Mar 19, 2004, at 1:21 PM, Eric Bloch wrote:
>>
>>> Hey there,
>>>
>>> I create/destroy http clients but always have them use the same
>>> connection manager.
>>>
>>> Will that cause thread thrashing?
>>>
>>> Thanks,
>>>
>>> -Eric
>>>
>>> Roland Weber wrote:
>>>
>>>> Hello Srini,
>>>>
>>>> you should *not* create a new HTTP Client for each request!
>>>> This will also create a new connection manager, and a new
>>>> cleanup thread. It beats the whole purpose of connection
>>>> management.
>>>>
>>>> You should create *one* HTTP Client object for the lifetime
>>>> of your application. Then, the connection manager of that
>>>> client will reuse and free connections as appropriate.
>>>>
>>>> best regards,
>>>> Roland
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Srinivas Vemula <sr...@mensamind.com>
>>>> 02.02.2004 11:56
>>>> Please respond to "Commons HttpClient Project"
>>>> To: Commons HttpClient Project
>>>> <co...@jakarta.apache.org>
>>>> cc: Subject: Re: Memory Leaks when web server
>>>> hangs
>>>>
>>>>
>>>> Oleg,
>>>> We are using JDK1.4.1 and connection time out is set to 30Ms
>>>> and read time out to 60Ms. We are trying to connect to a IP
>>>> Camera, and send a heart beat command to the camera to check for
>>>> its availability every 30 Secs from the time we find it is un
>>>> available. If the web server is down, we feel that HttpClient is
>>>> not cleaning up failed connections properly and thats resulting in
>>>> Memory Leaks at the server side.
>>>>
>>>> The code we use is the standard way of using HttpClient and all
>>>> the code is in a method and new HttpClient object is created for
>>>> every request. Are there any ways to make sure the connections are
>>>> all being purged properly? Are there any precautionary measures or
>>>> flags we can set on HttpClient API when communicating with a web
>>>> server (running on firm ware) with relatively less RAM and
>>>> processing power (IpCamera with built in web server)?
>>>>
>>>> Thanks for your time and help.
>>>> Srini
>>>>
>>>> Kalnichevski, Oleg wrote:
>>>>
>>>>
>>>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>>>> HttpClient
>>>>>
>>>>>
>>>> is
>>>>
>>>>>> attempting to create a new timeout thread on every connection - does
>>>>>
>>>>>
>>>> this
>>>>
>>>>>> always occur or is it just one timeout thread per httpclient
>>>>>> instance?
>>>>>
>>>>>
>>>> It
>>>>
>>>>>> would be ideal if we could reduce this to one static thread as
>>>>>> starting
>>>>>> threads is never nice for server side apps. Not sure how
>>>>>> feasible that
>>>>>
>>>>>
>>>> is
>>>>
>>>>>> though.
>>>>>>
>>>>>>
>>>>>>
>>>>> Hi Srini,
>>>>> HttpClient uses an additional controller thread to work around the
>>>>
>>>>
>>>> limitation of older (< 1.4) JDKs which do not provide a
>>>> possibility to set connect timeout. If you do not really need to
>>>> control connect timeout (for instance, when communicating with an
>>>> intranet site with good availability) simply set connect timeout
>>>> to. That will prevent HttpClient from spawning an additional
>>>> thread per request.
>>>>
>>>>> Adrian, et al
>>>>> Another possibility to use reflection to set connect timeout
>>>>> using the
>>>>
>>>>
>>>> Socket methods when running in JVM 1.4 or above
>>>>
>>>>> Oleg
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Adrian Sutton [mailto:adrian@intencha.com]
>>>>> Sent: Monday, February 02, 2004 09:03
>>>>> To: Commons HttpClient Project
>>>>> Subject: Re: Memory Leaks when web server hangs
>>>>>
>>>>>
>>>>> On 2/2/04 2:00 PM, "Srinivas Vemula" <sr...@mensamind.com>
>>>>
>>>>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>>> Hi All,
>>>>>> We are seeing thread leaks when having client open
>>>>>> connections to
>>>>>
>>>>>
>>>> a web
>>>>
>>>>>> server that hangs. Has any one seen this happening?? How do we
>>>>>> ensure
>>>>>
>>>>>
>>>> that the
>>>>
>>>>>> library correctly closes socket connections on failures,
>>>>>> cleaning up
>>>>>
>>>>>
>>>> system
>>>>
>>>>>> resources, and threads actually finish in the timeout period
>>>>>> and get
>>>>>
>>>>>
>>>> freed
>>>>
>>>>>> up. Would using MultiThreadedHttpConnectionManager
>>>>>> <
>>>>>>
>>>> file:///D:/silkroad/http-commons/commons-httpclient-2.0-rc3/docs/
>>>> threading.ht
>>>>
>>>>
>>>>>> ml#MultiThreadedHttpConnectionManager> be of any help??
>>>>>>
>>>>>>
>>>>>>
>>>>> Hi Srini,
>>>>> If you're using the same HttpClient instance across multiple
>>>>> threads, you
>>>>> must use the MultiThreadedHttpConnectionManager or you'll run into
>>>>
>>>>
>>>> strange
>>>>
>>>>> problems. If you're using separate HttpClient instances, you may
>>>>> as well
>>>>> stick with the single threaded (default) connection manager.
>>>>>
>>>>> In terms of connections hanging - you probably want to look into the
>>>>> setConnectionTimeout and setTimeout methods of the HttpClient class.
>>>>
>>>>
>>>> These
>>>>
>>>>> allow you to control how long HttpClient waits when making a
>>>>> connection
>>>>
>>>>
>>>> and
>>>>
>>>>> how long it waits for data once the connection is established.
>>>>> If you
>>>>
>>>>
>>>> have
>>>>
>>>>> set either of these to 0 (not sure what the default is, it may be
>>>>
>>>>
>>>> platform
>>>>
>>>>> specific) the connection will never timeout which sounds a lot
>>>>> like what
>>>>> you're seeing.
>>>>>
>>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>>> HttpClient
>>>>
>>>>
>>>> is
>>>>
>>>>> attempting to create a new timeout thread on every connection -
>>>>> does this
>>>>> always occur or is it just one timeout thread per httpclient
>>>>> instance? It
>>>>> would be ideal if we could reduce this to one static thread as
>>>>> starting
>>>>> threads is never nice for server side apps. Not sure how
>>>>> feasible that
>>>>
>>>>
>>>> is
>>>>
>>>>> though.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Srini
>>>>>>
>>>>>>
>>>>>>
>>>>> Regards,
>>>>>
>>>>> Adrian Sutton.
>>>>>
>>>>> ----------------------------------------------
>>>>> Intencha "tomorrow's technology today"
>>>>> Ph: 38478913 0422236329
>>>>> Suite 8/29 Oatland Crescent
>>>>> Holland Park West 4121
>>>>> Australia QLD
>>>>> www.intencha.com
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>>
>>>>
>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>
>>>>> For additional commands, e-mail:
>>>>
>>>>
>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>>
>>>>
>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>>
>>>>> For additional commands, e-mail:
>>>>
>>>>
>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Laszlo Systems, Inc.
>>> 1040 Mariposa Street, SF, CA, USA 94107
>>>
>>> voice: 415.241.2721
>>> fax: 415.865.2914
>>> email: bloch@laszlosystems.com
>>> --
>>>
>>> Laszlo allows Behr to deliver a breakthrough experience with
>>> ColorSmart by BEHR application at http://www.behr.com
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail:
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> commons-httpclient-dev-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: Memory Leaks when web server hangs
Posted by Eric Bloch <bl...@laszlosystems.com>.
I am creating one MultiThreadedHttpConnectionManager and
creating/destroying HttpClient each time but always constructing with
the same connection manager.
Thanks,
Eric
Michael Becke wrote:
> Hi Eric,
>
> What exactly do you mean by "thread thrashing"? Which connection
> manager are you using?
>
> Mike
>
> On Mar 19, 2004, at 1:21 PM, Eric Bloch wrote:
>
>> Hey there,
>>
>> I create/destroy http clients but always have them use the same
>> connection manager.
>>
>> Will that cause thread thrashing?
>>
>> Thanks,
>>
>> -Eric
>>
>> Roland Weber wrote:
>>
>>> Hello Srini,
>>>
>>> you should *not* create a new HTTP Client for each request!
>>> This will also create a new connection manager, and a new
>>> cleanup thread. It beats the whole purpose of connection
>>> management.
>>>
>>> You should create *one* HTTP Client object for the lifetime
>>> of your application. Then, the connection manager of that
>>> client will reuse and free connections as appropriate.
>>>
>>> best regards,
>>> Roland
>>>
>>>
>>>
>>>
>>>
>>>
>>> Srinivas Vemula <sr...@mensamind.com>
>>> 02.02.2004 11:56
>>> Please respond to "Commons HttpClient Project"
>>> To: Commons HttpClient Project
>>> <co...@jakarta.apache.org>
>>> cc: Subject: Re: Memory Leaks when web server
>>> hangs
>>>
>>>
>>> Oleg,
>>> We are using JDK1.4.1 and connection time out is set to 30Ms and
>>> read time out to 60Ms. We are trying to connect to a IP Camera, and
>>> send a heart beat command to the camera to check for its
>>> availability every 30 Secs from the time we find it is un
>>> available. If the web server is down, we feel that HttpClient is
>>> not cleaning up failed connections properly and thats resulting in
>>> Memory Leaks at the server side.
>>>
>>> The code we use is the standard way of using HttpClient and all
>>> the code is in a method and new HttpClient object is created for
>>> every request. Are there any ways to make sure the connections are
>>> all being purged properly? Are there any precautionary measures or
>>> flags we can set on HttpClient API when communicating with a web
>>> server (running on firm ware) with relatively less RAM and
>>> processing power (IpCamera with built in web server)?
>>>
>>> Thanks for your time and help.
>>> Srini
>>>
>>> Kalnichevski, Oleg wrote:
>>>
>>>
>>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>>> HttpClient
>>>>
>>> is
>>>
>>>>> attempting to create a new timeout thread on every connection - does
>>>>
>>> this
>>>
>>>>> always occur or is it just one timeout thread per httpclient
>>>>> instance?
>>>>
>>> It
>>>
>>>>> would be ideal if we could reduce this to one static thread as
>>>>> starting
>>>>> threads is never nice for server side apps. Not sure how
>>>>> feasible that
>>>>
>>> is
>>>
>>>>> though.
>>>>>
>>>>>
>>>>>
>>>> Hi Srini,
>>>> HttpClient uses an additional controller thread to work around the
>>>
>>> limitation of older (< 1.4) JDKs which do not provide a possibility
>>> to set connect timeout. If you do not really need to control
>>> connect timeout (for instance, when communicating with an intranet
>>> site with good availability) simply set connect timeout to. That
>>> will prevent HttpClient from spawning an additional thread per
>>> request.
>>>
>>>> Adrian, et al
>>>> Another possibility to use reflection to set connect timeout using
>>>> the
>>>
>>> Socket methods when running in JVM 1.4 or above
>>>
>>>> Oleg
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Adrian Sutton [mailto:adrian@intencha.com]
>>>> Sent: Monday, February 02, 2004 09:03
>>>> To: Commons HttpClient Project
>>>> Subject: Re: Memory Leaks when web server hangs
>>>>
>>>>
>>>> On 2/2/04 2:00 PM, "Srinivas Vemula" <sr...@mensamind.com>
>>>
>>> wrote:
>>>
>>>>
>>>>
>>>>> Hi All,
>>>>> We are seeing thread leaks when having client open
>>>>> connections to
>>>>
>>> a web
>>>
>>>>> server that hangs. Has any one seen this happening?? How do we
>>>>> ensure
>>>>
>>> that the
>>>
>>>>> library correctly closes socket connections on failures,
>>>>> cleaning up
>>>>
>>> system
>>>
>>>>> resources, and threads actually finish in the timeout period
>>>>> and get
>>>>
>>> freed
>>>
>>>>> up. Would using MultiThreadedHttpConnectionManager
>>>>> <
>>>>>
>>> file:///D:/silkroad/http-commons/commons-httpclient-2.0-rc3/docs/
>>> threading.ht
>>>
>>>
>>>>> ml#MultiThreadedHttpConnectionManager> be of any help??
>>>>>
>>>>>
>>>>>
>>>> Hi Srini,
>>>> If you're using the same HttpClient instance across multiple
>>>> threads, you
>>>> must use the MultiThreadedHttpConnectionManager or you'll run into
>>>
>>> strange
>>>
>>>> problems. If you're using separate HttpClient instances, you may
>>>> as well
>>>> stick with the single threaded (default) connection manager.
>>>>
>>>> In terms of connections hanging - you probably want to look into the
>>>> setConnectionTimeout and setTimeout methods of the HttpClient class.
>>>
>>> These
>>>
>>>> allow you to control how long HttpClient waits when making a
>>>> connection
>>>
>>> and
>>>
>>>> how long it waits for data once the connection is established. If
>>>> you
>>>
>>> have
>>>
>>>> set either of these to 0 (not sure what the default is, it may be
>>>
>>> platform
>>>
>>>> specific) the connection will never timeout which sounds a lot
>>>> like what
>>>> you're seeing.
>>>>
>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>> HttpClient
>>>
>>> is
>>>
>>>> attempting to create a new timeout thread on every connection -
>>>> does this
>>>> always occur or is it just one timeout thread per httpclient
>>>> instance? It
>>>> would be ideal if we could reduce this to one static thread as
>>>> starting
>>>> threads is never nice for server side apps. Not sure how feasible
>>>> that
>>>
>>> is
>>>
>>>> though.
>>>>
>>>>
>>>>
>>>>
>>>>> Srini
>>>>>
>>>>>
>>>>>
>>>> Regards,
>>>>
>>>> Adrian Sutton.
>>>>
>>>> ----------------------------------------------
>>>> Intencha "tomorrow's technology today"
>>>> Ph: 38478913 0422236329
>>>> Suite 8/29 Oatland Crescent
>>>> Holland Park West 4121
>>>> Australia QLD
>>>> www.intencha.com
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail:
>>>
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>
>>>> For additional commands, e-mail:
>>>
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail:
>>>
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>
>>>> For additional commands, e-mail:
>>>
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>>>
>>>>
>>>
>>>
>>
>>
>> --
>> Laszlo Systems, Inc.
>> 1040 Mariposa Street, SF, CA, USA 94107
>>
>> voice: 415.241.2721
>> fax: 415.865.2914
>> email: bloch@laszlosystems.com
>> --
>>
>> Laszlo allows Behr to deliver a breakthrough experience with
>> ColorSmart by BEHR application at http://www.behr.com
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> commons-httpclient-dev-help@jakarta.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: Memory Leaks when web server hangs
Posted by Michael Becke <be...@u.washington.edu>.
Hi Eric,
What exactly do you mean by "thread thrashing"? Which connection
manager are you using?
Mike
On Mar 19, 2004, at 1:21 PM, Eric Bloch wrote:
> Hey there,
>
> I create/destroy http clients but always have them use the same
> connection manager.
>
> Will that cause thread thrashing?
>
> Thanks,
>
> -Eric
>
> Roland Weber wrote:
>
>> Hello Srini,
>>
>> you should *not* create a new HTTP Client for each request!
>> This will also create a new connection manager, and a new
>> cleanup thread. It beats the whole purpose of connection
>> management.
>>
>> You should create *one* HTTP Client object for the lifetime
>> of your application. Then, the connection manager of that
>> client will reuse and free connections as appropriate.
>>
>> best regards,
>> Roland
>>
>>
>>
>>
>>
>>
>> Srinivas Vemula <sr...@mensamind.com>
>> 02.02.2004 11:56
>> Please respond to "Commons HttpClient Project"
>> To: Commons HttpClient Project
>> <co...@jakarta.apache.org>
>> cc: Subject: Re: Memory Leaks when web server
>> hangs
>>
>>
>> Oleg,
>> We are using JDK1.4.1 and connection time out is set to 30Ms and
>> read time out to 60Ms. We are trying to connect to a IP Camera, and
>> send a heart beat command to the camera to check for its availability
>> every 30 Secs from the time we find it is un available. If the web
>> server is down, we feel that HttpClient is not cleaning up failed
>> connections properly and thats resulting in Memory Leaks at the
>> server side.
>>
>> The code we use is the standard way of using HttpClient and all
>> the code is in a method and new HttpClient object is created for
>> every request. Are there any ways to make sure the connections are
>> all being purged properly? Are there any precautionary measures or
>> flags we can set on HttpClient API when communicating with a web
>> server (running on firm ware) with relatively less RAM and processing
>> power (IpCamera with built in web server)?
>>
>> Thanks for your time and help.
>> Srini
>>
>> Kalnichevski, Oleg wrote:
>>
>>
>>>> Mike and Oleg, the stack trace Srini included indicates that
>>>> HttpClient
>> is
>>
>>>> attempting to create a new timeout thread on every connection - does
>> this
>>
>>>> always occur or is it just one timeout thread per httpclient
>>>> instance?
>> It
>>
>>>> would be ideal if we could reduce this to one static thread as
>>>> starting
>>>> threads is never nice for server side apps. Not sure how feasible
>>>> that
>> is
>>
>>>> though.
>>>>
>>>>
>>>>
>>> Hi Srini,
>>> HttpClient uses an additional controller thread to work around the
>> limitation of older (< 1.4) JDKs which do not provide a possibility
>> to set connect timeout. If you do not really need to control connect
>> timeout (for instance, when communicating with an intranet site with
>> good availability) simply set connect timeout to. That will prevent
>> HttpClient from spawning an additional thread per request.
>>
>>> Adrian, et al
>>> Another possibility to use reflection to set connect timeout using
>>> the
>> Socket methods when running in JVM 1.4 or above
>>
>>> Oleg
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Adrian Sutton [mailto:adrian@intencha.com]
>>> Sent: Monday, February 02, 2004 09:03
>>> To: Commons HttpClient Project
>>> Subject: Re: Memory Leaks when web server hangs
>>>
>>>
>>> On 2/2/04 2:00 PM, "Srinivas Vemula" <sr...@mensamind.com>
>> wrote:
>>
>>>
>>>
>>>> Hi All,
>>>> We are seeing thread leaks when having client open connections
>>>> to
>> a web
>>
>>>> server that hangs. Has any one seen this happening?? How do we
>>>> ensure
>> that the
>>
>>>> library correctly closes socket connections on failures, cleaning
>>>> up
>> system
>>
>>>> resources, and threads actually finish in the timeout period and
>>>> get
>> freed
>>
>>>> up. Would using MultiThreadedHttpConnectionManager
>>>> <
>>>>
>> file:///D:/silkroad/http-commons/commons-httpclient-2.0-rc3/docs/
>> threading.ht
>>
>>
>>>> ml#MultiThreadedHttpConnectionManager> be of any help??
>>>>
>>>>
>>>>
>>> Hi Srini,
>>> If you're using the same HttpClient instance across multiple
>>> threads, you
>>> must use the MultiThreadedHttpConnectionManager or you'll run into
>> strange
>>
>>> problems. If you're using separate HttpClient instances, you may as
>>> well
>>> stick with the single threaded (default) connection manager.
>>>
>>> In terms of connections hanging - you probably want to look into the
>>> setConnectionTimeout and setTimeout methods of the HttpClient class.
>> These
>>
>>> allow you to control how long HttpClient waits when making a
>>> connection
>> and
>>
>>> how long it waits for data once the connection is established. If
>>> you
>> have
>>
>>> set either of these to 0 (not sure what the default is, it may be
>> platform
>>
>>> specific) the connection will never timeout which sounds a lot like
>>> what
>>> you're seeing.
>>>
>>> Mike and Oleg, the stack trace Srini included indicates that
>>> HttpClient
>> is
>>
>>> attempting to create a new timeout thread on every connection - does
>>> this
>>> always occur or is it just one timeout thread per httpclient
>>> instance? It
>>> would be ideal if we could reduce this to one static thread as
>>> starting
>>> threads is never nice for server side apps. Not sure how feasible
>>> that
>> is
>>
>>> though.
>>>
>>>
>>>
>>>
>>>> Srini
>>>>
>>>>
>>>>
>>> Regards,
>>>
>>> Adrian Sutton.
>>>
>>> ----------------------------------------------
>>> Intencha "tomorrow's technology today"
>>> Ph: 38478913 0422236329
>>> Suite 8/29 Oatland Crescent
>>> Holland Park West 4121
>>> Australia QLD
>>> www.intencha.com
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>
>>> For additional commands, e-mail:
>> commons-httpclient-dev-help@jakarta.apache.org
>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>
>>> For additional commands, e-mail:
>> commons-httpclient-dev-help@jakarta.apache.org
>>
>>>
>>>
>>
>>
>
>
> --
> Laszlo Systems, Inc.
> 1040 Mariposa Street, SF, CA, USA 94107
>
> voice: 415.241.2721
> fax: 415.865.2914
> email: bloch@laszlosystems.com
> --
>
> Laszlo allows Behr to deliver a breakthrough experience with
> ColorSmart by BEHR application at http://www.behr.com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org