You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Jan Gonsalves <JG...@Xtensible.com> on 2003/05/29 16:36:57 UTC
Memory issues with long connections?
Does anyone know of any memory issues with opening a connection and
continuely re-using it, only closing it when a socket timeout happens?
I'm currently using ALPHA 3. I have to "fake out a browser" and
navigate through a website to extract information because they don't
offer a straight DB connection. I re-use this connection the whole
time, never closing it, only recycling it, unless I get a socket
timeout, ConnectionException, or HTTPRecoverableException. Our
Production server has 4GB of ram. Last night we ran and the system went
down to 4MB free, 4! I'm just wondering if I'm leaking due to HTTP
Client. I've gone over my stuff and it looks relatively solid, I'm
wondering now if I'm using HttpClient incorrectly. Does Beta 1 help
with any memory issues?
Jan Gonsalves
Re: Java can leak...
Posted by Brett Knights <br...@knightsofthenet.com>.
A useful profiler is included with the JSDK. Just check the
documentation for the java -Xrunhprof option.
HTH
Brett Knights
----- Original Message -----
From: "Jan Gonsalves" <JG...@xtensible.com>
To: "Commons HttpClient Project"
<co...@jakarta.apache.org>
Sent: Thursday, May 29, 2003 5:45 PM
Subject: Re: Java can leak...
> We need to do more investigation on our side for sure, I just wanted
to
> know if anyone had noticed memory leaks over long recycled
connections.
> The JVM my app runs in alots 768Mb as the max, but I work in
conjunction
> with serveral apps. I'm the only app using HttpClient. When I run
and
> I really get going processing large chunks of data memory usage for
the
> system really takes a hit. My personal memory footprint based off
qps
> for linux says I max out at about 380mb. If I loose a reference to
an
> object ROOT has, does that count against my memory usage or does
that
> consume memory but no under a particular JVM?
>
> My personal belief is that my app does things a new way in our
system
> and I'm putting stress on other objects in the system that aren't
> normally accustomed to it and they are having difficulty. Anyone
know
> of a good, user friendly (read inexperienced to memory profilling)
> application?
>
> Jan
>
> Michael Becke wrote:
>
> > My impression of your first email was that the server had used up
the
> > 4GB of memory. Are you saying it was HttpClient? If so, you must
> > have run the JVM and specified the max heap size to something like
> > -Xmx4GB. If that's the case then this is probably something you
don't
> > want to do. Once Java gets over the minimum heap size, specified
by
> > -Xms, it will continue to expand all the way up to the max, even
if it
> > doesn't need to. Once the JVM allocates memory to the heap it
never
> > releases it. Is this what's happening?
> >
> > Mike
> >
> > On Thursday, May 29, 2003, at 06:09 PM, Jan Gonsalves wrote:
> >
> >> Oleg,
> >>
> >> I understand what you mean my mem alloc, but Java can leak, it's
a
> >> proven fact. Anytime ROOT has a reference to something it makes
it
> >> unable to GC, however, if you loose your reference and ROOT still
has
> >> it's, that's a memory leak. ROOT will never GC that memory. Ask
> >> youself this, if Memory leaks were impossible in JAVA, how can
> >> programs such as JProfiler or OptimizIt sell?
> >>
> >> Jan
> >>
> >> Oleg Kalnichevski wrote:
> >>
> >>> Jan,
> >>>
> >>> HttpClient is a pure Java application and as such does not
manage
> >>> memory
> >>> allocation directly. I suggest that you check if there are any
known
> >>> issues with the jvm you are using. Upgrading to HttpClient
beta-1 would
> >>> not hurt, anyways.
> >>>
> >>> Oleg
> >>>
> >>>
> >>> On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote:
> >>>
> >>>> Does anyone know of any memory issues with opening a connection
and
> >>>> continuely re-using it, only closing it when a socket timeout
> >>>> happens? I'm currently using ALPHA 3. I have to "fake out a
> >>>> browser" and navigate through a website to extract information
> >>>> because they don't offer a straight DB connection. I re-use
this
> >>>> connection the whole time, never closing it, only recycling it,
> >>>> unless I get a socket timeout, ConnectionException, or
> >>>> HTTPRecoverableException. Our Production server has 4GB of
ram.
> >>>> Last night we ran and the system went down to 4MB free, 4! I'm
> >>>> just wondering if I'm leaking due to HTTP Client. I've gone
over
> >>>> my stuff and it looks relatively solid, I'm wondering now if
I'm
> >>>> using HttpClient incorrectly. Does Beta 1 help with any memory
> >>>> issues?
> >>>>
> >>>> Jan Gonsalves
> >>>>
> >>>>
>
>>>> -----------------------------------------------------------------
----
> >>>> To unsubscribe, e-mail:
> >>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> >>>> For additional commands, e-mail:
> >>>> commons-httpclient-dev-help@jakarta.apache.org
> >>>>
> >>>>
> >>>
> >>>
>
>>> ------------------------------------------------------------------
---
> >>> To unsubscribe, e-mail:
> >>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> >>> For additional commands, e-mail:
> >>> commons-httpclient-dev-help@jakarta.apache.org
> >>>
> >>>
> >>>
> >>
> >>
>
>> -------------------------------------------------------------------
--
> >> To unsubscribe, e-mail:
> >> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> >> For additional commands, e-mail:
> >> commons-httpclient-dev-help@jakarta.apache.org
> >>
> >
> >
>
> --------------------------------------------------------------------
-
> > To unsubscribe, e-mail:
> > commons-httpclient-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > commons-httpclient-dev-help@jakarta.apache.org
> >
> >
>
>
> --------------------------------------------------------------------
-
> To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org
>
>
>
Re: Java can leak...
Posted by Jan Gonsalves <JG...@xtensible.com>.
We need to do more investigation on our side for sure, I just wanted to
know if anyone had noticed memory leaks over long recycled connections.
The JVM my app runs in alots 768Mb as the max, but I work in conjunction
with serveral apps. I'm the only app using HttpClient. When I run and
I really get going processing large chunks of data memory usage for the
system really takes a hit. My personal memory footprint based off qps
for linux says I max out at about 380mb. If I loose a reference to an
object ROOT has, does that count against my memory usage or does that
consume memory but no under a particular JVM?
My personal belief is that my app does things a new way in our system
and I'm putting stress on other objects in the system that aren't
normally accustomed to it and they are having difficulty. Anyone know
of a good, user friendly (read inexperienced to memory profilling)
application?
Jan
Michael Becke wrote:
> My impression of your first email was that the server had used up the
> 4GB of memory. Are you saying it was HttpClient? If so, you must
> have run the JVM and specified the max heap size to something like
> -Xmx4GB. If that's the case then this is probably something you don't
> want to do. Once Java gets over the minimum heap size, specified by
> -Xms, it will continue to expand all the way up to the max, even if it
> doesn't need to. Once the JVM allocates memory to the heap it never
> releases it. Is this what's happening?
>
> Mike
>
> On Thursday, May 29, 2003, at 06:09 PM, Jan Gonsalves wrote:
>
>> Oleg,
>>
>> I understand what you mean my mem alloc, but Java can leak, it's a
>> proven fact. Anytime ROOT has a reference to something it makes it
>> unable to GC, however, if you loose your reference and ROOT still has
>> it's, that's a memory leak. ROOT will never GC that memory. Ask
>> youself this, if Memory leaks were impossible in JAVA, how can
>> programs such as JProfiler or OptimizIt sell?
>>
>> Jan
>>
>> Oleg Kalnichevski wrote:
>>
>>> Jan,
>>>
>>> HttpClient is a pure Java application and as such does not manage
>>> memory
>>> allocation directly. I suggest that you check if there are any known
>>> issues with the jvm you are using. Upgrading to HttpClient beta-1 would
>>> not hurt, anyways.
>>>
>>> Oleg
>>>
>>>
>>> On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote:
>>>
>>>> Does anyone know of any memory issues with opening a connection and
>>>> continuely re-using it, only closing it when a socket timeout
>>>> happens? I'm currently using ALPHA 3. I have to "fake out a
>>>> browser" and navigate through a website to extract information
>>>> because they don't offer a straight DB connection. I re-use this
>>>> connection the whole time, never closing it, only recycling it,
>>>> unless I get a socket timeout, ConnectionException, or
>>>> HTTPRecoverableException. Our Production server has 4GB of ram.
>>>> Last night we ran and the system went down to 4MB free, 4! I'm
>>>> just wondering if I'm leaking due to HTTP Client. I've gone over
>>>> my stuff and it looks relatively solid, I'm wondering now if I'm
>>>> using HttpClient incorrectly. Does Beta 1 help with any memory
>>>> issues?
>>>>
>>>> Jan Gonsalves
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail:
>>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail:
>>>> commons-httpclient-dev-help@jakarta.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail:
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> commons-httpclient-dev-help@jakarta.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
>
>
Re: Java can leak...
Posted by Michael Becke <be...@u.washington.edu>.
My impression of your first email was that the server had used up the
4GB of memory. Are you saying it was HttpClient? If so, you must have
run the JVM and specified the max heap size to something like -Xmx4GB.
If that's the case then this is probably something you don't want to
do. Once Java gets over the minimum heap size, specified by -Xms, it
will continue to expand all the way up to the max, even if it doesn't
need to. Once the JVM allocates memory to the heap it never releases
it. Is this what's happening?
Mike
On Thursday, May 29, 2003, at 06:09 PM, Jan Gonsalves wrote:
> Oleg,
>
> I understand what you mean my mem alloc, but Java can leak, it's a
> proven fact. Anytime ROOT has a reference to something it makes it
> unable to GC, however, if you loose your reference and ROOT still has
> it's, that's a memory leak. ROOT will never GC that memory. Ask
> youself this, if Memory leaks were impossible in JAVA, how can
> programs such as JProfiler or OptimizIt sell?
>
> Jan
>
> Oleg Kalnichevski wrote:
>
>> Jan,
>>
>> HttpClient is a pure Java application and as such does not manage
>> memory
>> allocation directly. I suggest that you check if there are any known
>> issues with the jvm you are using. Upgrading to HttpClient beta-1
>> would
>> not hurt, anyways.
>>
>> Oleg
>>
>>
>> On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote:
>>
>>> Does anyone know of any memory issues with opening a connection and
>>> continuely re-using it, only closing it when a socket timeout
>>> happens? I'm currently using ALPHA 3. I have to "fake out a
>>> browser" and navigate through a website to extract information
>>> because they don't offer a straight DB connection. I re-use this
>>> connection the whole time, never closing it, only recycling it,
>>> unless I get a socket timeout, ConnectionException, or
>>> HTTPRecoverableException. Our Production server has 4GB of ram.
>>> Last night we ran and the system went down to 4MB free, 4! I'm just
>>> wondering if I'm leaking due to HTTP Client. I've gone over my
>>> stuff and it looks relatively solid, I'm wondering now if I'm using
>>> HttpClient incorrectly. Does Beta 1 help with any memory issues?
>>>
>>> Jan Gonsalves
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail:
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> commons-httpclient-dev-help@jakarta.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
>
Java can leak...
Posted by Jan Gonsalves <JG...@Xtensible.com>.
Oleg,
I understand what you mean my mem alloc, but Java can leak, it's a
proven fact. Anytime ROOT has a reference to something it makes it
unable to GC, however, if you loose your reference and ROOT still has
it's, that's a memory leak. ROOT will never GC that memory. Ask
youself this, if Memory leaks were impossible in JAVA, how can programs
such as JProfiler or OptimizIt sell?
Jan
Oleg Kalnichevski wrote:
>Jan,
>
>HttpClient is a pure Java application and as such does not manage memory
>allocation directly. I suggest that you check if there are any known
>issues with the jvm you are using. Upgrading to HttpClient beta-1 would
>not hurt, anyways.
>
>Oleg
>
>
>On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote:
>
>
>>Does anyone know of any memory issues with opening a connection and
>>continuely re-using it, only closing it when a socket timeout happens?
>>I'm currently using ALPHA 3. I have to "fake out a browser" and
>>navigate through a website to extract information because they don't
>>offer a straight DB connection. I re-use this connection the whole
>>time, never closing it, only recycling it, unless I get a socket
>>timeout, ConnectionException, or HTTPRecoverableException. Our
>>Production server has 4GB of ram. Last night we ran and the system went
>>down to 4MB free, 4! I'm just wondering if I'm leaking due to HTTP
>>Client. I've gone over my stuff and it looks relatively solid, I'm
>>wondering now if I'm using HttpClient incorrectly. Does Beta 1 help
>>with any memory issues?
>>
>>Jan Gonsalves
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
>>
>>
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
>
>
>
>
Re: Memory issues with long connections?
Posted by Oleg Kalnichevski <ol...@apache.org>.
Jan,
HttpClient is a pure Java application and as such does not manage memory
allocation directly. I suggest that you check if there are any known
issues with the jvm you are using. Upgrading to HttpClient beta-1 would
not hurt, anyways.
Oleg
On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote:
> Does anyone know of any memory issues with opening a connection and
> continuely re-using it, only closing it when a socket timeout happens?
> I'm currently using ALPHA 3. I have to "fake out a browser" and
> navigate through a website to extract information because they don't
> offer a straight DB connection. I re-use this connection the whole
> time, never closing it, only recycling it, unless I get a socket
> timeout, ConnectionException, or HTTPRecoverableException. Our
> Production server has 4GB of ram. Last night we ran and the system went
> down to 4MB free, 4! I'm just wondering if I'm leaking due to HTTP
> Client. I've gone over my stuff and it looks relatively solid, I'm
> wondering now if I'm using HttpClient incorrectly. Does Beta 1 help
> with any memory issues?
>
> Jan Gonsalves
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
>