You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by unjc email <un...@gmail.com> on 2012/10/10 18:21:22 UTC
HTTP Sampler Memory Leak
Hi all,
I have memory and cpu issues when running heavy load testing with
HTTP(S) sampler. The purpose of my test is to hammer the server as
fast as Jmeter can, using a big list of hostnames and URIs, via a
proxy server. The test plan I have setup is configured with
100-thread thread group that contains a HTTPS Request sampler (using
HTTPClient4) and Summary Results generator.
The machine I am using has 12GB memory and 16-core cpu. I monitor the
top output throughout the test. I notice the RES size of the jmeter
java process is growing really fast - it reaches 2GB heap size limit
in less 5 minutes. The test could never go beyond 30-minute mark
without OutOfMemory exception. I know there are plenty of free memory
for me to expand the heap size; but my concern is whether there is a
memory leak in the code. I worry the process would suck up all
available memory anyway if I run a stress test for a day or longer.
I have skimmed through the code of HTTPHC4Impl and found that there is
cache "map" for HttpClient with regard to URL. I suspect this cache
map could go pretty crazy in tests that deal with large number of (or
dynamic) URLs. Just curiosity, I tried running a test with using a
custom sampler as like HTTPHC4Impl but without the cache map; it seems
help slowing down the pace of the heap size. However, the heap size
(RES size in top output) never seem settle at one level over time.
Does anyone have any experience in running similar test as I do? Does
it look like a potential "memory leak" to you? Any comment or advice
will be appreciated.
Thanks,
Jacky
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by Philippe Mouawad <ph...@gmail.com>.
Another question, which version are you using ?
On Wed, Oct 10, 2012 at 9:20 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> Hello,
> Which map are you talking about in HTTPHC4Impl ?
>
> If it's this one:
>
> - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
>
> Then it only exists once per thread and per combination of :
>
> - url.getProtocol()+"://"+url.getAuthority();
> this.hasProxy = b;
> this.proxyHost = proxyHost;
> this.proxyPort = proxyPort;
> this.proxyUser = proxyUser;
> this.proxyPass = proxyPass;
>
> so unless you change these per url , it won't grow a lot.
>
> Check what Deepak is suggesting and if issue persists, then generate a
> Heap Dump and put it somewhere where we can download it.
>
> Regards
> Philippe
>
> On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com> wrote:
>
>> Hi
>> its more likely that you have some listener turned on (like view results
>> tree) - Even if a URL is 2000 characters you can calculate how many URLs
>> before you use up say 100MB of memory. Do you really have that many URLs
>> even if they are dynamic?
>> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>>
>> regards
>> deepak
>>
>> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com> wrote:
>>
>> > Hi all,
>> >
>> > I have memory and cpu issues when running heavy load testing with
>> > HTTP(S) sampler. The purpose of my test is to hammer the server as
>> > fast as Jmeter can, using a big list of hostnames and URIs, via a
>> > proxy server. The test plan I have setup is configured with
>> > 100-thread thread group that contains a HTTPS Request sampler (using
>> > HTTPClient4) and Summary Results generator.
>> >
>> > The machine I am using has 12GB memory and 16-core cpu. I monitor the
>> > top output throughout the test. I notice the RES size of the jmeter
>> > java process is growing really fast - it reaches 2GB heap size limit
>> > in less 5 minutes. The test could never go beyond 30-minute mark
>> > without OutOfMemory exception. I know there are plenty of free memory
>> > for me to expand the heap size; but my concern is whether there is a
>> > memory leak in the code. I worry the process would suck up all
>> > available memory anyway if I run a stress test for a day or longer.
>> >
>> > I have skimmed through the code of HTTPHC4Impl and found that there is
>> > cache "map" for HttpClient with regard to URL. I suspect this cache
>> > map could go pretty crazy in tests that deal with large number of (or
>> > dynamic) URLs. Just curiosity, I tried running a test with using a
>> > custom sampler as like HTTPHC4Impl but without the cache map; it seems
>> > help slowing down the pace of the heap size. However, the heap size
>> > (RES size in top output) never seem settle at one level over time.
>> >
>> > Does anyone have any experience in running similar test as I do? Does
>> > it look like a potential "memory leak" to you? Any comment or advice
>> > will be appreciated.
>> >
>> >
>> >
>> >
>> > Thanks,
>> > Jacky
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> > For additional commands, e-mail: user-help@jmeter.apache.org
>> >
>> >
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: HTTP Sampler Memory Leak
Posted by Deepak Shetty <sh...@gmail.com>.
Hi
im curious what did you open this dump in (I get invalid hprof version on
Mem Analyzer in eclipse)
Does this caching apply only to HTTPClient based samplers or even the other
HTTP Request?
regards
deepak
On Thu, Oct 11, 2012 at 5:07 AM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> Hello,
> I looked at your heap dump, it seems you are load testing lots of
> differents WEBSITES (I mean HOSTs are all different).
> This is clearly a very "weird" use case that we don't handle for now, and I
> am not sure we will.
>
> Regards
> Philippe
>
>
> On Thu, Oct 11, 2012 at 12:10 AM, unjc email <un...@gmail.com> wrote:
>
> > I have uploaded the heap dump file online and it's available at
> > http://dl.free.fr/qZBPvLRB8.
> >
> >
> >
> > Thanks,
> > Jacky
> >
> > On Wed, Oct 10, 2012 at 3:20 PM, Philippe Mouawad
> > <ph...@gmail.com> wrote:
> > > Hello,
> > > Which map are you talking about in HTTPHC4Impl ?
> > >
> > > If it's this one:
> > >
> > > - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
> > >
> > > Then it only exists once per thread and per combination of :
> > >
> > > - url.getProtocol()+"://"+url.getAuthority();
> > > this.hasProxy = b;
> > > this.proxyHost = proxyHost;
> > > this.proxyPort = proxyPort;
> > > this.proxyUser = proxyUser;
> > > this.proxyPass = proxyPass;
> > >
> > > so unless you change these per url , it won't grow a lot.
> > >
> > > Check what Deepak is suggesting and if issue persists, then generate a
> > Heap
> > > Dump and put it somewhere where we can download it.
> > >
> > > Regards
> > > Philippe
> > >
> > > On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com>
> > wrote:
> > >
> > >> Hi
> > >> its more likely that you have some listener turned on (like view
> results
> > >> tree) - Even if a URL is 2000 characters you can calculate how many
> URLs
> > >> before you use up say 100MB of memory. Do you really have that many
> URLs
> > >> even if they are dynamic?
> > >> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
> > >>
> > >> regards
> > >> deepak
> > >>
> > >> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com>
> > wrote:
> > >>
> > >> > Hi all,
> > >> >
> > >> > I have memory and cpu issues when running heavy load testing with
> > >> > HTTP(S) sampler. The purpose of my test is to hammer the server as
> > >> > fast as Jmeter can, using a big list of hostnames and URIs, via a
> > >> > proxy server. The test plan I have setup is configured with
> > >> > 100-thread thread group that contains a HTTPS Request sampler (using
> > >> > HTTPClient4) and Summary Results generator.
> > >> >
> > >> > The machine I am using has 12GB memory and 16-core cpu. I monitor
> the
> > >> > top output throughout the test. I notice the RES size of the jmeter
> > >> > java process is growing really fast - it reaches 2GB heap size limit
> > >> > in less 5 minutes. The test could never go beyond 30-minute mark
> > >> > without OutOfMemory exception. I know there are plenty of free
> memory
> > >> > for me to expand the heap size; but my concern is whether there is a
> > >> > memory leak in the code. I worry the process would suck up all
> > >> > available memory anyway if I run a stress test for a day or longer.
> > >> >
> > >> > I have skimmed through the code of HTTPHC4Impl and found that there
> is
> > >> > cache "map" for HttpClient with regard to URL. I suspect this cache
> > >> > map could go pretty crazy in tests that deal with large number of
> (or
> > >> > dynamic) URLs. Just curiosity, I tried running a test with using a
> > >> > custom sampler as like HTTPHC4Impl but without the cache map; it
> seems
> > >> > help slowing down the pace of the heap size. However, the heap size
> > >> > (RES size in top output) never seem settle at one level over time.
> > >> >
> > >> > Does anyone have any experience in running similar test as I do?
> Does
> > >> > it look like a potential "memory leak" to you? Any comment or
> advice
> > >> > will be appreciated.
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > Thanks,
> > >> > Jacky
> > >> >
> > >> >
> ---------------------------------------------------------------------
> > >> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > >> > For additional commands, e-mail: user-help@jmeter.apache.org
> > >> >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Cordialement.
> > > Philippe Mouawad.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
> >
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
Re: HTTP Sampler Memory Leak
Posted by Philippe Mouawad <ph...@gmail.com>.
Try using Java Impl, as it does not have this thread local Map,it could
work.
On Mon, Oct 15, 2012 at 4:01 PM, unjc email <un...@gmail.com> wrote:
> Hi Deepak,
>
> I am not using any custom sampler nor any custom code in the test dump
> I sent. The sampler I used is HTTP Request Sampler with HttpClient4
> Implementation (no request parameters); I used a static proxy server
> though. I am not sure what specific code that has the leak you are
> referring. Could you explain more?
>
>
>
> Thanks,
> Jacky
>
> On Sat, Oct 13, 2012 at 3:47 PM, Deepak Goel <de...@gmail.com> wrote:
> > Hey
> >
> > It looks like you are creating a lot of new objects and not closing
> > them. The transaction most used by you, should have the code which is
> > leaking
> >
> > :)
> > Deepak
> >
> > On 10/12/12, sebb <se...@gmail.com> wrote:
> >> On 11 October 2012 15:02, unjc email <un...@gmail.com> wrote:
> >>> Hi Philippe,
> >>>
> >>> Thanks for your time on analyzing the heap dump. It is unfortunate to
> >>> hear that there will be no support of my test case from Jmeter.
> >>
> >> The problem is knowing when it is safe to drop a cache entry.
> >>
> >>> I suppose I could try to write my own plugin targeted on the
> >>> performance for my test case. Is there any chance you could give me
> >>> advice on which part of the code I should look into, in order to keep
> >>> the memory utilization at a low level for my test?
> >>
> >> If you want to experiment, you could edit the class to ignore the cache.
> >> However this might have other adverse effects.
> >>
> >>>
> >>> Thanks and much appreciated.
> >>>
> >>> Jacky
> >>>
> >>> On Thu, Oct 11, 2012 at 8:07 AM, Philippe Mouawad
> >>> <ph...@gmail.com> wrote:
> >>>> Hello,
> >>>> I looked at your heap dump, it seems you are load testing lots of
> >>>> differents WEBSITES (I mean HOSTs are all different).
> >>>> This is clearly a very "weird" use case that we don't handle for now,
> and
> >>>> I
> >>>> am not sure we will.
> >>>>
> >>>> Regards
> >>>> Philippe
> >>>>
> >>>>
> >>>> On Thu, Oct 11, 2012 at 12:10 AM, unjc email <un...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> I have uploaded the heap dump file online and it's available at
> >>>>> http://dl.free.fr/qZBPvLRB8.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>> Jacky
> >>>>>
> >>>>> On Wed, Oct 10, 2012 at 3:20 PM, Philippe Mouawad
> >>>>> <ph...@gmail.com> wrote:
> >>>>> > Hello,
> >>>>> > Which map are you talking about in HTTPHC4Impl ?
> >>>>> >
> >>>>> > If it's this one:
> >>>>> >
> >>>>> > - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
> >>>>> >
> >>>>> > Then it only exists once per thread and per combination of :
> >>>>> >
> >>>>> > - url.getProtocol()+"://"+url.getAuthority();
> >>>>> > this.hasProxy = b;
> >>>>> > this.proxyHost = proxyHost;
> >>>>> > this.proxyPort = proxyPort;
> >>>>> > this.proxyUser = proxyUser;
> >>>>> > this.proxyPass = proxyPass;
> >>>>> >
> >>>>> > so unless you change these per url , it won't grow a lot.
> >>>>> >
> >>>>> > Check what Deepak is suggesting and if issue persists, then
> generate
> >>>>> > a
> >>>>> Heap
> >>>>> > Dump and put it somewhere where we can download it.
> >>>>> >
> >>>>> > Regards
> >>>>> > Philippe
> >>>>> >
> >>>>> > On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com>
> >>>>> wrote:
> >>>>> >
> >>>>> >> Hi
> >>>>> >> its more likely that you have some listener turned on (like view
> >>>>> >> results
> >>>>> >> tree) - Even if a URL is 2000 characters you can calculate how
> many
> >>>>> >> URLs
> >>>>> >> before you use up say 100MB of memory. Do you really have that
> many
> >>>>> >> URLs
> >>>>> >> even if they are dynamic?
> >>>>> >> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
> >>>>> >>
> >>>>> >> regards
> >>>>> >> deepak
> >>>>> >>
> >>>>> >> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <unjc.email@gmail.com
> >
> >>>>> wrote:
> >>>>> >>
> >>>>> >> > Hi all,
> >>>>> >> >
> >>>>> >> > I have memory and cpu issues when running heavy load testing
> with
> >>>>> >> > HTTP(S) sampler. The purpose of my test is to hammer the server
> >>>>> >> > as
> >>>>> >> > fast as Jmeter can, using a big list of hostnames and URIs, via
> a
> >>>>> >> > proxy server. The test plan I have setup is configured with
> >>>>> >> > 100-thread thread group that contains a HTTPS Request sampler
> >>>>> >> > (using
> >>>>> >> > HTTPClient4) and Summary Results generator.
> >>>>> >> >
> >>>>> >> > The machine I am using has 12GB memory and 16-core cpu. I
> monitor
> >>>>> >> > the
> >>>>> >> > top output throughout the test. I notice the RES size of the
> >>>>> >> > jmeter
> >>>>> >> > java process is growing really fast - it reaches 2GB heap size
> >>>>> >> > limit
> >>>>> >> > in less 5 minutes. The test could never go beyond 30-minute
> mark
> >>>>> >> > without OutOfMemory exception. I know there are plenty of free
> >>>>> >> > memory
> >>>>> >> > for me to expand the heap size; but my concern is whether there
> is
> >>>>> >> > a
> >>>>> >> > memory leak in the code. I worry the process would suck up all
> >>>>> >> > available memory anyway if I run a stress test for a day or
> >>>>> >> > longer.
> >>>>> >> >
> >>>>> >> > I have skimmed through the code of HTTPHC4Impl and found that
> there
> >>>>> >> > is
> >>>>> >> > cache "map" for HttpClient with regard to URL. I suspect this
> >>>>> >> > cache
> >>>>> >> > map could go pretty crazy in tests that deal with large number
> of
> >>>>> >> > (or
> >>>>> >> > dynamic) URLs. Just curiosity, I tried running a test with
> using
> >>>>> >> > a
> >>>>> >> > custom sampler as like HTTPHC4Impl but without the cache map; it
> >>>>> >> > seems
> >>>>> >> > help slowing down the pace of the heap size. However, the heap
> >>>>> >> > size
> >>>>> >> > (RES size in top output) never seem settle at one level over
> time.
> >>>>> >> >
> >>>>> >> > Does anyone have any experience in running similar test as I do?
> >>>>> >> > Does
> >>>>> >> > it look like a potential "memory leak" to you? Any comment or
> >>>>> >> > advice
> >>>>> >> > will be appreciated.
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > Thanks,
> >>>>> >> > Jacky
> >>>>> >> >
> >>>>> >> >
> ---------------------------------------------------------------------
> >>>>> >> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>>> >> > For additional commands, e-mail: user-help@jmeter.apache.org
> >>>>> >> >
> >>>>> >> >
> >>>>> >>
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > --
> >>>>> > Cordialement.
> >>>>> > Philippe Mouawad.
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Cordialement.
> >>>> Philippe Mouawad.
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >> For additional commands, e-mail: user-help@jmeter.apache.org
> >>
> >>
> >
> >
> > --
> > Namaskara~Nalama~Guten Tag~Bonjour
> >
> >
> > --
> > Keigu
> >
> > Deepak
> > +91-9765089593
> > deicool@gmail.com
> > http://www.simtree.net
> >
> > Skype: thumsupdeicool
> > Google talk: deicool
> > Blog: http://loveandfearless.wordpress.com
> > Facebook: http://www.facebook.com/deicool
> >
> > "Contribute to the world, environment and more :
> http://www.gridrepublic.org
> > "
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
--
Cordialement.
Philippe Mouawad.
Re: HTTP Sampler Memory Leak
Posted by unjc email <un...@gmail.com>.
Hi Deepak,
I am not using any custom sampler nor any custom code in the test dump
I sent. The sampler I used is HTTP Request Sampler with HttpClient4
Implementation (no request parameters); I used a static proxy server
though. I am not sure what specific code that has the leak you are
referring. Could you explain more?
Thanks,
Jacky
On Sat, Oct 13, 2012 at 3:47 PM, Deepak Goel <de...@gmail.com> wrote:
> Hey
>
> It looks like you are creating a lot of new objects and not closing
> them. The transaction most used by you, should have the code which is
> leaking
>
> :)
> Deepak
>
> On 10/12/12, sebb <se...@gmail.com> wrote:
>> On 11 October 2012 15:02, unjc email <un...@gmail.com> wrote:
>>> Hi Philippe,
>>>
>>> Thanks for your time on analyzing the heap dump. It is unfortunate to
>>> hear that there will be no support of my test case from Jmeter.
>>
>> The problem is knowing when it is safe to drop a cache entry.
>>
>>> I suppose I could try to write my own plugin targeted on the
>>> performance for my test case. Is there any chance you could give me
>>> advice on which part of the code I should look into, in order to keep
>>> the memory utilization at a low level for my test?
>>
>> If you want to experiment, you could edit the class to ignore the cache.
>> However this might have other adverse effects.
>>
>>>
>>> Thanks and much appreciated.
>>>
>>> Jacky
>>>
>>> On Thu, Oct 11, 2012 at 8:07 AM, Philippe Mouawad
>>> <ph...@gmail.com> wrote:
>>>> Hello,
>>>> I looked at your heap dump, it seems you are load testing lots of
>>>> differents WEBSITES (I mean HOSTs are all different).
>>>> This is clearly a very "weird" use case that we don't handle for now, and
>>>> I
>>>> am not sure we will.
>>>>
>>>> Regards
>>>> Philippe
>>>>
>>>>
>>>> On Thu, Oct 11, 2012 at 12:10 AM, unjc email <un...@gmail.com>
>>>> wrote:
>>>>
>>>>> I have uploaded the heap dump file online and it's available at
>>>>> http://dl.free.fr/qZBPvLRB8.
>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Jacky
>>>>>
>>>>> On Wed, Oct 10, 2012 at 3:20 PM, Philippe Mouawad
>>>>> <ph...@gmail.com> wrote:
>>>>> > Hello,
>>>>> > Which map are you talking about in HTTPHC4Impl ?
>>>>> >
>>>>> > If it's this one:
>>>>> >
>>>>> > - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
>>>>> >
>>>>> > Then it only exists once per thread and per combination of :
>>>>> >
>>>>> > - url.getProtocol()+"://"+url.getAuthority();
>>>>> > this.hasProxy = b;
>>>>> > this.proxyHost = proxyHost;
>>>>> > this.proxyPort = proxyPort;
>>>>> > this.proxyUser = proxyUser;
>>>>> > this.proxyPass = proxyPass;
>>>>> >
>>>>> > so unless you change these per url , it won't grow a lot.
>>>>> >
>>>>> > Check what Deepak is suggesting and if issue persists, then generate
>>>>> > a
>>>>> Heap
>>>>> > Dump and put it somewhere where we can download it.
>>>>> >
>>>>> > Regards
>>>>> > Philippe
>>>>> >
>>>>> > On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> >> Hi
>>>>> >> its more likely that you have some listener turned on (like view
>>>>> >> results
>>>>> >> tree) - Even if a URL is 2000 characters you can calculate how many
>>>>> >> URLs
>>>>> >> before you use up say 100MB of memory. Do you really have that many
>>>>> >> URLs
>>>>> >> even if they are dynamic?
>>>>> >> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>>>>> >>
>>>>> >> regards
>>>>> >> deepak
>>>>> >>
>>>>> >> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> > Hi all,
>>>>> >> >
>>>>> >> > I have memory and cpu issues when running heavy load testing with
>>>>> >> > HTTP(S) sampler. The purpose of my test is to hammer the server
>>>>> >> > as
>>>>> >> > fast as Jmeter can, using a big list of hostnames and URIs, via a
>>>>> >> > proxy server. The test plan I have setup is configured with
>>>>> >> > 100-thread thread group that contains a HTTPS Request sampler
>>>>> >> > (using
>>>>> >> > HTTPClient4) and Summary Results generator.
>>>>> >> >
>>>>> >> > The machine I am using has 12GB memory and 16-core cpu. I monitor
>>>>> >> > the
>>>>> >> > top output throughout the test. I notice the RES size of the
>>>>> >> > jmeter
>>>>> >> > java process is growing really fast - it reaches 2GB heap size
>>>>> >> > limit
>>>>> >> > in less 5 minutes. The test could never go beyond 30-minute mark
>>>>> >> > without OutOfMemory exception. I know there are plenty of free
>>>>> >> > memory
>>>>> >> > for me to expand the heap size; but my concern is whether there is
>>>>> >> > a
>>>>> >> > memory leak in the code. I worry the process would suck up all
>>>>> >> > available memory anyway if I run a stress test for a day or
>>>>> >> > longer.
>>>>> >> >
>>>>> >> > I have skimmed through the code of HTTPHC4Impl and found that there
>>>>> >> > is
>>>>> >> > cache "map" for HttpClient with regard to URL. I suspect this
>>>>> >> > cache
>>>>> >> > map could go pretty crazy in tests that deal with large number of
>>>>> >> > (or
>>>>> >> > dynamic) URLs. Just curiosity, I tried running a test with using
>>>>> >> > a
>>>>> >> > custom sampler as like HTTPHC4Impl but without the cache map; it
>>>>> >> > seems
>>>>> >> > help slowing down the pace of the heap size. However, the heap
>>>>> >> > size
>>>>> >> > (RES size in top output) never seem settle at one level over time.
>>>>> >> >
>>>>> >> > Does anyone have any experience in running similar test as I do?
>>>>> >> > Does
>>>>> >> > it look like a potential "memory leak" to you? Any comment or
>>>>> >> > advice
>>>>> >> > will be appreciated.
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> > Thanks,
>>>>> >> > Jacky
>>>>> >> >
>>>>> >> > ---------------------------------------------------------------------
>>>>> >> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>>> >> > For additional commands, e-mail: user-help@jmeter.apache.org
>>>>> >> >
>>>>> >> >
>>>>> >>
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Cordialement.
>>>>> > Philippe Mouawad.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Cordialement.
>>>> Philippe Mouawad.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
>
>
> --
> Namaskara~Nalama~Guten Tag~Bonjour
>
>
> --
> Keigu
>
> Deepak
> +91-9765089593
> deicool@gmail.com
> http://www.simtree.net
>
> Skype: thumsupdeicool
> Google talk: deicool
> Blog: http://loveandfearless.wordpress.com
> Facebook: http://www.facebook.com/deicool
>
> "Contribute to the world, environment and more : http://www.gridrepublic.org
> "
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by Deepak Goel <de...@gmail.com>.
Hey
It looks like you are creating a lot of new objects and not closing
them. The transaction most used by you, should have the code which is
leaking
:)
Deepak
On 10/12/12, sebb <se...@gmail.com> wrote:
> On 11 October 2012 15:02, unjc email <un...@gmail.com> wrote:
>> Hi Philippe,
>>
>> Thanks for your time on analyzing the heap dump. It is unfortunate to
>> hear that there will be no support of my test case from Jmeter.
>
> The problem is knowing when it is safe to drop a cache entry.
>
>> I suppose I could try to write my own plugin targeted on the
>> performance for my test case. Is there any chance you could give me
>> advice on which part of the code I should look into, in order to keep
>> the memory utilization at a low level for my test?
>
> If you want to experiment, you could edit the class to ignore the cache.
> However this might have other adverse effects.
>
>>
>> Thanks and much appreciated.
>>
>> Jacky
>>
>> On Thu, Oct 11, 2012 at 8:07 AM, Philippe Mouawad
>> <ph...@gmail.com> wrote:
>>> Hello,
>>> I looked at your heap dump, it seems you are load testing lots of
>>> differents WEBSITES (I mean HOSTs are all different).
>>> This is clearly a very "weird" use case that we don't handle for now, and
>>> I
>>> am not sure we will.
>>>
>>> Regards
>>> Philippe
>>>
>>>
>>> On Thu, Oct 11, 2012 at 12:10 AM, unjc email <un...@gmail.com>
>>> wrote:
>>>
>>>> I have uploaded the heap dump file online and it's available at
>>>> http://dl.free.fr/qZBPvLRB8.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>> Jacky
>>>>
>>>> On Wed, Oct 10, 2012 at 3:20 PM, Philippe Mouawad
>>>> <ph...@gmail.com> wrote:
>>>> > Hello,
>>>> > Which map are you talking about in HTTPHC4Impl ?
>>>> >
>>>> > If it's this one:
>>>> >
>>>> > - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
>>>> >
>>>> > Then it only exists once per thread and per combination of :
>>>> >
>>>> > - url.getProtocol()+"://"+url.getAuthority();
>>>> > this.hasProxy = b;
>>>> > this.proxyHost = proxyHost;
>>>> > this.proxyPort = proxyPort;
>>>> > this.proxyUser = proxyUser;
>>>> > this.proxyPass = proxyPass;
>>>> >
>>>> > so unless you change these per url , it won't grow a lot.
>>>> >
>>>> > Check what Deepak is suggesting and if issue persists, then generate
>>>> > a
>>>> Heap
>>>> > Dump and put it somewhere where we can download it.
>>>> >
>>>> > Regards
>>>> > Philippe
>>>> >
>>>> > On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com>
>>>> wrote:
>>>> >
>>>> >> Hi
>>>> >> its more likely that you have some listener turned on (like view
>>>> >> results
>>>> >> tree) - Even if a URL is 2000 characters you can calculate how many
>>>> >> URLs
>>>> >> before you use up say 100MB of memory. Do you really have that many
>>>> >> URLs
>>>> >> even if they are dynamic?
>>>> >> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>>>> >>
>>>> >> regards
>>>> >> deepak
>>>> >>
>>>> >> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> > Hi all,
>>>> >> >
>>>> >> > I have memory and cpu issues when running heavy load testing with
>>>> >> > HTTP(S) sampler. The purpose of my test is to hammer the server
>>>> >> > as
>>>> >> > fast as Jmeter can, using a big list of hostnames and URIs, via a
>>>> >> > proxy server. The test plan I have setup is configured with
>>>> >> > 100-thread thread group that contains a HTTPS Request sampler
>>>> >> > (using
>>>> >> > HTTPClient4) and Summary Results generator.
>>>> >> >
>>>> >> > The machine I am using has 12GB memory and 16-core cpu. I monitor
>>>> >> > the
>>>> >> > top output throughout the test. I notice the RES size of the
>>>> >> > jmeter
>>>> >> > java process is growing really fast - it reaches 2GB heap size
>>>> >> > limit
>>>> >> > in less 5 minutes. The test could never go beyond 30-minute mark
>>>> >> > without OutOfMemory exception. I know there are plenty of free
>>>> >> > memory
>>>> >> > for me to expand the heap size; but my concern is whether there is
>>>> >> > a
>>>> >> > memory leak in the code. I worry the process would suck up all
>>>> >> > available memory anyway if I run a stress test for a day or
>>>> >> > longer.
>>>> >> >
>>>> >> > I have skimmed through the code of HTTPHC4Impl and found that there
>>>> >> > is
>>>> >> > cache "map" for HttpClient with regard to URL. I suspect this
>>>> >> > cache
>>>> >> > map could go pretty crazy in tests that deal with large number of
>>>> >> > (or
>>>> >> > dynamic) URLs. Just curiosity, I tried running a test with using
>>>> >> > a
>>>> >> > custom sampler as like HTTPHC4Impl but without the cache map; it
>>>> >> > seems
>>>> >> > help slowing down the pace of the heap size. However, the heap
>>>> >> > size
>>>> >> > (RES size in top output) never seem settle at one level over time.
>>>> >> >
>>>> >> > Does anyone have any experience in running similar test as I do?
>>>> >> > Does
>>>> >> > it look like a potential "memory leak" to you? Any comment or
>>>> >> > advice
>>>> >> > will be appreciated.
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > Thanks,
>>>> >> > Jacky
>>>> >> >
>>>> >> > ---------------------------------------------------------------------
>>>> >> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> >> > For additional commands, e-mail: user-help@jmeter.apache.org
>>>> >> >
>>>> >> >
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Cordialement.
>>>> > Philippe Mouawad.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Cordialement.
>>> Philippe Mouawad.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
--
Namaskara~Nalama~Guten Tag~Bonjour
--
Keigu
Deepak
+91-9765089593
deicool@gmail.com
http://www.simtree.net
Skype: thumsupdeicool
Google talk: deicool
Blog: http://loveandfearless.wordpress.com
Facebook: http://www.facebook.com/deicool
"Contribute to the world, environment and more : http://www.gridrepublic.org
"
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by sebb <se...@gmail.com>.
On 11 October 2012 15:02, unjc email <un...@gmail.com> wrote:
> Hi Philippe,
>
> Thanks for your time on analyzing the heap dump. It is unfortunate to
> hear that there will be no support of my test case from Jmeter.
The problem is knowing when it is safe to drop a cache entry.
> I suppose I could try to write my own plugin targeted on the
> performance for my test case. Is there any chance you could give me
> advice on which part of the code I should look into, in order to keep
> the memory utilization at a low level for my test?
If you want to experiment, you could edit the class to ignore the cache.
However this might have other adverse effects.
>
> Thanks and much appreciated.
>
> Jacky
>
> On Thu, Oct 11, 2012 at 8:07 AM, Philippe Mouawad
> <ph...@gmail.com> wrote:
>> Hello,
>> I looked at your heap dump, it seems you are load testing lots of
>> differents WEBSITES (I mean HOSTs are all different).
>> This is clearly a very "weird" use case that we don't handle for now, and I
>> am not sure we will.
>>
>> Regards
>> Philippe
>>
>>
>> On Thu, Oct 11, 2012 at 12:10 AM, unjc email <un...@gmail.com> wrote:
>>
>>> I have uploaded the heap dump file online and it's available at
>>> http://dl.free.fr/qZBPvLRB8.
>>>
>>>
>>>
>>> Thanks,
>>> Jacky
>>>
>>> On Wed, Oct 10, 2012 at 3:20 PM, Philippe Mouawad
>>> <ph...@gmail.com> wrote:
>>> > Hello,
>>> > Which map are you talking about in HTTPHC4Impl ?
>>> >
>>> > If it's this one:
>>> >
>>> > - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
>>> >
>>> > Then it only exists once per thread and per combination of :
>>> >
>>> > - url.getProtocol()+"://"+url.getAuthority();
>>> > this.hasProxy = b;
>>> > this.proxyHost = proxyHost;
>>> > this.proxyPort = proxyPort;
>>> > this.proxyUser = proxyUser;
>>> > this.proxyPass = proxyPass;
>>> >
>>> > so unless you change these per url , it won't grow a lot.
>>> >
>>> > Check what Deepak is suggesting and if issue persists, then generate a
>>> Heap
>>> > Dump and put it somewhere where we can download it.
>>> >
>>> > Regards
>>> > Philippe
>>> >
>>> > On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com>
>>> wrote:
>>> >
>>> >> Hi
>>> >> its more likely that you have some listener turned on (like view results
>>> >> tree) - Even if a URL is 2000 characters you can calculate how many URLs
>>> >> before you use up say 100MB of memory. Do you really have that many URLs
>>> >> even if they are dynamic?
>>> >> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>>> >>
>>> >> regards
>>> >> deepak
>>> >>
>>> >> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com>
>>> wrote:
>>> >>
>>> >> > Hi all,
>>> >> >
>>> >> > I have memory and cpu issues when running heavy load testing with
>>> >> > HTTP(S) sampler. The purpose of my test is to hammer the server as
>>> >> > fast as Jmeter can, using a big list of hostnames and URIs, via a
>>> >> > proxy server. The test plan I have setup is configured with
>>> >> > 100-thread thread group that contains a HTTPS Request sampler (using
>>> >> > HTTPClient4) and Summary Results generator.
>>> >> >
>>> >> > The machine I am using has 12GB memory and 16-core cpu. I monitor the
>>> >> > top output throughout the test. I notice the RES size of the jmeter
>>> >> > java process is growing really fast - it reaches 2GB heap size limit
>>> >> > in less 5 minutes. The test could never go beyond 30-minute mark
>>> >> > without OutOfMemory exception. I know there are plenty of free memory
>>> >> > for me to expand the heap size; but my concern is whether there is a
>>> >> > memory leak in the code. I worry the process would suck up all
>>> >> > available memory anyway if I run a stress test for a day or longer.
>>> >> >
>>> >> > I have skimmed through the code of HTTPHC4Impl and found that there is
>>> >> > cache "map" for HttpClient with regard to URL. I suspect this cache
>>> >> > map could go pretty crazy in tests that deal with large number of (or
>>> >> > dynamic) URLs. Just curiosity, I tried running a test with using a
>>> >> > custom sampler as like HTTPHC4Impl but without the cache map; it seems
>>> >> > help slowing down the pace of the heap size. However, the heap size
>>> >> > (RES size in top output) never seem settle at one level over time.
>>> >> >
>>> >> > Does anyone have any experience in running similar test as I do? Does
>>> >> > it look like a potential "memory leak" to you? Any comment or advice
>>> >> > will be appreciated.
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > Thanks,
>>> >> > Jacky
>>> >> >
>>> >> > ---------------------------------------------------------------------
>>> >> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> >> > For additional commands, e-mail: user-help@jmeter.apache.org
>>> >> >
>>> >> >
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>
>>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by unjc email <un...@gmail.com>.
Hi Philippe,
Thanks for your time on analyzing the heap dump. It is unfortunate to
hear that there will be no support of my test case from Jmeter.
I suppose I could try to write my own plugin targeted on the
performance for my test case. Is there any chance you could give me
advice on which part of the code I should look into, in order to keep
the memory utilization at a low level for my test?
Thanks and much appreciated.
Jacky
On Thu, Oct 11, 2012 at 8:07 AM, Philippe Mouawad
<ph...@gmail.com> wrote:
> Hello,
> I looked at your heap dump, it seems you are load testing lots of
> differents WEBSITES (I mean HOSTs are all different).
> This is clearly a very "weird" use case that we don't handle for now, and I
> am not sure we will.
>
> Regards
> Philippe
>
>
> On Thu, Oct 11, 2012 at 12:10 AM, unjc email <un...@gmail.com> wrote:
>
>> I have uploaded the heap dump file online and it's available at
>> http://dl.free.fr/qZBPvLRB8.
>>
>>
>>
>> Thanks,
>> Jacky
>>
>> On Wed, Oct 10, 2012 at 3:20 PM, Philippe Mouawad
>> <ph...@gmail.com> wrote:
>> > Hello,
>> > Which map are you talking about in HTTPHC4Impl ?
>> >
>> > If it's this one:
>> >
>> > - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
>> >
>> > Then it only exists once per thread and per combination of :
>> >
>> > - url.getProtocol()+"://"+url.getAuthority();
>> > this.hasProxy = b;
>> > this.proxyHost = proxyHost;
>> > this.proxyPort = proxyPort;
>> > this.proxyUser = proxyUser;
>> > this.proxyPass = proxyPass;
>> >
>> > so unless you change these per url , it won't grow a lot.
>> >
>> > Check what Deepak is suggesting and if issue persists, then generate a
>> Heap
>> > Dump and put it somewhere where we can download it.
>> >
>> > Regards
>> > Philippe
>> >
>> > On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com>
>> wrote:
>> >
>> >> Hi
>> >> its more likely that you have some listener turned on (like view results
>> >> tree) - Even if a URL is 2000 characters you can calculate how many URLs
>> >> before you use up say 100MB of memory. Do you really have that many URLs
>> >> even if they are dynamic?
>> >> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>> >>
>> >> regards
>> >> deepak
>> >>
>> >> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com>
>> wrote:
>> >>
>> >> > Hi all,
>> >> >
>> >> > I have memory and cpu issues when running heavy load testing with
>> >> > HTTP(S) sampler. The purpose of my test is to hammer the server as
>> >> > fast as Jmeter can, using a big list of hostnames and URIs, via a
>> >> > proxy server. The test plan I have setup is configured with
>> >> > 100-thread thread group that contains a HTTPS Request sampler (using
>> >> > HTTPClient4) and Summary Results generator.
>> >> >
>> >> > The machine I am using has 12GB memory and 16-core cpu. I monitor the
>> >> > top output throughout the test. I notice the RES size of the jmeter
>> >> > java process is growing really fast - it reaches 2GB heap size limit
>> >> > in less 5 minutes. The test could never go beyond 30-minute mark
>> >> > without OutOfMemory exception. I know there are plenty of free memory
>> >> > for me to expand the heap size; but my concern is whether there is a
>> >> > memory leak in the code. I worry the process would suck up all
>> >> > available memory anyway if I run a stress test for a day or longer.
>> >> >
>> >> > I have skimmed through the code of HTTPHC4Impl and found that there is
>> >> > cache "map" for HttpClient with regard to URL. I suspect this cache
>> >> > map could go pretty crazy in tests that deal with large number of (or
>> >> > dynamic) URLs. Just curiosity, I tried running a test with using a
>> >> > custom sampler as like HTTPHC4Impl but without the cache map; it seems
>> >> > help slowing down the pace of the heap size. However, the heap size
>> >> > (RES size in top output) never seem settle at one level over time.
>> >> >
>> >> > Does anyone have any experience in running similar test as I do? Does
>> >> > it look like a potential "memory leak" to you? Any comment or advice
>> >> > will be appreciated.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > Thanks,
>> >> > Jacky
>> >> >
>> >> > ---------------------------------------------------------------------
>> >> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> >> > For additional commands, e-mail: user-help@jmeter.apache.org
>> >> >
>> >> >
>> >>
>> >
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
I looked at your heap dump, it seems you are load testing lots of
differents WEBSITES (I mean HOSTs are all different).
This is clearly a very "weird" use case that we don't handle for now, and I
am not sure we will.
Regards
Philippe
On Thu, Oct 11, 2012 at 12:10 AM, unjc email <un...@gmail.com> wrote:
> I have uploaded the heap dump file online and it's available at
> http://dl.free.fr/qZBPvLRB8.
>
>
>
> Thanks,
> Jacky
>
> On Wed, Oct 10, 2012 at 3:20 PM, Philippe Mouawad
> <ph...@gmail.com> wrote:
> > Hello,
> > Which map are you talking about in HTTPHC4Impl ?
> >
> > If it's this one:
> >
> > - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
> >
> > Then it only exists once per thread and per combination of :
> >
> > - url.getProtocol()+"://"+url.getAuthority();
> > this.hasProxy = b;
> > this.proxyHost = proxyHost;
> > this.proxyPort = proxyPort;
> > this.proxyUser = proxyUser;
> > this.proxyPass = proxyPass;
> >
> > so unless you change these per url , it won't grow a lot.
> >
> > Check what Deepak is suggesting and if issue persists, then generate a
> Heap
> > Dump and put it somewhere where we can download it.
> >
> > Regards
> > Philippe
> >
> > On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com>
> wrote:
> >
> >> Hi
> >> its more likely that you have some listener turned on (like view results
> >> tree) - Even if a URL is 2000 characters you can calculate how many URLs
> >> before you use up say 100MB of memory. Do you really have that many URLs
> >> even if they are dynamic?
> >> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
> >>
> >> regards
> >> deepak
> >>
> >> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com>
> wrote:
> >>
> >> > Hi all,
> >> >
> >> > I have memory and cpu issues when running heavy load testing with
> >> > HTTP(S) sampler. The purpose of my test is to hammer the server as
> >> > fast as Jmeter can, using a big list of hostnames and URIs, via a
> >> > proxy server. The test plan I have setup is configured with
> >> > 100-thread thread group that contains a HTTPS Request sampler (using
> >> > HTTPClient4) and Summary Results generator.
> >> >
> >> > The machine I am using has 12GB memory and 16-core cpu. I monitor the
> >> > top output throughout the test. I notice the RES size of the jmeter
> >> > java process is growing really fast - it reaches 2GB heap size limit
> >> > in less 5 minutes. The test could never go beyond 30-minute mark
> >> > without OutOfMemory exception. I know there are plenty of free memory
> >> > for me to expand the heap size; but my concern is whether there is a
> >> > memory leak in the code. I worry the process would suck up all
> >> > available memory anyway if I run a stress test for a day or longer.
> >> >
> >> > I have skimmed through the code of HTTPHC4Impl and found that there is
> >> > cache "map" for HttpClient with regard to URL. I suspect this cache
> >> > map could go pretty crazy in tests that deal with large number of (or
> >> > dynamic) URLs. Just curiosity, I tried running a test with using a
> >> > custom sampler as like HTTPHC4Impl but without the cache map; it seems
> >> > help slowing down the pace of the heap size. However, the heap size
> >> > (RES size in top output) never seem settle at one level over time.
> >> >
> >> > Does anyone have any experience in running similar test as I do? Does
> >> > it look like a potential "memory leak" to you? Any comment or advice
> >> > will be appreciated.
> >> >
> >> >
> >> >
> >> >
> >> > Thanks,
> >> > Jacky
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >> > For additional commands, e-mail: user-help@jmeter.apache.org
> >> >
> >> >
> >>
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
--
Cordialement.
Philippe Mouawad.
Re: HTTP Sampler Memory Leak
Posted by unjc email <un...@gmail.com>.
I have uploaded the heap dump file online and it's available at
http://dl.free.fr/qZBPvLRB8.
Thanks,
Jacky
On Wed, Oct 10, 2012 at 3:20 PM, Philippe Mouawad
<ph...@gmail.com> wrote:
> Hello,
> Which map are you talking about in HTTPHC4Impl ?
>
> If it's this one:
>
> - Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
>
> Then it only exists once per thread and per combination of :
>
> - url.getProtocol()+"://"+url.getAuthority();
> this.hasProxy = b;
> this.proxyHost = proxyHost;
> this.proxyPort = proxyPort;
> this.proxyUser = proxyUser;
> this.proxyPass = proxyPass;
>
> so unless you change these per url , it won't grow a lot.
>
> Check what Deepak is suggesting and if issue persists, then generate a Heap
> Dump and put it somewhere where we can download it.
>
> Regards
> Philippe
>
> On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com> wrote:
>
>> Hi
>> its more likely that you have some listener turned on (like view results
>> tree) - Even if a URL is 2000 characters you can calculate how many URLs
>> before you use up say 100MB of memory. Do you really have that many URLs
>> even if they are dynamic?
>> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>>
>> regards
>> deepak
>>
>> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com> wrote:
>>
>> > Hi all,
>> >
>> > I have memory and cpu issues when running heavy load testing with
>> > HTTP(S) sampler. The purpose of my test is to hammer the server as
>> > fast as Jmeter can, using a big list of hostnames and URIs, via a
>> > proxy server. The test plan I have setup is configured with
>> > 100-thread thread group that contains a HTTPS Request sampler (using
>> > HTTPClient4) and Summary Results generator.
>> >
>> > The machine I am using has 12GB memory and 16-core cpu. I monitor the
>> > top output throughout the test. I notice the RES size of the jmeter
>> > java process is growing really fast - it reaches 2GB heap size limit
>> > in less 5 minutes. The test could never go beyond 30-minute mark
>> > without OutOfMemory exception. I know there are plenty of free memory
>> > for me to expand the heap size; but my concern is whether there is a
>> > memory leak in the code. I worry the process would suck up all
>> > available memory anyway if I run a stress test for a day or longer.
>> >
>> > I have skimmed through the code of HTTPHC4Impl and found that there is
>> > cache "map" for HttpClient with regard to URL. I suspect this cache
>> > map could go pretty crazy in tests that deal with large number of (or
>> > dynamic) URLs. Just curiosity, I tried running a test with using a
>> > custom sampler as like HTTPHC4Impl but without the cache map; it seems
>> > help slowing down the pace of the heap size. However, the heap size
>> > (RES size in top output) never seem settle at one level over time.
>> >
>> > Does anyone have any experience in running similar test as I do? Does
>> > it look like a potential "memory leak" to you? Any comment or advice
>> > will be appreciated.
>> >
>> >
>> >
>> >
>> > Thanks,
>> > Jacky
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> > For additional commands, e-mail: user-help@jmeter.apache.org
>> >
>> >
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
Which map are you talking about in HTTPHC4Impl ?
If it's this one:
- Map<HttpClientKey, HttpClient> map = HTTPCLIENTS.get();
Then it only exists once per thread and per combination of :
- url.getProtocol()+"://"+url.getAuthority();
this.hasProxy = b;
this.proxyHost = proxyHost;
this.proxyPort = proxyPort;
this.proxyUser = proxyUser;
this.proxyPass = proxyPass;
so unless you change these per url , it won't grow a lot.
Check what Deepak is suggesting and if issue persists, then generate a Heap
Dump and put it somewhere where we can download it.
Regards
Philippe
On Wed, Oct 10, 2012 at 8:53 PM, Deepak Shetty <sh...@gmail.com> wrote:
> Hi
> its more likely that you have some listener turned on (like view results
> tree) - Even if a URL is 2000 characters you can calculate how many URLs
> before you use up say 100MB of memory. Do you really have that many URLs
> even if they are dynamic?
> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>
> regards
> deepak
>
> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com> wrote:
>
> > Hi all,
> >
> > I have memory and cpu issues when running heavy load testing with
> > HTTP(S) sampler. The purpose of my test is to hammer the server as
> > fast as Jmeter can, using a big list of hostnames and URIs, via a
> > proxy server. The test plan I have setup is configured with
> > 100-thread thread group that contains a HTTPS Request sampler (using
> > HTTPClient4) and Summary Results generator.
> >
> > The machine I am using has 12GB memory and 16-core cpu. I monitor the
> > top output throughout the test. I notice the RES size of the jmeter
> > java process is growing really fast - it reaches 2GB heap size limit
> > in less 5 minutes. The test could never go beyond 30-minute mark
> > without OutOfMemory exception. I know there are plenty of free memory
> > for me to expand the heap size; but my concern is whether there is a
> > memory leak in the code. I worry the process would suck up all
> > available memory anyway if I run a stress test for a day or longer.
> >
> > I have skimmed through the code of HTTPHC4Impl and found that there is
> > cache "map" for HttpClient with regard to URL. I suspect this cache
> > map could go pretty crazy in tests that deal with large number of (or
> > dynamic) URLs. Just curiosity, I tried running a test with using a
> > custom sampler as like HTTPHC4Impl but without the cache map; it seems
> > help slowing down the pace of the heap size. However, the heap size
> > (RES size in top output) never seem settle at one level over time.
> >
> > Does anyone have any experience in running similar test as I do? Does
> > it look like a potential "memory leak" to you? Any comment or advice
> > will be appreciated.
> >
> >
> >
> >
> > Thanks,
> > Jacky
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
> >
>
--
Cordialement.
Philippe Mouawad.
Re: HTTP Sampler Memory Leak
Posted by sebb <se...@gmail.com>.
On 10 October 2012 19:53, Deepak Shetty <sh...@gmail.com> wrote:
> Hi
> its more likely that you have some listener turned on (like view results
> tree) - Even if a URL is 2000 characters you can calculate how many URLs
> before you use up say 100MB of memory. Do you really have that many URLs
> even if they are dynamic?
> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
Also, the cache key is
protocol://[user:pass@]host:[port]
plus proxy details (if used).
Unless you are targetting lots of different hosts, this is unlikely to
be the main problem.
> regards
> deepak
>
> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com> wrote:
>
>> Hi all,
>>
>> I have memory and cpu issues when running heavy load testing with
>> HTTP(S) sampler. The purpose of my test is to hammer the server as
>> fast as Jmeter can, using a big list of hostnames and URIs, via a
>> proxy server. The test plan I have setup is configured with
>> 100-thread thread group that contains a HTTPS Request sampler (using
>> HTTPClient4) and Summary Results generator.
>>
>> The machine I am using has 12GB memory and 16-core cpu. I monitor the
>> top output throughout the test. I notice the RES size of the jmeter
>> java process is growing really fast - it reaches 2GB heap size limit
>> in less 5 minutes. The test could never go beyond 30-minute mark
>> without OutOfMemory exception. I know there are plenty of free memory
>> for me to expand the heap size; but my concern is whether there is a
>> memory leak in the code. I worry the process would suck up all
>> available memory anyway if I run a stress test for a day or longer.
>>
>> I have skimmed through the code of HTTPHC4Impl and found that there is
>> cache "map" for HttpClient with regard to URL. I suspect this cache
>> map could go pretty crazy in tests that deal with large number of (or
>> dynamic) URLs. Just curiosity, I tried running a test with using a
>> custom sampler as like HTTPHC4Impl but without the cache map; it seems
>> help slowing down the pace of the heap size. However, the heap size
>> (RES size in top output) never seem settle at one level over time.
>>
>> Does anyone have any experience in running similar test as I do? Does
>> it look like a potential "memory leak" to you? Any comment or advice
>> will be appreciated.
>>
>>
>>
>>
>> Thanks,
>> Jacky
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by Deepak Shetty <sh...@gmail.com>.
Hi
if the name you are using for the URL is different (i,.e result gets
recorded as its own entry) then Generate Summary Report will use memory
proportional to number of URLs.
Run a non -gui mode with all listeners disabled and see how it goes
regards
deepak
On Wed, Oct 10, 2012 at 12:08 PM, unjc email <un...@gmail.com> wrote:
> HI Deepak,
>
> Thanks for your reply. I don't have any view results tree; "Generate
> Summary Result" is the only listener that prints result summary to
> screen.
> I don't know the jmeter source code inside out, but if you are talking
> about the cache map, I believe it holds more than just URLs as it's
> the key, the map saves the httpClient object as well. As I mentioned
> before, I found that the heap size still keeps on growing even though
> I tested with a custom sampler that has the cache map removed.
>
> FYI, I have a poisoned DNS before the server; therefore, literally,
> there could be millions of URLs.
>
>
> Thanks,
> Jacky
>
> On Wed, Oct 10, 2012 at 2:53 PM, Deepak Shetty <sh...@gmail.com> wrote:
> > Hi
> > its more likely that you have some listener turned on (like view results
> > tree) - Even if a URL is 2000 characters you can calculate how many URLs
> > before you use up say 100MB of memory. Do you really have that many URLs
> > even if they are dynamic?
> > http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
> >
> > regards
> > deepak
> >
> > On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com>
> wrote:
> >
> >> Hi all,
> >>
> >> I have memory and cpu issues when running heavy load testing with
> >> HTTP(S) sampler. The purpose of my test is to hammer the server as
> >> fast as Jmeter can, using a big list of hostnames and URIs, via a
> >> proxy server. The test plan I have setup is configured with
> >> 100-thread thread group that contains a HTTPS Request sampler (using
> >> HTTPClient4) and Summary Results generator.
> >>
> >> The machine I am using has 12GB memory and 16-core cpu. I monitor the
> >> top output throughout the test. I notice the RES size of the jmeter
> >> java process is growing really fast - it reaches 2GB heap size limit
> >> in less 5 minutes. The test could never go beyond 30-minute mark
> >> without OutOfMemory exception. I know there are plenty of free memory
> >> for me to expand the heap size; but my concern is whether there is a
> >> memory leak in the code. I worry the process would suck up all
> >> available memory anyway if I run a stress test for a day or longer.
> >>
> >> I have skimmed through the code of HTTPHC4Impl and found that there is
> >> cache "map" for HttpClient with regard to URL. I suspect this cache
> >> map could go pretty crazy in tests that deal with large number of (or
> >> dynamic) URLs. Just curiosity, I tried running a test with using a
> >> custom sampler as like HTTPHC4Impl but without the cache map; it seems
> >> help slowing down the pace of the heap size. However, the heap size
> >> (RES size in top output) never seem settle at one level over time.
> >>
> >> Does anyone have any experience in running similar test as I do? Does
> >> it look like a potential "memory leak" to you? Any comment or advice
> >> will be appreciated.
> >>
> >>
> >>
> >>
> >> Thanks,
> >> Jacky
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >> For additional commands, e-mail: user-help@jmeter.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
Re: HTTP Sampler Memory Leak
Posted by unjc email <un...@gmail.com>.
This is a snapshot of the top output of the jmeter process.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31902 root 20 0 11.4g 2.1g 13m S 189.4 17.9 21:13.96 java
-server -XX:+HeapDumpOnOutOfMemoryError -XX:+AggressiveOpts
-XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:MaxTenuringThreshold=2
-Dsun.rmi.dgc.client.gcInterval
On Wed, Oct 10, 2012 at 3:08 PM, unjc email <un...@gmail.com> wrote:
> HI Deepak,
>
> Thanks for your reply. I don't have any view results tree; "Generate
> Summary Result" is the only listener that prints result summary to
> screen.
> I don't know the jmeter source code inside out, but if you are talking
> about the cache map, I believe it holds more than just URLs as it's
> the key, the map saves the httpClient object as well. As I mentioned
> before, I found that the heap size still keeps on growing even though
> I tested with a custom sampler that has the cache map removed.
>
> FYI, I have a poisoned DNS before the server; therefore, literally,
> there could be millions of URLs.
>
>
> Thanks,
> Jacky
>
> On Wed, Oct 10, 2012 at 2:53 PM, Deepak Shetty <sh...@gmail.com> wrote:
>> Hi
>> its more likely that you have some listener turned on (like view results
>> tree) - Even if a URL is 2000 characters you can calculate how many URLs
>> before you use up say 100MB of memory. Do you really have that many URLs
>> even if they are dynamic?
>> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>>
>> regards
>> deepak
>>
>> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I have memory and cpu issues when running heavy load testing with
>>> HTTP(S) sampler. The purpose of my test is to hammer the server as
>>> fast as Jmeter can, using a big list of hostnames and URIs, via a
>>> proxy server. The test plan I have setup is configured with
>>> 100-thread thread group that contains a HTTPS Request sampler (using
>>> HTTPClient4) and Summary Results generator.
>>>
>>> The machine I am using has 12GB memory and 16-core cpu. I monitor the
>>> top output throughout the test. I notice the RES size of the jmeter
>>> java process is growing really fast - it reaches 2GB heap size limit
>>> in less 5 minutes. The test could never go beyond 30-minute mark
>>> without OutOfMemory exception. I know there are plenty of free memory
>>> for me to expand the heap size; but my concern is whether there is a
>>> memory leak in the code. I worry the process would suck up all
>>> available memory anyway if I run a stress test for a day or longer.
>>>
>>> I have skimmed through the code of HTTPHC4Impl and found that there is
>>> cache "map" for HttpClient with regard to URL. I suspect this cache
>>> map could go pretty crazy in tests that deal with large number of (or
>>> dynamic) URLs. Just curiosity, I tried running a test with using a
>>> custom sampler as like HTTPHC4Impl but without the cache map; it seems
>>> help slowing down the pace of the heap size. However, the heap size
>>> (RES size in top output) never seem settle at one level over time.
>>>
>>> Does anyone have any experience in running similar test as I do? Does
>>> it look like a potential "memory leak" to you? Any comment or advice
>>> will be appreciated.
>>>
>>>
>>>
>>>
>>> Thanks,
>>> Jacky
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by unjc email <un...@gmail.com>.
HI Deepak,
Thanks for your reply. I don't have any view results tree; "Generate
Summary Result" is the only listener that prints result summary to
screen.
I don't know the jmeter source code inside out, but if you are talking
about the cache map, I believe it holds more than just URLs as it's
the key, the map saves the httpClient object as well. As I mentioned
before, I found that the heap size still keeps on growing even though
I tested with a custom sampler that has the cache map removed.
FYI, I have a poisoned DNS before the server; therefore, literally,
there could be millions of URLs.
Thanks,
Jacky
On Wed, Oct 10, 2012 at 2:53 PM, Deepak Shetty <sh...@gmail.com> wrote:
> Hi
> its more likely that you have some listener turned on (like view results
> tree) - Even if a URL is 2000 characters you can calculate how many URLs
> before you use up say 100MB of memory. Do you really have that many URLs
> even if they are dynamic?
> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
>
> regards
> deepak
>
> On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com> wrote:
>
>> Hi all,
>>
>> I have memory and cpu issues when running heavy load testing with
>> HTTP(S) sampler. The purpose of my test is to hammer the server as
>> fast as Jmeter can, using a big list of hostnames and URIs, via a
>> proxy server. The test plan I have setup is configured with
>> 100-thread thread group that contains a HTTPS Request sampler (using
>> HTTPClient4) and Summary Results generator.
>>
>> The machine I am using has 12GB memory and 16-core cpu. I monitor the
>> top output throughout the test. I notice the RES size of the jmeter
>> java process is growing really fast - it reaches 2GB heap size limit
>> in less 5 minutes. The test could never go beyond 30-minute mark
>> without OutOfMemory exception. I know there are plenty of free memory
>> for me to expand the heap size; but my concern is whether there is a
>> memory leak in the code. I worry the process would suck up all
>> available memory anyway if I run a stress test for a day or longer.
>>
>> I have skimmed through the code of HTTPHC4Impl and found that there is
>> cache "map" for HttpClient with regard to URL. I suspect this cache
>> map could go pretty crazy in tests that deal with large number of (or
>> dynamic) URLs. Just curiosity, I tried running a test with using a
>> custom sampler as like HTTPHC4Impl but without the cache map; it seems
>> help slowing down the pace of the heap size. However, the heap size
>> (RES size in top output) never seem settle at one level over time.
>>
>> Does anyone have any experience in running similar test as I do? Does
>> it look like a potential "memory leak" to you? Any comment or advice
>> will be appreciated.
>>
>>
>>
>>
>> Thanks,
>> Jacky
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: HTTP Sampler Memory Leak
Posted by Deepak Shetty <sh...@gmail.com>.
Hi
its more likely that you have some listener turned on (like view results
tree) - Even if a URL is 2000 characters you can calculate how many URLs
before you use up say 100MB of memory. Do you really have that many URLs
even if they are dynamic?
http://jmeter.apache.org/usermanual/best-practices.html#lean_mean
regards
deepak
On Wed, Oct 10, 2012 at 9:21 AM, unjc email <un...@gmail.com> wrote:
> Hi all,
>
> I have memory and cpu issues when running heavy load testing with
> HTTP(S) sampler. The purpose of my test is to hammer the server as
> fast as Jmeter can, using a big list of hostnames and URIs, via a
> proxy server. The test plan I have setup is configured with
> 100-thread thread group that contains a HTTPS Request sampler (using
> HTTPClient4) and Summary Results generator.
>
> The machine I am using has 12GB memory and 16-core cpu. I monitor the
> top output throughout the test. I notice the RES size of the jmeter
> java process is growing really fast - it reaches 2GB heap size limit
> in less 5 minutes. The test could never go beyond 30-minute mark
> without OutOfMemory exception. I know there are plenty of free memory
> for me to expand the heap size; but my concern is whether there is a
> memory leak in the code. I worry the process would suck up all
> available memory anyway if I run a stress test for a day or longer.
>
> I have skimmed through the code of HTTPHC4Impl and found that there is
> cache "map" for HttpClient with regard to URL. I suspect this cache
> map could go pretty crazy in tests that deal with large number of (or
> dynamic) URLs. Just curiosity, I tried running a test with using a
> custom sampler as like HTTPHC4Impl but without the cache map; it seems
> help slowing down the pace of the heap size. However, the heap size
> (RES size in top output) never seem settle at one level over time.
>
> Does anyone have any experience in running similar test as I do? Does
> it look like a potential "memory leak" to you? Any comment or advice
> will be appreciated.
>
>
>
>
> Thanks,
> Jacky
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>