You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tapestry.apache.org by Bryan Dotzour <BD...@widen.com> on 2007/08/24 17:20:02 UTC
Memory consumption in T4.1.2 - Hard data
I and another colleague of mine have been investigating what seems to be
a "memory leak" in our Tapestry application for about a month since we
upgraded to T4.1.2. I won't bore you with the saga of the last month,
but I would like to present the data I've gathered and look to the list
for a proposed solution. I was reading a recent thread in which Jesse
said (08/09/2007):
"There is a map that grows as large as the system using it internally to
javassist of various cached reflection info - but it doesn't leak in any
way."
This is precisely what I've found in profiling our application and it
*appears* to be this map that is causing our applications to eventually
run out of memory.
The YourKit profiler shows me that, as time goes on, there is an
instance of HiveMindClassPool that grows and grows as class instances
are created. This class extends from javassist.ClassPool and is the map
that Jesse is talking about in his quote above. And he's right, I
wouldn't say that the class pool "leaks" either because it looks like
it's designed to retain that memory until the class pool itself is no
longer needed.
Take this quote from the javassist.ClassPool javadocs:
"Memory consumption memo:
ClassPool objects hold all the CtClasses that have been created so that
the consistency among modified classes can be guaranteed. Thus if a
large number of CtClasses are processed, the ClassPool will consume a
huge amount of memory. To avoid this, a ClassPool object should be
recreated, for example, every hundred classes processed. Note that
getDefault() is a singleton factory. Otherwise, detach() in CtClass
should be used to avoid huge memory consumption. "
This huge memory consumption by the ClassPool is what I was seeing. In
particular it is the ClassPool that is held onto by OgnlRuntime.
Inspecting this object in the profiler showed that it has a map
containing about 45,000 classes. All of the keys into this map were
things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
values are instances of javassist.CtNewClass. Each entry in this map
looks like it retains about 1,900 bytes, for a grand total of about 90
MB of memory used.
These numbers came from my staging deployment where I had the profiler
attached, using some reflection tricks I was able to look at a
production site and found that it had about 240,000 items in that class
pool.. approximately 450 MB of memory.
So I guess the questions in my mind are: Why are there so many classes
in the pool? Why does the number only ever go up? Do those classes
really need to stay in the pool forever?
Re: Memory consumption in T4.1.2 - Hard data
Posted by Kalle Korhonen <ka...@gmail.com>.
Now that's an OGNL compiler error. Have you tried with OGNL
2.7.1-SNAPSHOT(just use the latest). OGNL
2.7 that comes with Tap 4.1.2 by default is known to have bugs with the
compiled expressions. Please try with the latest OGNL; it might just do the
trick.
Kalle
On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>
>
> Andreas, give me a break! I am not trying to trash Tapestry if that's
> what you are thinking. I am not the only one experiencing these problems.
> Have a look at some of my postings (and others posts) and you will see
> what I am talking about. I have been having these problems since
> upgrading from 4.1.1 I never experienced these problems before with
> uptime of months at a time.
>
> Look at this log, id is an integer with the value 728:
>
> 21 Aug 2007 13:01:49,353 - INFO $Portfolio_89 - Setting the portfolio with
> the id: 728
> 21 Aug 2007 13:01:50,479 - ERROR $Error_118 - Unable to parse OGNL
> expression 'id': Unable to parse OGNL expression 'id': Unable to parse OGNL
> expression 'id': ............
> Exception [classpath:/org/apache/tapestry/form/TextArea.jwc, line 49,
> column 67]
> at org.apache.tapestry.binding.ExpressionBinding.resolveExpression
> (ExpressionBinding.java:145)
> at org.apache.tapestry.binding.ExpressionBinding.getObject(
> ExpressionBinding.java:125)
>
> And then take a look at this:
>
> 21 Aug 2007 07:56:50,613 - ERROR
> org.apache.tapestry.services.impl.HiveMindExpressionCompiler - Error
> generating OGNL getter for expression exceptions with root
> $Exception_119@3c1[framework:Exception] and body:
> { return (($Exception_119)$2).getExceptions();}
> org.apache.hivemind.ApplicationRuntimeException: Unable to add method
> java.lang.Object get(ognl.OgnlContext, java.lang.Object) to class
> $ASTProperty_1146a471a52: [source error] no such class: $Exception_119
>
> This happening after 3-4 days of normal operation? Suddenly exceptions
> occur on random objects because OGNL can no longer compile them and this
> is because Tapestry can no longer find instances of these classes, it
> relates to JavaAssist dynamic class loading. Also, currently the server
> is idle with no connections and JConsole shows 18000 classes loaded,
> yesterday there were 7000 classes, this number never goes down only up,
> so you tell me what's going on?
>
> If Jessie can't explain it, then I have to go back a version we cant
> afford for clients to see broken pages, my boss doesn't care about the
> problem, he just wants a solution and its my job and reputation is at
> stake.
>
>
> Andreas Andreou wrote:
> > Peter,
> > though not officially final (I believe),
> > http://news247.gr (tap-4.1.2) has been getting 400000 req/day
> > for an uptime of 26 days with memory set to no more than 128MB.
> >
> >
> > On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> >
> >> Hi Jessie
> >>
> >> Any progress on this? .... sorry to bug you, but I have to take a
> >> decision soon, I have two production machines that will need to upgrade
> >> or downgrade Tapestry.
> >>
> >> Best wishes,
> >> Peter
> >>
> >> Jon Oakes wrote:
> >>
> >>> Hi Bryan,
> >>>
> >>> I am a relative newbie but I was wondering if you are running with
> >>> caching enabled or disabled? It might make sense to keep things
> >>> around when caching is disabled whereas I think it would clearly be a
> >>> bug to keep things around with it disabled.
> >>>
> >>> Jon Oakes
> >>>
> >>>
> >>> Jesse Kuhnert wrote:
> >>>
> >>>> Hmmm...well, I don't think I like the sound of any of that. I'm
> just
> >>>> going to pretend this problem doesn't exist. (just kidding)
> >>>>
> >>>> I had thought I was doing something special with the ognl
> compilations
> >>>> that would cause its generated classes to not hang around afterwards
> >>>> in any pools.
> >>>>
> >>>> I'll take a look at things this weekend and am sure a fix will appear
> >>>> in between now and Monday - if there is a reasonable fix to be found.
> >>>> (in 4.1.3 snapshot form)
> >>>>
> >>>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <mailto:
> >>>>
> >> BDotzour@widen.com> wrote:
> >>
> >>>>> I and another colleague of mine have been investigating what seems
> to
> >>>>>
> >> be
> >>
> >>>>> a "memory leak" in our Tapestry application for about a month since
> we
> >>>>> upgraded to T4.1.2. I won't bore you with the saga of the last
> month,
> >>>>> but I would like to present the data I've gathered and look to the
> >>>>>
> >> list
> >>
> >>>>> for a proposed solution. I was reading a recent thread in which
> Jesse
> >>>>> said (08/09/2007):
> >>>>>
> >>>>>
> >>>>>
> >>>>> "There is a map that grows as large as the system using it
> internally
> >>>>>
> >> to
> >>
> >>>>> javassist of various cached reflection info - but it doesn't leak in
> >>>>>
> >> any
> >>
> >>>>> way."
> >>>>>
> >>>>> This is precisely what I've found in profiling our application and
> it
> >>>>> *appears* to be this map that is causing our applications to
> >>>>>
> >> eventually
> >>
> >>>>> run out of memory.
> >>>>>
> >>>>> The YourKit profiler shows me that, as time goes on, there is an
> >>>>> instance of HiveMindClassPool that grows and grows as class
> instances
> >>>>> are created. This class extends from javassist.ClassPool and is the
> >>>>>
> >> map
> >>
> >>>>> that Jesse is talking about in his quote above. And he's right, I
> >>>>> wouldn't say that the class pool "leaks" either because it looks
> like
> >>>>> it's designed to retain that memory until the class pool itself is
> no
> >>>>> longer needed.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Take this quote from the javassist.ClassPool javadocs:
> >>>>>
> >>>>> "Memory consumption memo:
> >>>>> ClassPool objects hold all the CtClasses that have been created so
> >>>>>
> >> that
> >>
> >>>>> the consistency among modified classes can be guaranteed. Thus if a
> >>>>> large number of CtClasses are processed, the ClassPool will consume
> a
> >>>>> huge amount of memory. To avoid this, a ClassPool object should be
> >>>>> recreated, for example, every hundred classes processed. Note that
> >>>>> getDefault() is a singleton factory. Otherwise, detach() in CtClass
> >>>>> should be used to avoid huge memory consumption. "
> >>>>>
> >>>>> This huge memory consumption by the ClassPool is what I was seeing.
> In
> >>>>> particular it is the ClassPool that is held onto by OgnlRuntime.
> >>>>> Inspecting this object in the profiler showed that it has a map
> >>>>> containing about 45,000 classes. All of the keys into this map were
> >>>>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
> >>>>> values are instances of javassist.CtNewClass. Each entry in this
> map
> >>>>> looks like it retains about 1,900 bytes, for a grand total of about
> 90
> >>>>> MB of memory used.
> >>>>>
> >>>>> These numbers came from my staging deployment where I had the
> profiler
> >>>>> attached, using some reflection tricks I was able to look at a
> >>>>> production site and found that it had about 240,000 items in that
> >>>>>
> >> class
> >>
> >>>>> pool.. approximately 450 MB of memory.
> >>>>>
> >>>>> So I guess the questions in my mind are: Why are there so many
> >>>>>
> >> classes
> >>
> >>>>> in the pool? Why does the number only ever go up? Do those classes
> >>>>> really need to stay in the pool forever?
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
> >>> additional commands, e-mail: users-help@tapestry.apache.org
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> >> For additional commands, e-mail: users-help@tapestry.apache.org
> >>
> >>
> >>
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
Re: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
No Marcus
Marcus.Schulte@bmw.ch wrote:
> So changing the page pool parameters didn't help then?
>
>
>> -----Original Message-----
>> From: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
>> Sent: Tuesday, August 28, 2007 10:34 AM
>> To: Tapestry users
>> Subject: Re: Memory consumption in T4.1.2 - Hard data
>>
>>
>> Andreas, give me a break! I am not trying to trash Tapestry
>> if that's what you are thinking. I am not the only one
>> experiencing these problems.
>> Have a look at some of my postings (and others posts) and you
>> will see what I am talking about. I have been having these
>> problems since upgrading from 4.1.1 I never experienced these
>> problems before with uptime of months at a time.
>>
>> Look at this log, id is an integer with the value 728:
>>
>> 21 Aug 2007 13:01:49,353 - INFO $Portfolio_89 - Setting the
>> portfolio with the id: 728
>> 21 Aug 2007 13:01:50,479 - ERROR $Error_118 - Unable to parse
>> OGNL expression 'id': Unable to parse OGNL expression 'id':
>> Unable to parse OGNL expression 'id': ............
>> Exception [classpath:/org/apache/tapestry/form/TextArea.jwc,
>> line 49, column 67]
>> at
>> org.apache.tapestry.binding.ExpressionBinding.resolveExpressio
>> n(ExpressionBinding.java:145)
>> at
>> org.apache.tapestry.binding.ExpressionBinding.getObject(Expres
>> sionBinding.java:125)
>>
>> And then take a look at this:
>>
>> 21 Aug 2007 07:56:50,613 - ERROR
>> org.apache.tapestry.services.impl.HiveMindExpressionCompiler
>> - Error generating OGNL getter for expression exceptions with
>> root $Exception_119@3c1[framework:Exception] and body:
>> { return (($Exception_119)$2).getExceptions();}
>> org.apache.hivemind.ApplicationRuntimeException: Unable to
>> add method java.lang.Object get(ognl.OgnlContext,
>> java.lang.Object) to class $ASTProperty_1146a471a52: [source
>> error] no such class: $Exception_119
>>
>> This happening after 3-4 days of normal operation? Suddenly
>> exceptions occur on random objects because OGNL can no longer
>> compile them and this is because Tapestry can no longer find
>> instances of these classes, it relates to JavaAssist dynamic
>> class loading. Also, currently the server is idle with no
>> connections and JConsole shows 18000 classes loaded,
>> yesterday there were 7000 classes, this number never goes
>> down only up, so you tell me what's going on?
>>
>> If Jessie can't explain it, then I have to go back a version
>> we cant afford for clients to see broken pages, my boss
>> doesn't care about the problem, he just wants a solution and
>> its my job and reputation is at stake.
>>
>>
>> Andreas Andreou wrote:
>>
>>> Peter,
>>> though not officially final (I believe), http://news247.gr
>>>
>> (tap-4.1.2)
>>
>>> has been getting 400000 req/day for an uptime of 26 days
>>>
>> with memory
>>
>>> set to no more than 128MB.
>>>
>>>
>>> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>>>
>>>
>>>> Hi Jessie
>>>>
>>>> Any progress on this? .... sorry to bug you, but I have to take a
>>>> decision soon, I have two production machines that will need to
>>>> upgrade or downgrade Tapestry.
>>>>
>>>> Best wishes,
>>>> Peter
>>>>
>>>> Jon Oakes wrote:
>>>>
>>>>
>>>>> Hi Bryan,
>>>>>
>>>>> I am a relative newbie but I was wondering if you are
>>>>>
>> running with
>>
>>>>> caching enabled or disabled? It might make sense to keep things
>>>>> around when caching is disabled whereas I think it would
>>>>>
>> clearly be
>>
>>>>> a bug to keep things around with it disabled.
>>>>>
>>>>> Jon Oakes
>>>>>
>>>>>
>>>>> Jesse Kuhnert wrote:
>>>>>
>>>>>
>>>>>> Hmmm...well, I don't think I like the sound of any of
>>>>>>
>> that. I'm just
>>
>>>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>>>
>>>>>> I had thought I was doing something special with the ognl
>>>>>> compilations that would cause its generated classes to not hang
>>>>>> around afterwards in any pools.
>>>>>>
>>>>>> I'll take a look at things this weekend and am sure a fix will
>>>>>> appear in between now and Monday - if there is a
>>>>>>
>> reasonable fix to be found.
>>
>>>>>> (in 4.1.3 snapshot form)
>>>>>>
>>>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <mailto:
>>>>>>
>>>>>>
>>>> BDotzour@widen.com> wrote:
>>>>
>>>>
>>>>>>> I and another colleague of mine have been investigating
>>>>>>>
>> what seems
>>
>>>>>>> to
>>>>>>>
>>>>>>>
>>>> be
>>>>
>>>>
>>>>>>> a "memory leak" in our Tapestry application for about a month
>>>>>>> since we upgraded to T4.1.2. I won't bore you with the saga of
>>>>>>> the last month, but I would like to present the data
>>>>>>>
>> I've gathered
>>
>>>>>>> and look to the
>>>>>>>
>>>>>>>
>>>> list
>>>>
>>>>
>>>>>>> for a proposed solution. I was reading a recent thread
>>>>>>>
>> in which
>>
>>>>>>> Jesse said (08/09/2007):
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> "There is a map that grows as large as the system using it
>>>>>>> internally
>>>>>>>
>>>>>>>
>>>> to
>>>>
>>>>
>>>>>>> javassist of various cached reflection info - but it
>>>>>>>
>> doesn't leak
>>
>>>>>>> in
>>>>>>>
>>>>>>>
>>>> any
>>>>
>>>>
>>>>>>> way."
>>>>>>>
>>>>>>> This is precisely what I've found in profiling our
>>>>>>>
>> application and
>>
>>>>>>> it
>>>>>>> *appears* to be this map that is causing our applications to
>>>>>>>
>>>>>>>
>>>> eventually
>>>>
>>>>
>>>>>>> run out of memory.
>>>>>>>
>>>>>>> The YourKit profiler shows me that, as time goes on,
>>>>>>>
>> there is an
>>
>>>>>>> instance of HiveMindClassPool that grows and grows as class
>>>>>>> instances are created. This class extends from
>>>>>>> javassist.ClassPool and is the
>>>>>>>
>>>>>>>
>>>> map
>>>>
>>>>
>>>>>>> that Jesse is talking about in his quote above. And
>>>>>>>
>> he's right, I
>>
>>>>>>> wouldn't say that the class pool "leaks" either because
>>>>>>>
>> it looks
>>
>>>>>>> like it's designed to retain that memory until the class pool
>>>>>>> itself is no longer needed.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>>>
>>>>>>> "Memory consumption memo:
>>>>>>> ClassPool objects hold all the CtClasses that have been
>>>>>>>
>> created so
>>
>>>>>>>
>>>>>>>
>>>> that
>>>>
>>>>
>>>>>>> the consistency among modified classes can be
>>>>>>>
>> guaranteed. Thus if
>>
>>>>>>> a large number of CtClasses are processed, the ClassPool will
>>>>>>> consume a huge amount of memory. To avoid this, a
>>>>>>>
>> ClassPool object
>>
>>>>>>> should be recreated, for example, every hundred classes
>>>>>>>
>> processed.
>>
>>>>>>> Note that
>>>>>>> getDefault() is a singleton factory. Otherwise, detach() in
>>>>>>> CtClass should be used to avoid huge memory consumption. "
>>>>>>>
>>>>>>> This huge memory consumption by the ClassPool is what I was
>>>>>>> seeing. In particular it is the ClassPool that is held
>>>>>>>
>> onto by OgnlRuntime.
>>
>>>>>>> Inspecting this object in the profiler showed that it has a map
>>>>>>> containing about 45,000 classes. All of the keys into this map
>>>>>>> were things like: "ASTTest_11494aca9af" and
>>>>>>>
>> "ASTAnd_11494ace4fb"
>>
>>>>>>> and the values are instances of javassist.CtNewClass.
>>>>>>>
>> Each entry
>>
>>>>>>> in this map looks like it retains about 1,900 bytes,
>>>>>>>
>> for a grand
>>
>>>>>>> total of about 90 MB of memory used.
>>>>>>>
>>>>>>> These numbers came from my staging deployment where I had the
>>>>>>> profiler attached, using some reflection tricks I was
>>>>>>>
>> able to look
>>
>>>>>>> at a production site and found that it had about
>>>>>>>
>> 240,000 items in
>>
>>>>>>> that
>>>>>>>
>>>>>>>
>>>> class
>>>>
>>>>
>>>>>>> pool.. approximately 450 MB of memory.
>>>>>>>
>>>>>>> So I guess the questions in my mind are: Why are there so many
>>>>>>>
>>>>>>>
>>>> classes
>>>>
>>>>
>>>>>>> in the pool? Why does the number only ever go up? Do those
>>>>>>> classes really need to stay in the pool forever?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>> --------------------------------------------------------------------
>>
>>>>> - To unsubscribe, e-mail:
>>>>>
>> users-unsubscribe@tapestry.apache.org For
>>
>>>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>
>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
RE: Memory consumption in T4.1.2 - Hard data
Posted by Ma...@bmw.ch.
So changing the page pool parameters didn't help then?
> -----Original Message-----
> From: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
> Sent: Tuesday, August 28, 2007 10:34 AM
> To: Tapestry users
> Subject: Re: Memory consumption in T4.1.2 - Hard data
>
>
> Andreas, give me a break! I am not trying to trash Tapestry
> if that's what you are thinking. I am not the only one
> experiencing these problems.
> Have a look at some of my postings (and others posts) and you
> will see what I am talking about. I have been having these
> problems since upgrading from 4.1.1 I never experienced these
> problems before with uptime of months at a time.
>
> Look at this log, id is an integer with the value 728:
>
> 21 Aug 2007 13:01:49,353 - INFO $Portfolio_89 - Setting the
> portfolio with the id: 728
> 21 Aug 2007 13:01:50,479 - ERROR $Error_118 - Unable to parse
> OGNL expression 'id': Unable to parse OGNL expression 'id':
> Unable to parse OGNL expression 'id': ............
> Exception [classpath:/org/apache/tapestry/form/TextArea.jwc,
> line 49, column 67]
> at
> org.apache.tapestry.binding.ExpressionBinding.resolveExpressio
> n(ExpressionBinding.java:145)
> at
> org.apache.tapestry.binding.ExpressionBinding.getObject(Expres
> sionBinding.java:125)
>
> And then take a look at this:
>
> 21 Aug 2007 07:56:50,613 - ERROR
> org.apache.tapestry.services.impl.HiveMindExpressionCompiler
> - Error generating OGNL getter for expression exceptions with
> root $Exception_119@3c1[framework:Exception] and body:
> { return (($Exception_119)$2).getExceptions();}
> org.apache.hivemind.ApplicationRuntimeException: Unable to
> add method java.lang.Object get(ognl.OgnlContext,
> java.lang.Object) to class $ASTProperty_1146a471a52: [source
> error] no such class: $Exception_119
>
> This happening after 3-4 days of normal operation? Suddenly
> exceptions occur on random objects because OGNL can no longer
> compile them and this is because Tapestry can no longer find
> instances of these classes, it relates to JavaAssist dynamic
> class loading. Also, currently the server is idle with no
> connections and JConsole shows 18000 classes loaded,
> yesterday there were 7000 classes, this number never goes
> down only up, so you tell me what's going on?
>
> If Jessie can't explain it, then I have to go back a version
> we cant afford for clients to see broken pages, my boss
> doesn't care about the problem, he just wants a solution and
> its my job and reputation is at stake.
>
>
> Andreas Andreou wrote:
> > Peter,
> > though not officially final (I believe), http://news247.gr
> (tap-4.1.2)
> > has been getting 400000 req/day for an uptime of 26 days
> with memory
> > set to no more than 128MB.
> >
> >
> > On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> >
> >> Hi Jessie
> >>
> >> Any progress on this? .... sorry to bug you, but I have to take a
> >> decision soon, I have two production machines that will need to
> >> upgrade or downgrade Tapestry.
> >>
> >> Best wishes,
> >> Peter
> >>
> >> Jon Oakes wrote:
> >>
> >>> Hi Bryan,
> >>>
> >>> I am a relative newbie but I was wondering if you are
> running with
> >>> caching enabled or disabled? It might make sense to keep things
> >>> around when caching is disabled whereas I think it would
> clearly be
> >>> a bug to keep things around with it disabled.
> >>>
> >>> Jon Oakes
> >>>
> >>>
> >>> Jesse Kuhnert wrote:
> >>>
> >>>> Hmmm...well, I don't think I like the sound of any of
> that. I'm just
> >>>> going to pretend this problem doesn't exist. (just kidding)
> >>>>
> >>>> I had thought I was doing something special with the ognl
> >>>> compilations that would cause its generated classes to not hang
> >>>> around afterwards in any pools.
> >>>>
> >>>> I'll take a look at things this weekend and am sure a fix will
> >>>> appear in between now and Monday - if there is a
> reasonable fix to be found.
> >>>> (in 4.1.3 snapshot form)
> >>>>
> >>>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <mailto:
> >>>>
> >> BDotzour@widen.com> wrote:
> >>
> >>>>> I and another colleague of mine have been investigating
> what seems
> >>>>> to
> >>>>>
> >> be
> >>
> >>>>> a "memory leak" in our Tapestry application for about a month
> >>>>> since we upgraded to T4.1.2. I won't bore you with the saga of
> >>>>> the last month, but I would like to present the data
> I've gathered
> >>>>> and look to the
> >>>>>
> >> list
> >>
> >>>>> for a proposed solution. I was reading a recent thread
> in which
> >>>>> Jesse said (08/09/2007):
> >>>>>
> >>>>>
> >>>>>
> >>>>> "There is a map that grows as large as the system using it
> >>>>> internally
> >>>>>
> >> to
> >>
> >>>>> javassist of various cached reflection info - but it
> doesn't leak
> >>>>> in
> >>>>>
> >> any
> >>
> >>>>> way."
> >>>>>
> >>>>> This is precisely what I've found in profiling our
> application and
> >>>>> it
> >>>>> *appears* to be this map that is causing our applications to
> >>>>>
> >> eventually
> >>
> >>>>> run out of memory.
> >>>>>
> >>>>> The YourKit profiler shows me that, as time goes on,
> there is an
> >>>>> instance of HiveMindClassPool that grows and grows as class
> >>>>> instances are created. This class extends from
> >>>>> javassist.ClassPool and is the
> >>>>>
> >> map
> >>
> >>>>> that Jesse is talking about in his quote above. And
> he's right, I
> >>>>> wouldn't say that the class pool "leaks" either because
> it looks
> >>>>> like it's designed to retain that memory until the class pool
> >>>>> itself is no longer needed.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Take this quote from the javassist.ClassPool javadocs:
> >>>>>
> >>>>> "Memory consumption memo:
> >>>>> ClassPool objects hold all the CtClasses that have been
> created so
> >>>>>
> >> that
> >>
> >>>>> the consistency among modified classes can be
> guaranteed. Thus if
> >>>>> a large number of CtClasses are processed, the ClassPool will
> >>>>> consume a huge amount of memory. To avoid this, a
> ClassPool object
> >>>>> should be recreated, for example, every hundred classes
> processed.
> >>>>> Note that
> >>>>> getDefault() is a singleton factory. Otherwise, detach() in
> >>>>> CtClass should be used to avoid huge memory consumption. "
> >>>>>
> >>>>> This huge memory consumption by the ClassPool is what I was
> >>>>> seeing. In particular it is the ClassPool that is held
> onto by OgnlRuntime.
> >>>>> Inspecting this object in the profiler showed that it has a map
> >>>>> containing about 45,000 classes. All of the keys into this map
> >>>>> were things like: "ASTTest_11494aca9af" and
> "ASTAnd_11494ace4fb"
> >>>>> and the values are instances of javassist.CtNewClass.
> Each entry
> >>>>> in this map looks like it retains about 1,900 bytes,
> for a grand
> >>>>> total of about 90 MB of memory used.
> >>>>>
> >>>>> These numbers came from my staging deployment where I had the
> >>>>> profiler attached, using some reflection tricks I was
> able to look
> >>>>> at a production site and found that it had about
> 240,000 items in
> >>>>> that
> >>>>>
> >> class
> >>
> >>>>> pool.. approximately 450 MB of memory.
> >>>>>
> >>>>> So I guess the questions in my mind are: Why are there so many
> >>>>>
> >> classes
> >>
> >>>>> in the pool? Why does the number only ever go up? Do those
> >>>>> classes really need to stay in the pool forever?
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>
> --------------------------------------------------------------------
> >>> - To unsubscribe, e-mail:
> users-unsubscribe@tapestry.apache.org For
> >>> additional commands, e-mail: users-help@tapestry.apache.org
> >>>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> >> For additional commands, e-mail: users-help@tapestry.apache.org
> >>
> >>
> >>
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Andreas Andreou <an...@di.uoa.gr>.
On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>
>
> Andreas, give me a break! I am not trying to trash Tapestry if that's
> what you are thinking. I am not the only one experiencing these problems.
Hey - i never meant that!
It's obvious from the reports that something's going on + Jesse was on to
it..
I just thought i'd offer a counter-example + throw in an example
of a T4.1.2 public site ( i'm not aware of a lot)
Have a look at some of my postings (and others posts) and you will see
> what I am talking about. I have been having these problems since
> upgrading from 4.1.1 I never experienced these problems before with
> uptime of months at a time.
>
> Look at this log, id is an integer with the value 728:
>
> 21 Aug 2007 13:01:49,353 - INFO $Portfolio_89 - Setting the portfolio with
> the id: 728
> 21 Aug 2007 13:01:50,479 - ERROR $Error_118 - Unable to parse OGNL
> expression 'id': Unable to parse OGNL expression 'id': Unable to parse OGNL
> expression 'id': ............
> Exception [classpath:/org/apache/tapestry/form/TextArea.jwc, line 49,
> column 67]
> at org.apache.tapestry.binding.ExpressionBinding.resolveExpression
> (ExpressionBinding.java:145)
> at org.apache.tapestry.binding.ExpressionBinding.getObject(
> ExpressionBinding.java:125)
>
> And then take a look at this:
>
> 21 Aug 2007 07:56:50,613 - ERROR
> org.apache.tapestry.services.impl.HiveMindExpressionCompiler - Error
> generating OGNL getter for expression exceptions with root
> $Exception_119@3c1[framework:Exception] and body:
> { return (($Exception_119)$2).getExceptions();}
> org.apache.hivemind.ApplicationRuntimeException: Unable to add method
> java.lang.Object get(ognl.OgnlContext, java.lang.Object) to class
> $ASTProperty_1146a471a52: [source error] no such class: $Exception_119
>
> This happening after 3-4 days of normal operation? Suddenly exceptions
> occur on random objects because OGNL can no longer compile them and this
> is because Tapestry can no longer find instances of these classes, it
> relates to JavaAssist dynamic class loading. Also, currently the server
> is idle with no connections and JConsole shows 18000 classes loaded,
> yesterday there were 7000 classes, this number never goes down only up,
> so you tell me what's going on?
>
> If Jessie can't explain it, then I have to go back a version we cant
> afford for clients to see broken pages, my boss doesn't care about the
> problem, he just wants a solution and its my job and reputation is at
> stake.
>
>
> Andreas Andreou wrote:
> > Peter,
> > though not officially final (I believe),
> > http://news247.gr (tap-4.1.2) has been getting 400000 req/day
> > for an uptime of 26 days with memory set to no more than 128MB.
> >
> >
> > On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> >
> >> Hi Jessie
> >>
> >> Any progress on this? .... sorry to bug you, but I have to take a
> >> decision soon, I have two production machines that will need to upgrade
> >> or downgrade Tapestry.
> >>
> >> Best wishes,
> >> Peter
> >>
> >> Jon Oakes wrote:
> >>
> >>> Hi Bryan,
> >>>
> >>> I am a relative newbie but I was wondering if you are running with
> >>> caching enabled or disabled? It might make sense to keep things
> >>> around when caching is disabled whereas I think it would clearly be a
> >>> bug to keep things around with it disabled.
> >>>
> >>> Jon Oakes
> >>>
> >>>
> >>> Jesse Kuhnert wrote:
> >>>
> >>>> Hmmm...well, I don't think I like the sound of any of that. I'm
> just
> >>>> going to pretend this problem doesn't exist. (just kidding)
> >>>>
> >>>> I had thought I was doing something special with the ognl
> compilations
> >>>> that would cause its generated classes to not hang around afterwards
> >>>> in any pools.
> >>>>
> >>>> I'll take a look at things this weekend and am sure a fix will appear
> >>>> in between now and Monday - if there is a reasonable fix to be found.
> >>>> (in 4.1.3 snapshot form)
> >>>>
> >>>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <mailto:
> >>>>
> >> BDotzour@widen.com> wrote:
> >>
> >>>>> I and another colleague of mine have been investigating what seems
> to
> >>>>>
> >> be
> >>
> >>>>> a "memory leak" in our Tapestry application for about a month since
> we
> >>>>> upgraded to T4.1.2. I won't bore you with the saga of the last
> month,
> >>>>> but I would like to present the data I've gathered and look to the
> >>>>>
> >> list
> >>
> >>>>> for a proposed solution. I was reading a recent thread in which
> Jesse
> >>>>> said (08/09/2007):
> >>>>>
> >>>>>
> >>>>>
> >>>>> "There is a map that grows as large as the system using it
> internally
> >>>>>
> >> to
> >>
> >>>>> javassist of various cached reflection info - but it doesn't leak in
> >>>>>
> >> any
> >>
> >>>>> way."
> >>>>>
> >>>>> This is precisely what I've found in profiling our application and
> it
> >>>>> *appears* to be this map that is causing our applications to
> >>>>>
> >> eventually
> >>
> >>>>> run out of memory.
> >>>>>
> >>>>> The YourKit profiler shows me that, as time goes on, there is an
> >>>>> instance of HiveMindClassPool that grows and grows as class
> instances
> >>>>> are created. This class extends from javassist.ClassPool and is the
> >>>>>
> >> map
> >>
> >>>>> that Jesse is talking about in his quote above. And he's right, I
> >>>>> wouldn't say that the class pool "leaks" either because it looks
> like
> >>>>> it's designed to retain that memory until the class pool itself is
> no
> >>>>> longer needed.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Take this quote from the javassist.ClassPool javadocs:
> >>>>>
> >>>>> "Memory consumption memo:
> >>>>> ClassPool objects hold all the CtClasses that have been created so
> >>>>>
> >> that
> >>
> >>>>> the consistency among modified classes can be guaranteed. Thus if a
> >>>>> large number of CtClasses are processed, the ClassPool will consume
> a
> >>>>> huge amount of memory. To avoid this, a ClassPool object should be
> >>>>> recreated, for example, every hundred classes processed. Note that
> >>>>> getDefault() is a singleton factory. Otherwise, detach() in CtClass
> >>>>> should be used to avoid huge memory consumption. "
> >>>>>
> >>>>> This huge memory consumption by the ClassPool is what I was seeing.
> In
> >>>>> particular it is the ClassPool that is held onto by OgnlRuntime.
> >>>>> Inspecting this object in the profiler showed that it has a map
> >>>>> containing about 45,000 classes. All of the keys into this map were
> >>>>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
> >>>>> values are instances of javassist.CtNewClass. Each entry in this
> map
> >>>>> looks like it retains about 1,900 bytes, for a grand total of about
> 90
> >>>>> MB of memory used.
> >>>>>
> >>>>> These numbers came from my staging deployment where I had the
> profiler
> >>>>> attached, using some reflection tricks I was able to look at a
> >>>>> production site and found that it had about 240,000 items in that
> >>>>>
> >> class
> >>
> >>>>> pool.. approximately 450 MB of memory.
> >>>>>
> >>>>> So I guess the questions in my mind are: Why are there so many
> >>>>>
> >> classes
> >>
> >>>>> in the pool? Why does the number only ever go up? Do those classes
> >>>>> really need to stay in the pool forever?
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
> >>> additional commands, e-mail: users-help@tapestry.apache.org
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> >> For additional commands, e-mail: users-help@tapestry.apache.org
> >>
> >>
> >>
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
--
Andreas Andreou - andyhot@apache.org - http://andyhot.di.uoa.gr
Tapestry / Tacos developer
Open Source / JEE Consulting
Re: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
Andreas, give me a break! I am not trying to trash Tapestry if that's
what you are thinking. I am not the only one experiencing these problems.
Have a look at some of my postings (and others posts) and you will see
what I am talking about. I have been having these problems since
upgrading from 4.1.1 I never experienced these problems before with
uptime of months at a time.
Look at this log, id is an integer with the value 728:
21 Aug 2007 13:01:49,353 - INFO $Portfolio_89 - Setting the portfolio with the id: 728
21 Aug 2007 13:01:50,479 - ERROR $Error_118 - Unable to parse OGNL expression 'id': Unable to parse OGNL expression 'id': Unable to parse OGNL expression 'id': ............
Exception [classpath:/org/apache/tapestry/form/TextArea.jwc, line 49, column 67]
at org.apache.tapestry.binding.ExpressionBinding.resolveExpression(ExpressionBinding.java:145)
at org.apache.tapestry.binding.ExpressionBinding.getObject(ExpressionBinding.java:125)
And then take a look at this:
21 Aug 2007 07:56:50,613 - ERROR org.apache.tapestry.services.impl.HiveMindExpressionCompiler - Error generating OGNL getter for expression exceptions with root $Exception_119@3c1[framework:Exception] and body:
{ return (($Exception_119)$2).getExceptions();}
org.apache.hivemind.ApplicationRuntimeException: Unable to add method java.lang.Object get(ognl.OgnlContext, java.lang.Object) to class $ASTProperty_1146a471a52: [source error] no such class: $Exception_119
This happening after 3-4 days of normal operation? Suddenly exceptions
occur on random objects because OGNL can no longer compile them and this
is because Tapestry can no longer find instances of these classes, it
relates to JavaAssist dynamic class loading. Also, currently the server
is idle with no connections and JConsole shows 18000 classes loaded,
yesterday there were 7000 classes, this number never goes down only up,
so you tell me what's going on?
If Jessie can't explain it, then I have to go back a version we cant
afford for clients to see broken pages, my boss doesn't care about the
problem, he just wants a solution and its my job and reputation is at
stake.
Andreas Andreou wrote:
> Peter,
> though not officially final (I believe),
> http://news247.gr (tap-4.1.2) has been getting 400000 req/day
> for an uptime of 26 days with memory set to no more than 128MB.
>
>
> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>
>> Hi Jessie
>>
>> Any progress on this? .... sorry to bug you, but I have to take a
>> decision soon, I have two production machines that will need to upgrade
>> or downgrade Tapestry.
>>
>> Best wishes,
>> Peter
>>
>> Jon Oakes wrote:
>>
>>> Hi Bryan,
>>>
>>> I am a relative newbie but I was wondering if you are running with
>>> caching enabled or disabled? It might make sense to keep things
>>> around when caching is disabled whereas I think it would clearly be a
>>> bug to keep things around with it disabled.
>>>
>>> Jon Oakes
>>>
>>>
>>> Jesse Kuhnert wrote:
>>>
>>>> Hmmm...well, I don't think I like the sound of any of that. I'm just
>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>
>>>> I had thought I was doing something special with the ognl compilations
>>>> that would cause its generated classes to not hang around afterwards
>>>> in any pools.
>>>>
>>>> I'll take a look at things this weekend and am sure a fix will appear
>>>> in between now and Monday - if there is a reasonable fix to be found.
>>>> (in 4.1.3 snapshot form)
>>>>
>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <mailto:
>>>>
>> BDotzour@widen.com> wrote:
>>
>>>>> I and another colleague of mine have been investigating what seems to
>>>>>
>> be
>>
>>>>> a "memory leak" in our Tapestry application for about a month since we
>>>>> upgraded to T4.1.2. I won't bore you with the saga of the last month,
>>>>> but I would like to present the data I've gathered and look to the
>>>>>
>> list
>>
>>>>> for a proposed solution. I was reading a recent thread in which Jesse
>>>>> said (08/09/2007):
>>>>>
>>>>>
>>>>>
>>>>> "There is a map that grows as large as the system using it internally
>>>>>
>> to
>>
>>>>> javassist of various cached reflection info - but it doesn't leak in
>>>>>
>> any
>>
>>>>> way."
>>>>>
>>>>> This is precisely what I've found in profiling our application and it
>>>>> *appears* to be this map that is causing our applications to
>>>>>
>> eventually
>>
>>>>> run out of memory.
>>>>>
>>>>> The YourKit profiler shows me that, as time goes on, there is an
>>>>> instance of HiveMindClassPool that grows and grows as class instances
>>>>> are created. This class extends from javassist.ClassPool and is the
>>>>>
>> map
>>
>>>>> that Jesse is talking about in his quote above. And he's right, I
>>>>> wouldn't say that the class pool "leaks" either because it looks like
>>>>> it's designed to retain that memory until the class pool itself is no
>>>>> longer needed.
>>>>>
>>>>>
>>>>>
>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>
>>>>> "Memory consumption memo:
>>>>> ClassPool objects hold all the CtClasses that have been created so
>>>>>
>> that
>>
>>>>> the consistency among modified classes can be guaranteed. Thus if a
>>>>> large number of CtClasses are processed, the ClassPool will consume a
>>>>> huge amount of memory. To avoid this, a ClassPool object should be
>>>>> recreated, for example, every hundred classes processed. Note that
>>>>> getDefault() is a singleton factory. Otherwise, detach() in CtClass
>>>>> should be used to avoid huge memory consumption. "
>>>>>
>>>>> This huge memory consumption by the ClassPool is what I was seeing. In
>>>>> particular it is the ClassPool that is held onto by OgnlRuntime.
>>>>> Inspecting this object in the profiler showed that it has a map
>>>>> containing about 45,000 classes. All of the keys into this map were
>>>>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
>>>>> values are instances of javassist.CtNewClass. Each entry in this map
>>>>> looks like it retains about 1,900 bytes, for a grand total of about 90
>>>>> MB of memory used.
>>>>>
>>>>> These numbers came from my staging deployment where I had the profiler
>>>>> attached, using some reflection tricks I was able to look at a
>>>>> production site and found that it had about 240,000 items in that
>>>>>
>> class
>>
>>>>> pool.. approximately 450 MB of memory.
>>>>>
>>>>> So I guess the questions in my mind are: Why are there so many
>>>>>
>> classes
>>
>>>>> in the pool? Why does the number only ever go up? Do those classes
>>>>> really need to stay in the pool forever?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Andreas Andreou <an...@di.uoa.gr>.
Peter,
though not officially final (I believe),
http://news247.gr (tap-4.1.2) has been getting 400000 req/day
for an uptime of 26 days with memory set to no more than 128MB.
On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>
> Hi Jessie
>
> Any progress on this? .... sorry to bug you, but I have to take a
> decision soon, I have two production machines that will need to upgrade
> or downgrade Tapestry.
>
> Best wishes,
> Peter
>
> Jon Oakes wrote:
> > Hi Bryan,
> >
> > I am a relative newbie but I was wondering if you are running with
> > caching enabled or disabled? It might make sense to keep things
> > around when caching is disabled whereas I think it would clearly be a
> > bug to keep things around with it disabled.
> >
> > Jon Oakes
> >
> >
> > Jesse Kuhnert wrote:
> >> Hmmm...well, I don't think I like the sound of any of that. I'm just
> >> going to pretend this problem doesn't exist. (just kidding)
> >>
> >> I had thought I was doing something special with the ognl compilations
> >> that would cause its generated classes to not hang around afterwards
> >> in any pools.
> >>
> >> I'll take a look at things this weekend and am sure a fix will appear
> >> in between now and Monday - if there is a reasonable fix to be found.
> >> (in 4.1.3 snapshot form)
> >>
> >> On 8/24/07, Bryan Dotzour <BD...@widen.com> <mailto:
> BDotzour@widen.com> wrote:
> >>
> >>> I and another colleague of mine have been investigating what seems to
> be
> >>> a "memory leak" in our Tapestry application for about a month since we
> >>> upgraded to T4.1.2. I won't bore you with the saga of the last month,
> >>> but I would like to present the data I've gathered and look to the
> list
> >>> for a proposed solution. I was reading a recent thread in which Jesse
> >>> said (08/09/2007):
> >>>
> >>>
> >>>
> >>> "There is a map that grows as large as the system using it internally
> to
> >>> javassist of various cached reflection info - but it doesn't leak in
> any
> >>> way."
> >>>
> >>> This is precisely what I've found in profiling our application and it
> >>> *appears* to be this map that is causing our applications to
> eventually
> >>> run out of memory.
> >>>
> >>> The YourKit profiler shows me that, as time goes on, there is an
> >>> instance of HiveMindClassPool that grows and grows as class instances
> >>> are created. This class extends from javassist.ClassPool and is the
> map
> >>> that Jesse is talking about in his quote above. And he's right, I
> >>> wouldn't say that the class pool "leaks" either because it looks like
> >>> it's designed to retain that memory until the class pool itself is no
> >>> longer needed.
> >>>
> >>>
> >>>
> >>> Take this quote from the javassist.ClassPool javadocs:
> >>>
> >>> "Memory consumption memo:
> >>> ClassPool objects hold all the CtClasses that have been created so
> that
> >>> the consistency among modified classes can be guaranteed. Thus if a
> >>> large number of CtClasses are processed, the ClassPool will consume a
> >>> huge amount of memory. To avoid this, a ClassPool object should be
> >>> recreated, for example, every hundred classes processed. Note that
> >>> getDefault() is a singleton factory. Otherwise, detach() in CtClass
> >>> should be used to avoid huge memory consumption. "
> >>>
> >>> This huge memory consumption by the ClassPool is what I was seeing. In
> >>> particular it is the ClassPool that is held onto by OgnlRuntime.
> >>> Inspecting this object in the profiler showed that it has a map
> >>> containing about 45,000 classes. All of the keys into this map were
> >>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
> >>> values are instances of javassist.CtNewClass. Each entry in this map
> >>> looks like it retains about 1,900 bytes, for a grand total of about 90
> >>> MB of memory used.
> >>>
> >>> These numbers came from my staging deployment where I had the profiler
> >>> attached, using some reflection tricks I was able to look at a
> >>> production site and found that it had about 240,000 items in that
> class
> >>> pool.. approximately 450 MB of memory.
> >>>
> >>> So I guess the questions in my mind are: Why are there so many
> classes
> >>> in the pool? Why does the number only ever go up? Do those classes
> >>> really need to stay in the pool forever?
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
> > additional commands, e-mail: users-help@tapestry.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
--
Andreas Andreou - andyhot@apache.org - http://andyhot.di.uoa.gr
Tapestry / Tacos developer
Open Source / JEE Consulting
Re: Memory consumption in T4.1.2 - Hard data
Posted by Patrick Moore <pa...@amplafi.com>.
Hi Jesse --
Thanks for your work on Tapestry. I have noticed an issue around memory
usage with our automated tests which extensively use hivemind. I was
wondering if anyone has investigated hivemind memory leaks. I am going to do
some further investigating as time permits after Tuesday. That being said,
we have a publicly-accessible machine with the latest yourkit 7m2 release on
it that could be used to get long-running memory information.Contact me
off-list about getting access to the instance.This is our production machine
- but since we are still in beta we don't have many users (i.e. no one will
notice a little memory profiling).
I would suggest the Tapestry community pitch in a little to help gather data
to track down this issue - or determine it is a non-issue. One of open
source's advantages is the power of the many to solve issues like this. I
for one don't want to rely just on jesse to solve what sounds to be very
serious barrier to my own personal success.
-Pat Moore
Amplafi
On 8/28/07, Jesse Kuhnert <jk...@gmail.com> wrote:
>
> Umm yeah. I have looked in to it as I just said.
>
> The only thing left is for me to look at this youkit snapshot profile
> and play from there but I'm not capable of debugging issues that
> aren't reported properly.
>
> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> > Hi Jessie
> >
> > I'm gonna try update my version of ognl, if that won't work I will
> > downgrade. I have not filed any bugs yet, but I sent a couple of emails
> > to the list. I don't get any exceptions during development, but that is
> > because its not running for 3 or 4 days. I strongly suggest you consider
> > looking into it, because I use the standard libraries provided by
> > Tapestry through Maven.
> >
> > my configuration is as follows:
> >
> > JDK 6.01 32bit JVM (I have also tested on a 64 bit with no luck)
> > Tomcat 5.5.20
> > Debian Linux (2.6.15.28 kernel)
> > Tapestry 4.1.2-SNAPSHOT with ognl 2.7
> >
> > regards,
> > Peter
> >
> > Jesse Kuhnert wrote:
> > > I'd downgrade then if I were you. Extensive profiling with yourkit
> > > hasn't shed any new light on whatever problems others are having.
> > > The OGNL classes indeed aren't being kept in the javassist pool from
> > > what I can tell.
> > >
> > > I have another yourkit snapshot binary to look at so we'll see what
> that says.
> > >
> > > I don't remember - have you filed any bug reports or reported any
> > > problems? Are you getting ognl exceptions during development as well
> > > as production? Do these errors sound like ummmmm....potential errors
> > > that should be looked at further?
> > >
> > > On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> > >
> > >> Hi Jessie
> > >>
> > >> Any progress on this? .... sorry to bug you, but I have to take a
> > >> decision soon, I have two production machines that will need to
> upgrade
> > >> or downgrade Tapestry.
> > >>
> > >> Best wishes,
> > >> Peter
> > >>
> > >> Jon Oakes wrote:
> > >>
> > >>> Hi Bryan,
> > >>>
> > >>> I am a relative newbie but I was wondering if you are running with
> > >>> caching enabled or disabled? It might make sense to keep things
> > >>> around when caching is disabled whereas I think it would clearly be
> a
> > >>> bug to keep things around with it disabled.
> > >>>
> > >>> Jon Oakes
> > >>>
> > >>>
> > >>> Jesse Kuhnert wrote:
> > >>>
> > >>>> Hmmm...well, I don't think I like the sound of any of that. I'm
> just
> > >>>> going to pretend this problem doesn't exist. (just kidding)
> > >>>>
> > >>>> I had thought I was doing something special with the ognl
> compilations
> > >>>> that would cause its generated classes to not hang around
> afterwards
> > >>>> in any pools.
> > >>>>
> > >>>> I'll take a look at things this weekend and am sure a fix will
> appear
> > >>>> in between now and Monday - if there is a reasonable fix to be
> found.
> > >>>> (in 4.1.3 snapshot form)
> > >>>>
> > >>>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <mailto:
> BDotzour@widen.com> wrote:
> > >>>>
> > >>>>
> > >>>>> I and another colleague of mine have been investigating what seems
> to be
> > >>>>> a "memory leak" in our Tapestry application for about a month
> since we
> > >>>>> upgraded to T4.1.2. I won't bore you with the saga of the last
> month,
> > >>>>> but I would like to present the data I've gathered and look to the
> list
> > >>>>> for a proposed solution. I was reading a recent thread in which
> Jesse
> > >>>>> said (08/09/2007):
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> "There is a map that grows as large as the system using it
> internally to
> > >>>>> javassist of various cached reflection info - but it doesn't leak
> in any
> > >>>>> way."
> > >>>>>
> > >>>>> This is precisely what I've found in profiling our application and
> it
> > >>>>> *appears* to be this map that is causing our applications to
> eventually
> > >>>>> run out of memory.
> > >>>>>
> > >>>>> The YourKit profiler shows me that, as time goes on, there is an
> > >>>>> instance of HiveMindClassPool that grows and grows as class
> instances
> > >>>>> are created. This class extends from javassist.ClassPool and is
> the map
> > >>>>> that Jesse is talking about in his quote above. And he's right, I
> > >>>>> wouldn't say that the class pool "leaks" either because it looks
> like
> > >>>>> it's designed to retain that memory until the class pool itself is
> no
> > >>>>> longer needed.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> Take this quote from the javassist.ClassPool javadocs:
> > >>>>>
> > >>>>> "Memory consumption memo:
> > >>>>> ClassPool objects hold all the CtClasses that have been created so
> that
> > >>>>> the consistency among modified classes can be guaranteed. Thus if
> a
> > >>>>> large number of CtClasses are processed, the ClassPool will
> consume a
> > >>>>> huge amount of memory. To avoid this, a ClassPool object should be
> > >>>>> recreated, for example, every hundred classes processed. Note that
> > >>>>> getDefault() is a singleton factory. Otherwise, detach() in
> CtClass
> > >>>>> should be used to avoid huge memory consumption. "
> > >>>>>
> > >>>>> This huge memory consumption by the ClassPool is what I was
> seeing. In
> > >>>>> particular it is the ClassPool that is held onto by OgnlRuntime.
> > >>>>> Inspecting this object in the profiler showed that it has a map
> > >>>>> containing about 45,000 classes. All of the keys into this map
> were
> > >>>>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and
> the
> > >>>>> values are instances of javassist.CtNewClass. Each entry in this
> map
> > >>>>> looks like it retains about 1,900 bytes, for a grand total of
> about 90
> > >>>>> MB of memory used.
> > >>>>>
> > >>>>> These numbers came from my staging deployment where I had the
> profiler
> > >>>>> attached, using some reflection tricks I was able to look at a
> > >>>>> production site and found that it had about 240,000 items in that
> class
> > >>>>> pool.. approximately 450 MB of memory.
> > >>>>>
> > >>>>> So I guess the questions in my mind are: Why are there so many
> classes
> > >>>>> in the pool? Why does the number only ever go up? Do those
> classes
> > >>>>> really need to stay in the pool forever?
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>
> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
> > >>> additional commands, e-mail: users-help@tapestry.apache.org
> > >>>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> > >> For additional commands, e-mail: users-help@tapestry.apache.org
> > >>
> > >>
> > >>
> > >
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> > For additional commands, e-mail: users-help@tapestry.apache.org
> >
> >
>
>
> --
> Jesse Kuhnert
> Tapestry/Dojo team member/developer
>
> Open source based consulting work centered around
> dojo/tapestry/tacos/hivemind. http://blog.opencomponentry.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
Re: Memory consumption in T4.1.2 - Hard data
Posted by Jesse Kuhnert <jk...@gmail.com>.
Umm yeah. I have looked in to it as I just said.
The only thing left is for me to look at this youkit snapshot profile
and play from there but I'm not capable of debugging issues that
aren't reported properly.
On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> Hi Jessie
>
> I'm gonna try update my version of ognl, if that won't work I will
> downgrade. I have not filed any bugs yet, but I sent a couple of emails
> to the list. I don't get any exceptions during development, but that is
> because its not running for 3 or 4 days. I strongly suggest you consider
> looking into it, because I use the standard libraries provided by
> Tapestry through Maven.
>
> my configuration is as follows:
>
> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no luck)
> Tomcat 5.5.20
> Debian Linux (2.6.15.28 kernel)
> Tapestry 4.1.2-SNAPSHOT with ognl 2.7
>
> regards,
> Peter
>
> Jesse Kuhnert wrote:
> > I'd downgrade then if I were you. Extensive profiling with yourkit
> > hasn't shed any new light on whatever problems others are having.
> > The OGNL classes indeed aren't being kept in the javassist pool from
> > what I can tell.
> >
> > I have another yourkit snapshot binary to look at so we'll see what that says.
> >
> > I don't remember - have you filed any bug reports or reported any
> > problems? Are you getting ognl exceptions during development as well
> > as production? Do these errors sound like ummmmm....potential errors
> > that should be looked at further?
> >
> > On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> >
> >> Hi Jessie
> >>
> >> Any progress on this? .... sorry to bug you, but I have to take a
> >> decision soon, I have two production machines that will need to upgrade
> >> or downgrade Tapestry.
> >>
> >> Best wishes,
> >> Peter
> >>
> >> Jon Oakes wrote:
> >>
> >>> Hi Bryan,
> >>>
> >>> I am a relative newbie but I was wondering if you are running with
> >>> caching enabled or disabled? It might make sense to keep things
> >>> around when caching is disabled whereas I think it would clearly be a
> >>> bug to keep things around with it disabled.
> >>>
> >>> Jon Oakes
> >>>
> >>>
> >>> Jesse Kuhnert wrote:
> >>>
> >>>> Hmmm...well, I don't think I like the sound of any of that. I'm just
> >>>> going to pretend this problem doesn't exist. (just kidding)
> >>>>
> >>>> I had thought I was doing something special with the ognl compilations
> >>>> that would cause its generated classes to not hang around afterwards
> >>>> in any pools.
> >>>>
> >>>> I'll take a look at things this weekend and am sure a fix will appear
> >>>> in between now and Monday - if there is a reasonable fix to be found.
> >>>> (in 4.1.3 snapshot form)
> >>>>
> >>>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <ma...@widen.com> wrote:
> >>>>
> >>>>
> >>>>> I and another colleague of mine have been investigating what seems to be
> >>>>> a "memory leak" in our Tapestry application for about a month since we
> >>>>> upgraded to T4.1.2. I won't bore you with the saga of the last month,
> >>>>> but I would like to present the data I've gathered and look to the list
> >>>>> for a proposed solution. I was reading a recent thread in which Jesse
> >>>>> said (08/09/2007):
> >>>>>
> >>>>>
> >>>>>
> >>>>> "There is a map that grows as large as the system using it internally to
> >>>>> javassist of various cached reflection info - but it doesn't leak in any
> >>>>> way."
> >>>>>
> >>>>> This is precisely what I've found in profiling our application and it
> >>>>> *appears* to be this map that is causing our applications to eventually
> >>>>> run out of memory.
> >>>>>
> >>>>> The YourKit profiler shows me that, as time goes on, there is an
> >>>>> instance of HiveMindClassPool that grows and grows as class instances
> >>>>> are created. This class extends from javassist.ClassPool and is the map
> >>>>> that Jesse is talking about in his quote above. And he's right, I
> >>>>> wouldn't say that the class pool "leaks" either because it looks like
> >>>>> it's designed to retain that memory until the class pool itself is no
> >>>>> longer needed.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Take this quote from the javassist.ClassPool javadocs:
> >>>>>
> >>>>> "Memory consumption memo:
> >>>>> ClassPool objects hold all the CtClasses that have been created so that
> >>>>> the consistency among modified classes can be guaranteed. Thus if a
> >>>>> large number of CtClasses are processed, the ClassPool will consume a
> >>>>> huge amount of memory. To avoid this, a ClassPool object should be
> >>>>> recreated, for example, every hundred classes processed. Note that
> >>>>> getDefault() is a singleton factory. Otherwise, detach() in CtClass
> >>>>> should be used to avoid huge memory consumption. "
> >>>>>
> >>>>> This huge memory consumption by the ClassPool is what I was seeing. In
> >>>>> particular it is the ClassPool that is held onto by OgnlRuntime.
> >>>>> Inspecting this object in the profiler showed that it has a map
> >>>>> containing about 45,000 classes. All of the keys into this map were
> >>>>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
> >>>>> values are instances of javassist.CtNewClass. Each entry in this map
> >>>>> looks like it retains about 1,900 bytes, for a grand total of about 90
> >>>>> MB of memory used.
> >>>>>
> >>>>> These numbers came from my staging deployment where I had the profiler
> >>>>> attached, using some reflection tricks I was able to look at a
> >>>>> production site and found that it had about 240,000 items in that class
> >>>>> pool.. approximately 450 MB of memory.
> >>>>>
> >>>>> So I guess the questions in my mind are: Why are there so many classes
> >>>>> in the pool? Why does the number only ever go up? Do those classes
> >>>>> really need to stay in the pool forever?
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
> >>> additional commands, e-mail: users-help@tapestry.apache.org
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> >> For additional commands, e-mail: users-help@tapestry.apache.org
> >>
> >>
> >>
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
--
Jesse Kuhnert
Tapestry/Dojo team member/developer
Open source based consulting work centered around
dojo/tapestry/tacos/hivemind. http://blog.opencomponentry.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
Yes
Marcus.Schulte@bmw.ch wrote:
>
>
>> my configuration is as follows:
>>
>> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no
>> luck) Tomcat 5.5.20 Debian Linux (2.6.15.28 kernel) Tapestry
>> 4.1.2-SNAPSHOT with ognl 2.7
>>
>>
>
> Hi Peter, 4.1.2-SNAPSHOT is a typo, I suppose?
>
>
>> regards,
>> Peter
>>
>> Jesse Kuhnert wrote:
>>
>>> I'd downgrade then if I were you. Extensive profiling with yourkit
>>> hasn't shed any new light on whatever problems others are having.
>>> The OGNL classes indeed aren't being kept in the javassist
>>>
>> pool from
>>
>>> what I can tell.
>>>
>>> I have another yourkit snapshot binary to look at so we'll
>>>
>> see what that says.
>>
>>> I don't remember - have you filed any bug reports or reported any
>>> problems? Are you getting ognl exceptions during
>>>
>> development as well
>>
>>> as production? Do these errors sound like
>>>
>> ummmmm....potential errors
>>
>>> that should be looked at further?
>>>
>>> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>>>
>>>
>>>> Hi Jessie
>>>>
>>>> Any progress on this? .... sorry to bug you, but I have to take a
>>>> decision soon, I have two production machines that will need to
>>>> upgrade or downgrade Tapestry.
>>>>
>>>> Best wishes,
>>>> Peter
>>>>
>>>> Jon Oakes wrote:
>>>>
>>>>
>>>>> Hi Bryan,
>>>>>
>>>>> I am a relative newbie but I was wondering if you are
>>>>>
>> running with
>>
>>>>> caching enabled or disabled? It might make sense to keep things
>>>>> around when caching is disabled whereas I think it would
>>>>>
>> clearly be
>>
>>>>> a bug to keep things around with it disabled.
>>>>>
>>>>> Jon Oakes
>>>>>
>>>>>
>>>>> Jesse Kuhnert wrote:
>>>>>
>>>>>
>>>>>> Hmmm...well, I don't think I like the sound of any of
>>>>>>
>> that. I'm just
>>
>>>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>>>
>>>>>> I had thought I was doing something special with the ognl
>>>>>> compilations that would cause its generated classes to not hang
>>>>>> around afterwards in any pools.
>>>>>>
>>>>>> I'll take a look at things this weekend and am sure a fix will
>>>>>> appear in between now and Monday - if there is a
>>>>>>
>> reasonable fix to be found.
>>
>>>>>> (in 4.1.3 snapshot form)
>>>>>>
>>>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com>
>>>>>>
>> <ma...@widen.com> wrote:
>>
>>>>>>
>>>>>>
>>>>>>> I and another colleague of mine have been investigating
>>>>>>>
>> what seems
>>
>>>>>>> to be a "memory leak" in our Tapestry application for about a
>>>>>>> month since we upgraded to T4.1.2. I won't bore you
>>>>>>>
>> with the saga
>>
>>>>>>> of the last month, but I would like to present the data I've
>>>>>>> gathered and look to the list for a proposed solution. I was
>>>>>>> reading a recent thread in which Jesse said (08/09/2007):
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> "There is a map that grows as large as the system using it
>>>>>>> internally to javassist of various cached reflection
>>>>>>>
>> info - but it
>>
>>>>>>> doesn't leak in any way."
>>>>>>>
>>>>>>> This is precisely what I've found in profiling our
>>>>>>>
>> application and
>>
>>>>>>> it
>>>>>>> *appears* to be this map that is causing our applications to
>>>>>>> eventually run out of memory.
>>>>>>>
>>>>>>> The YourKit profiler shows me that, as time goes on,
>>>>>>>
>> there is an
>>
>>>>>>> instance of HiveMindClassPool that grows and grows as class
>>>>>>> instances are created. This class extends from
>>>>>>> javassist.ClassPool and is the map that Jesse is
>>>>>>>
>> talking about in
>>
>>>>>>> his quote above. And he's right, I wouldn't say that the class
>>>>>>> pool "leaks" either because it looks like it's designed
>>>>>>>
>> to retain
>>
>>>>>>> that memory until the class pool itself is no longer needed.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>>>
>>>>>>> "Memory consumption memo:
>>>>>>> ClassPool objects hold all the CtClasses that have been
>>>>>>>
>> created so
>>
>>>>>>> that the consistency among modified classes can be guaranteed.
>>>>>>> Thus if a large number of CtClasses are processed, the
>>>>>>>
>> ClassPool
>>
>>>>>>> will consume a huge amount of memory. To avoid this, a
>>>>>>>
>> ClassPool
>>
>>>>>>> object should be recreated, for example, every hundred classes
>>>>>>> processed. Note that
>>>>>>> getDefault() is a singleton factory. Otherwise, detach() in
>>>>>>> CtClass should be used to avoid huge memory consumption. "
>>>>>>>
>>>>>>> This huge memory consumption by the ClassPool is what I was
>>>>>>> seeing. In particular it is the ClassPool that is held
>>>>>>>
>> onto by OgnlRuntime.
>>
>>>>>>> Inspecting this object in the profiler showed that it has a map
>>>>>>> containing about 45,000 classes. All of the keys into this map
>>>>>>> were things like: "ASTTest_11494aca9af" and
>>>>>>>
>> "ASTAnd_11494ace4fb"
>>
>>>>>>> and the values are instances of javassist.CtNewClass.
>>>>>>>
>> Each entry
>>
>>>>>>> in this map looks like it retains about 1,900 bytes,
>>>>>>>
>> for a grand
>>
>>>>>>> total of about 90 MB of memory used.
>>>>>>>
>>>>>>> These numbers came from my staging deployment where I had the
>>>>>>> profiler attached, using some reflection tricks I was
>>>>>>>
>> able to look
>>
>>>>>>> at a production site and found that it had about
>>>>>>>
>> 240,000 items in
>>
>>>>>>> that class pool.. approximately 450 MB of memory.
>>>>>>>
>>>>>>> So I guess the questions in my mind are: Why are there so many
>>>>>>> classes in the pool? Why does the number only ever go up? Do
>>>>>>> those classes really need to stay in the pool forever?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>> --------------------------------------------------------------------
>>
>>>>> - To unsubscribe, e-mail:
>>>>>
>> users-unsubscribe@tapestry.apache.org For
>>
>>>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>
>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: AW: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
The problem continues... I will file a bug report.
Peter Stavrinides wrote:
> Okay, I will try and build with this version... lets see if it makes a
> difference.
>
> Holger Stolzenberg wrote:
>> T4.1.2 is released and in the main repo:
>>
>> http://mvnrepository.com/artifact/org.apache.tapestry/tapestry-framework
>>
>> My POM:
>>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-framework</artifactId>
>> <version>4.1.2</version>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-annotations</artifactId>
>> <version>4.1.2</version>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-contrib</artifactId>
>> <version>4.1.2</version>
>> </dependency>
>> <dependency>
>> <groupId>com.ewerk</groupId>
>> <artifactId>ewerk-tapestry-components</artifactId>
>> <version>0.0.7</version>
>> </dependency>
>> <dependency>
>> <groupId>com.javaforge.tapestry</groupId>
>> <artifactId>tapestry-spring</artifactId>
>> <version>1.0.0</version>
>> <!-- exclude the old referenced version of tapestry -->
>> <exclusions>
>> <exclusion>
>> <groupId>tapestry</groupId>
>> <artifactId>tapestry</artifactId>
>> </exclusion>
>> <exclusion>
>> <groupId>tapestry</groupId>
>> <artifactId>tapestry-annotations</artifactId>
>> </exclusion>
>> </exclusions>
>> </dependency>
>>
>> Mit lieben Grüßen aus dem eWerk
>>
>> | Holger Stolzenberg
>> | Softwareentwickler
>> |
>> | Geschäftsführer: | Frank Richter, Erik Wende, Hendrik Schubert
>> |
>> | eWerk IT GmbH
>> | Markt 16
>> | Leipzig 04109
>> | http://www.ewerk.com
>> | HRB 9065, AG Leipzig
>> | Hauptniederlassung Leipzig
>> |
>> | fon +49.341.4 26 49-0
>> | fax +49.341.4 26 49-88
>> | mailto:h.stolzenberg@ewerk.com
>> |
>> | Support:
>> | fon 0700 CALLME24 (0700 22556324)
>> | fax 0700 CALLME24 (0700 22556324)
>> |
>> | Auskünfte und Angebote per Mail
>> | sind freibleibend und unverbindlich.
>> -----Ursprüngliche Nachricht-----
>> Von: Peter Stavrinides [mailto:p.stavrinides@albourne.com] Gesendet:
>> Mittwoch, 29. August 2007 09:31
>> An: Tapestry users
>> Betreff: Re: Memory consumption in T4.1.2 - Hard data
>>
>> Correct me if I am wrong 4.1.2 is not released as yet, and not in the
>> main repository, so how else would I get to it?
>>
>> Marcus.Schulte@bmw.ch wrote:
>>
>>> But that POM does use snapshots.
>>> You shouldn't need the repository
>>> http://people.apache.org/repo/m2-snapshot-repository
>>> at all.
>>> Probably, there are no very significant differences between the latest
>>> 4.1.2 snapshot and the release. But nevertheless, for a productive
>>> environment, I'd always go with a released version.
>>>
>>>
>>>> -----Original Message-----
>>>> From: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
>>>> Sent: Wednesday, August 29, 2007 7:45 AM
>>>> To: Tapestry users
>>>> Subject: Re: Memory consumption in T4.1.2 - Hard data
>>>>
>>>> This is my POM:
>>>>
>>>> <repositories>
>>>> <repository>
>>>> <releases>
>>>> <updatePolicy>always</updatePolicy>
>>>> <checksumPolicy>warn</checksumPolicy>
>>>> </releases>
>>>> <snapshots>
>>>> <updatePolicy>always</updatePolicy>
>>>> <checksumPolicy>fail</checksumPolicy>
>>>> <repository>
>>>> <id>apache.snapshots</id>
>>>> <url>http://people.apache.org/repo/m2-snapshot-repository</url>
>>>> </repository>
>>>> </repositories>
>>>>
>>>> <dependency>
>>>> <groupId>org.apache.tapestry</groupId>
>>>> <artifactId>tapestry-framework</artifactId>
>>>> <version>4.1.2-SNAPSHOT</version>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.tapestry</groupId>
>>>> <artifactId>tapestry-annotations</artifactId>
>>>> <version>4.1.2-SNAPSHOT</version>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.tapestry</groupId>
>>>> <artifactId>tapestry-contrib</artifactId>
>>>> <version>4.1.2-SNAPSHOT</version>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.tapestry</groupId>
>>>> <artifactId>tapestry-portlet</artifactId>
>>>> <version>4.1.2-SNAPSHOT</version>
>>>> <scope>test</scope>
>>>> </dependency>
>>>>
>>>> Marcus.Schulte@bmw.ch wrote:
>>>>
>>>>>
>>>>>
>>>>>> my configuration is as follows:
>>>>>>
>>>>>> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no
>>>>>> luck) Tomcat 5.5.20 Debian Linux (2.6.15.28 kernel) Tapestry
>>>>>> 4.1.2-SNAPSHOT with ognl 2.7
>>>>>>
>>>>>>
>>>>> Hi Peter, 4.1.2-SNAPSHOT is a typo, I suppose?
>>>>>
>>>>>
>>>>>> regards,
>>>>>> Peter
>>>>>>
>>>>>> Jesse Kuhnert wrote:
>>>>>>
>>>>>>> I'd downgrade then if I were you. Extensive profiling
>>>>>>>
>>>> with yourkit
>>>>
>>>>>>> hasn't shed any new light on whatever problems others are having.
>>>>>>> The OGNL classes indeed aren't being kept in the javassist
>>>>>>>
>>>>>> pool from
>>>>>>
>>>>>>> what I can tell.
>>>>>>>
>>>>>>> I have another yourkit snapshot binary to look at so we'll
>>>>>>>
>>>>>> see what that says.
>>>>>>
>>>>>>> I don't remember - have you filed any bug reports or reported
>>>>>>> any problems? Are you getting ognl exceptions during
>>>>>>>
>>>>>> development as well
>>>>>>
>>>>>>> as production? Do these errors sound like
>>>>>>>
>>>>>> ummmmm....potential errors
>>>>>>
>>>>>>> that should be looked at further?
>>>>>>>
>>>>>>> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>>>>>>>
>>>>>>>> Hi Jessie
>>>>>>>>
>>>>>>>> Any progress on this? .... sorry to bug you, but I have
>>>>>>>>
>>>> to take a
>>>>
>>>>>>>> decision soon, I have two production machines that will need to
>>>>>>>> upgrade or downgrade Tapestry.
>>>>>>>>
>>>>>>>> Best wishes,
>>>>>>>> Peter
>>>>>>>>
>>>>>>>> Jon Oakes wrote:
>>>>>>>>
>>>>>>>>> Hi Bryan,
>>>>>>>>>
>>>>>>>>> I am a relative newbie but I was wondering if you are
>>>>>>>>>
>>>>>> running with
>>>>>>
>>>>>>>>> caching enabled or disabled? It might make sense to
>>>>>>>>>
>>>> keep things
>>>>
>>>>>>>>> around when caching is disabled whereas I think it would
>>>>>>>>>
>>>>>> clearly be
>>>>>>
>>>>>>>>> a bug to keep things around with it disabled.
>>>>>>>>>
>>>>>>>>> Jon Oakes
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Jesse Kuhnert wrote:
>>>>>>>>>
>>>>>>>>>> Hmmm...well, I don't think I like the sound of any of
>>>>>>>>>>
>>>>>> that. I'm just
>>>>>>
>>>>>>>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>>>>>>>
>>>>>>>>>> I had thought I was doing something special with the ognl
>>>>>>>>>> compilations that would cause its generated classes to
>>>>>>>>>>
>>>> not hang
>>>>
>>>>>>>>>> around afterwards in any pools.
>>>>>>>>>>
>>>>>>>>>> I'll take a look at things this weekend and am sure a fix
>>>>>>>>>> will appear in between now and Monday - if there is a
>>>>>>>>>>
>>>>>> reasonable fix to be found.
>>>>>>
>>>>>>>>>> (in 4.1.3 snapshot form)
>>>>>>>>>>
>>>>>>>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com>
>>>>>>>>>>
>>>>>> <ma...@widen.com> wrote:
>>>>>>
>>>>>>>>>>
>>>>>>>>>>> I and another colleague of mine have been investigating
>>>>>>>>>>>
>>>>>> what seems
>>>>>>
>>>>>>>>>>> to be a "memory leak" in our Tapestry application for about
>>>>>>>>>>> a month since we upgraded to T4.1.2. I won't bore you
>>>>>>>>>>>
>>>>>> with the saga
>>>>>>
>>>>>>>>>>> of the last month, but I would like to present the data I've
>>>>>>>>>>> gathered and look to the list for a proposed solution. I
>>>>>>>>>>> was reading a recent thread in which Jesse said (08/09/2007):
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> "There is a map that grows as large as the system using it
>>>>>>>>>>> internally to javassist of various cached reflection
>>>>>>>>>>>
>>>>>> info - but it
>>>>>>
>>>>>>>>>>> doesn't leak in any way."
>>>>>>>>>>>
>>>>>>>>>>> This is precisely what I've found in profiling our
>>>>>>>>>>>
>>>>>> application and
>>>>>>
>>>>>>>>>>> it
>>>>>>>>>>> *appears* to be this map that is causing our applications to
>>>>>>>>>>> eventually run out of memory.
>>>>>>>>>>>
>>>>>>>>>>> The YourKit profiler shows me that, as time goes on,
>>>>>>>>>>>
>>>>>> there is an
>>>>>>
>>>>>>>>>>> instance of HiveMindClassPool that grows and grows as class
>>>>>>>>>>> instances are created. This class extends from
>>>>>>>>>>> javassist.ClassPool and is the map that Jesse is
>>>>>>>>>>>
>>>>>> talking about in
>>>>>>
>>>>>>>>>>> his quote above. And he's right, I wouldn't say that
>>>>>>>>>>>
>>>> the class
>>>>
>>>>>>>>>>> pool "leaks" either because it looks like it's designed
>>>>>>>>>>>
>>>>>> to retain
>>>>>>
>>>>>>>>>>> that memory until the class pool itself is no longer needed.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>>>>>>>
>>>>>>>>>>> "Memory consumption memo:
>>>>>>>>>>> ClassPool objects hold all the CtClasses that have been
>>>>>>>>>>>
>>>>>> created so
>>>>>>
>>>>>>>>>>> that the consistency among modified classes can be
>>>>>>>>>>>
>>>> guaranteed.
>>>>>>>>>>> Thus if a large number of CtClasses are processed, the
>>>>>>>>>>>
>>>>>> ClassPool
>>>>>>
>>>>>>>>>>> will consume a huge amount of memory. To avoid this, a
>>>>>>>>>>>
>>>>>> ClassPool
>>>>>>
>>>>>>>>>>> object should be recreated, for example, every
>>>>>>>>>>>
>>>> hundred classes
>>>>
>>>>>>>>>>> processed. Note that
>>>>>>>>>>> getDefault() is a singleton factory. Otherwise, detach() in
>>>>>>>>>>> CtClass should be used to avoid huge memory consumption. "
>>>>>>>>>>>
>>>>>>>>>>> This huge memory consumption by the ClassPool is what I was
>>>>>>>>>>> seeing. In particular it is the ClassPool that is held
>>>>>>>>>>>
>>>>>> onto by OgnlRuntime.
>>>>>>
>>>>>>>>>>> Inspecting this object in the profiler showed that it
>>>>>>>>>>>
>>>> has a map
>>>>
>>>>>>>>>>> containing about 45,000 classes. All of the keys
>>>>>>>>>>>
>>>> into this map
>>>>
>>>>>>>>>>> were things like: "ASTTest_11494aca9af" and
>>>>>>>>>>>
>>>>>> "ASTAnd_11494ace4fb"
>>>>>>>>>>> and the values are instances of javassist.CtNewClass.
>>>>>>>>>>>
>>>>>> Each entry
>>>>>>
>>>>>>>>>>> in this map looks like it retains about 1,900 bytes,
>>>>>>>>>>>
>>>>>> for a grand
>>>>>>
>>>>>>>>>>> total of about 90 MB of memory used.
>>>>>>>>>>>
>>>>>>>>>>> These numbers came from my staging deployment where I had
>>>>>>>>>>> the profiler attached, using some reflection tricks I was
>>>>>>>>>>>
>>>>>> able to look
>>>>>>
>>>>>>>>>>> at a production site and found that it had about
>>>>>>>>>>>
>>>>>> 240,000 items in
>>>>>>
>>>>>>>>>>> that class pool.. approximately 450 MB of memory.
>>>>>>>>>>>
>>>>>>>>>>> So I guess the questions in my mind are: Why are
>>>>>>>>>>>
>>>> there so many
>>>>
>>>>>>>>>>> classes in the pool? Why does the number only ever
>>>>>>>>>>>
>>>> go up? Do
>>>>
>>>>>>>>>>> those classes really need to stay in the pool forever?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>> --------------------------------------------------------------------
>>>>
>>>>>>
>>>>>>>>> - To unsubscribe, e-mail:
>>>>>> users-unsubscribe@tapestry.apache.org For
>>>>>>
>>>>>>>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>>>>
>>>> ---------------------------------------------------------------------
>>>>
>>>>>>
>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>>>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>> ---------------------------------------------------------------------
>>>>
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>>
>>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: AW: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
Okay, I will try and build with this version... lets see if it makes a
difference.
Holger Stolzenberg wrote:
> T4.1.2 is released and in the main repo:
>
> http://mvnrepository.com/artifact/org.apache.tapestry/tapestry-framework
>
> My POM:
>
> <dependency>
> <groupId>org.apache.tapestry</groupId>
> <artifactId>tapestry-framework</artifactId>
> <version>4.1.2</version>
> </dependency>
> <dependency>
> <groupId>org.apache.tapestry</groupId>
> <artifactId>tapestry-annotations</artifactId>
> <version>4.1.2</version>
> </dependency>
> <dependency>
> <groupId>org.apache.tapestry</groupId>
> <artifactId>tapestry-contrib</artifactId>
> <version>4.1.2</version>
> </dependency>
> <dependency>
> <groupId>com.ewerk</groupId>
> <artifactId>ewerk-tapestry-components</artifactId>
> <version>0.0.7</version>
> </dependency>
> <dependency>
> <groupId>com.javaforge.tapestry</groupId>
> <artifactId>tapestry-spring</artifactId>
> <version>1.0.0</version>
> <!-- exclude the old referenced version of tapestry -->
> <exclusions>
> <exclusion>
> <groupId>tapestry</groupId>
> <artifactId>tapestry</artifactId>
> </exclusion>
> <exclusion>
> <groupId>tapestry</groupId>
> <artifactId>tapestry-annotations</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
>
>
> Mit lieben Grüßen aus dem eWerk
>
> | Holger Stolzenberg
> | Softwareentwickler
> |
> | Geschäftsführer:
> | Frank Richter, Erik Wende, Hendrik Schubert
> |
> | eWerk IT GmbH
> | Markt 16
> | Leipzig 04109
> | http://www.ewerk.com
> | HRB 9065, AG Leipzig
> | Hauptniederlassung Leipzig
> |
> | fon +49.341.4 26 49-0
> | fax +49.341.4 26 49-88
> | mailto:h.stolzenberg@ewerk.com
> |
> | Support:
> | fon 0700 CALLME24 (0700 22556324)
> | fax 0700 CALLME24 (0700 22556324)
> |
> | Auskünfte und Angebote per Mail
> | sind freibleibend und unverbindlich.
>
> -----Ursprüngliche Nachricht-----
> Von: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
> Gesendet: Mittwoch, 29. August 2007 09:31
> An: Tapestry users
> Betreff: Re: Memory consumption in T4.1.2 - Hard data
>
> Correct me if I am wrong 4.1.2 is not released as yet, and not in the main repository, so how else would I get to it?
>
> Marcus.Schulte@bmw.ch wrote:
>
>> But that POM does use snapshots.
>> You shouldn't need the repository
>> http://people.apache.org/repo/m2-snapshot-repository
>> at all.
>> Probably, there are no very significant differences between the latest
>> 4.1.2 snapshot and the release. But nevertheless, for a productive
>> environment, I'd always go with a released version.
>>
>>
>>
>>> -----Original Message-----
>>> From: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
>>> Sent: Wednesday, August 29, 2007 7:45 AM
>>> To: Tapestry users
>>> Subject: Re: Memory consumption in T4.1.2 - Hard data
>>>
>>> This is my POM:
>>>
>>> <repositories>
>>> <repository>
>>> <releases>
>>> <updatePolicy>always</updatePolicy>
>>> <checksumPolicy>warn</checksumPolicy>
>>> </releases>
>>> <snapshots>
>>> <updatePolicy>always</updatePolicy>
>>> <checksumPolicy>fail</checksumPolicy>
>>> <repository>
>>> <id>apache.snapshots</id>
>>> <url>http://people.apache.org/repo/m2-snapshot-repository</url>
>>> </repository>
>>> </repositories>
>>>
>>> <dependency>
>>> <groupId>org.apache.tapestry</groupId>
>>> <artifactId>tapestry-framework</artifactId>
>>> <version>4.1.2-SNAPSHOT</version>
>>> <scope>test</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.tapestry</groupId>
>>> <artifactId>tapestry-annotations</artifactId>
>>> <version>4.1.2-SNAPSHOT</version>
>>> <scope>test</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.tapestry</groupId>
>>> <artifactId>tapestry-contrib</artifactId>
>>> <version>4.1.2-SNAPSHOT</version>
>>> <scope>test</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.tapestry</groupId>
>>> <artifactId>tapestry-portlet</artifactId>
>>> <version>4.1.2-SNAPSHOT</version>
>>> <scope>test</scope>
>>> </dependency>
>>>
>>> Marcus.Schulte@bmw.ch wrote:
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>> my configuration is as follows:
>>>>>
>>>>> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no
>>>>> luck) Tomcat 5.5.20 Debian Linux (2.6.15.28 kernel) Tapestry
>>>>> 4.1.2-SNAPSHOT with ognl 2.7
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Hi Peter, 4.1.2-SNAPSHOT is a typo, I suppose?
>>>>
>>>>
>>>>
>>>>
>>>>> regards,
>>>>> Peter
>>>>>
>>>>> Jesse Kuhnert wrote:
>>>>>
>>>>>
>>>>>
>>>>>> I'd downgrade then if I were you. Extensive profiling
>>>>>>
>>>>>>
>>> with yourkit
>>>
>>>
>>>>>> hasn't shed any new light on whatever problems others are having.
>>>>>> The OGNL classes indeed aren't being kept in the javassist
>>>>>>
>>>>>>
>>>>>>
>>>>> pool from
>>>>>
>>>>>
>>>>>
>>>>>> what I can tell.
>>>>>>
>>>>>> I have another yourkit snapshot binary to look at so we'll
>>>>>>
>>>>>>
>>>>>>
>>>>> see what that says.
>>>>>
>>>>>
>>>>>
>>>>>> I don't remember - have you filed any bug reports or reported any
>>>>>> problems? Are you getting ognl exceptions during
>>>>>>
>>>>>>
>>>>>>
>>>>> development as well
>>>>>
>>>>>
>>>>>
>>>>>> as production? Do these errors sound like
>>>>>>
>>>>>>
>>>>>>
>>>>> ummmmm....potential errors
>>>>>
>>>>>
>>>>>
>>>>>> that should be looked at further?
>>>>>>
>>>>>> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi Jessie
>>>>>>>
>>>>>>> Any progress on this? .... sorry to bug you, but I have
>>>>>>>
>>>>>>>
>>> to take a
>>>
>>>
>>>>>>> decision soon, I have two production machines that will need to
>>>>>>> upgrade or downgrade Tapestry.
>>>>>>>
>>>>>>> Best wishes,
>>>>>>> Peter
>>>>>>>
>>>>>>> Jon Oakes wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hi Bryan,
>>>>>>>>
>>>>>>>> I am a relative newbie but I was wondering if you are
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> running with
>>>>>
>>>>>
>>>>>
>>>>>>>> caching enabled or disabled? It might make sense to
>>>>>>>>
>>>>>>>>
>>> keep things
>>>
>>>
>>>>>>>> around when caching is disabled whereas I think it would
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> clearly be
>>>>>
>>>>>
>>>>>
>>>>>>>> a bug to keep things around with it disabled.
>>>>>>>>
>>>>>>>> Jon Oakes
>>>>>>>>
>>>>>>>>
>>>>>>>> Jesse Kuhnert wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hmmm...well, I don't think I like the sound of any of
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> that. I'm just
>>>>>
>>>>>
>>>>>
>>>>>>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>>>>>>
>>>>>>>>> I had thought I was doing something special with the ognl
>>>>>>>>> compilations that would cause its generated classes to
>>>>>>>>>
>>>>>>>>>
>>> not hang
>>>
>>>
>>>>>>>>> around afterwards in any pools.
>>>>>>>>>
>>>>>>>>> I'll take a look at things this weekend and am sure a fix will
>>>>>>>>> appear in between now and Monday - if there is a
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> reasonable fix to be found.
>>>>>
>>>>>
>>>>>
>>>>>>>>> (in 4.1.3 snapshot form)
>>>>>>>>>
>>>>>>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> <ma...@widen.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I and another colleague of mine have been investigating
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> what seems
>>>>>
>>>>>
>>>>>
>>>>>>>>>> to be a "memory leak" in our Tapestry application for about a
>>>>>>>>>> month since we upgraded to T4.1.2. I won't bore you
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> with the saga
>>>>>
>>>>>
>>>>>
>>>>>>>>>> of the last month, but I would like to present the data I've
>>>>>>>>>> gathered and look to the list for a proposed solution. I was
>>>>>>>>>> reading a recent thread in which Jesse said (08/09/2007):
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> "There is a map that grows as large as the system using it
>>>>>>>>>> internally to javassist of various cached reflection
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> info - but it
>>>>>
>>>>>
>>>>>
>>>>>>>>>> doesn't leak in any way."
>>>>>>>>>>
>>>>>>>>>> This is precisely what I've found in profiling our
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> application and
>>>>>
>>>>>
>>>>>
>>>>>>>>>> it
>>>>>>>>>> *appears* to be this map that is causing our applications to
>>>>>>>>>> eventually run out of memory.
>>>>>>>>>>
>>>>>>>>>> The YourKit profiler shows me that, as time goes on,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> there is an
>>>>>
>>>>>
>>>>>
>>>>>>>>>> instance of HiveMindClassPool that grows and grows as class
>>>>>>>>>> instances are created. This class extends from
>>>>>>>>>> javassist.ClassPool and is the map that Jesse is
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> talking about in
>>>>>
>>>>>
>>>>>
>>>>>>>>>> his quote above. And he's right, I wouldn't say that
>>>>>>>>>>
>>>>>>>>>>
>>> the class
>>>
>>>
>>>>>>>>>> pool "leaks" either because it looks like it's designed
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> to retain
>>>>>
>>>>>
>>>>>
>>>>>>>>>> that memory until the class pool itself is no longer needed.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>>>>>>
>>>>>>>>>> "Memory consumption memo:
>>>>>>>>>> ClassPool objects hold all the CtClasses that have been
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> created so
>>>>>
>>>>>
>>>>>
>>>>>>>>>> that the consistency among modified classes can be
>>>>>>>>>>
>>>>>>>>>>
>>> guaranteed.
>>>
>>>
>>>>>>>>>> Thus if a large number of CtClasses are processed, the
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> ClassPool
>>>>>
>>>>>
>>>>>
>>>>>>>>>> will consume a huge amount of memory. To avoid this, a
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> ClassPool
>>>>>
>>>>>
>>>>>
>>>>>>>>>> object should be recreated, for example, every
>>>>>>>>>>
>>>>>>>>>>
>>> hundred classes
>>>
>>>
>>>>>>>>>> processed. Note that
>>>>>>>>>> getDefault() is a singleton factory. Otherwise, detach() in
>>>>>>>>>> CtClass should be used to avoid huge memory consumption. "
>>>>>>>>>>
>>>>>>>>>> This huge memory consumption by the ClassPool is what I was
>>>>>>>>>> seeing. In particular it is the ClassPool that is held
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> onto by OgnlRuntime.
>>>>>
>>>>>
>>>>>
>>>>>>>>>> Inspecting this object in the profiler showed that it
>>>>>>>>>>
>>>>>>>>>>
>>> has a map
>>>
>>>
>>>>>>>>>> containing about 45,000 classes. All of the keys
>>>>>>>>>>
>>>>>>>>>>
>>> into this map
>>>
>>>
>>>>>>>>>> were things like: "ASTTest_11494aca9af" and
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> "ASTAnd_11494ace4fb"
>>>>>
>>>>>
>>>>>
>>>>>>>>>> and the values are instances of javassist.CtNewClass.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> Each entry
>>>>>
>>>>>
>>>>>
>>>>>>>>>> in this map looks like it retains about 1,900 bytes,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> for a grand
>>>>>
>>>>>
>>>>>
>>>>>>>>>> total of about 90 MB of memory used.
>>>>>>>>>>
>>>>>>>>>> These numbers came from my staging deployment where I had the
>>>>>>>>>> profiler attached, using some reflection tricks I was
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> able to look
>>>>>
>>>>>
>>>>>
>>>>>>>>>> at a production site and found that it had about
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> 240,000 items in
>>>>>
>>>>>
>>>>>
>>>>>>>>>> that class pool.. approximately 450 MB of memory.
>>>>>>>>>>
>>>>>>>>>> So I guess the questions in my mind are: Why are
>>>>>>>>>>
>>>>>>>>>>
>>> there so many
>>>
>>>
>>>>>>>>>> classes in the pool? Why does the number only ever
>>>>>>>>>>
>>>>>>>>>>
>>> go up? Do
>>>
>>>
>>>>>>>>>> those classes really need to stay in the pool forever?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>> --------------------------------------------------------------------
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>>>> - To unsubscribe, e-mail:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> users-unsubscribe@tapestry.apache.org For
>>>>>
>>>>>
>>>>>
>>>>>>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>> ---------------------------------------------------------------------
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>> ---------------------------------------------------------------------
>>>
>>>
>>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>>
>>>
>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>
>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
AW: Memory consumption in T4.1.2 - Hard data
Posted by Holger Stolzenberg <h....@ewerk.com>.
T4.1.2 is released and in the main repo:
http://mvnrepository.com/artifact/org.apache.tapestry/tapestry-framework
My POM:
<dependency>
<groupId>org.apache.tapestry</groupId>
<artifactId>tapestry-framework</artifactId>
<version>4.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.tapestry</groupId>
<artifactId>tapestry-annotations</artifactId>
<version>4.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.tapestry</groupId>
<artifactId>tapestry-contrib</artifactId>
<version>4.1.2</version>
</dependency>
<dependency>
<groupId>com.ewerk</groupId>
<artifactId>ewerk-tapestry-components</artifactId>
<version>0.0.7</version>
</dependency>
<dependency>
<groupId>com.javaforge.tapestry</groupId>
<artifactId>tapestry-spring</artifactId>
<version>1.0.0</version>
<!-- exclude the old referenced version of tapestry -->
<exclusions>
<exclusion>
<groupId>tapestry</groupId>
<artifactId>tapestry</artifactId>
</exclusion>
<exclusion>
<groupId>tapestry</groupId>
<artifactId>tapestry-annotations</artifactId>
</exclusion>
</exclusions>
</dependency>
Mit lieben Grüßen aus dem eWerk
| Holger Stolzenberg
| Softwareentwickler
|
| Geschäftsführer:
| Frank Richter, Erik Wende, Hendrik Schubert
|
| eWerk IT GmbH
| Markt 16
| Leipzig 04109
| http://www.ewerk.com
| HRB 9065, AG Leipzig
| Hauptniederlassung Leipzig
|
| fon +49.341.4 26 49-0
| fax +49.341.4 26 49-88
| mailto:h.stolzenberg@ewerk.com
|
| Support:
| fon 0700 CALLME24 (0700 22556324)
| fax 0700 CALLME24 (0700 22556324)
|
| Auskünfte und Angebote per Mail
| sind freibleibend und unverbindlich.
-----Ursprüngliche Nachricht-----
Von: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
Gesendet: Mittwoch, 29. August 2007 09:31
An: Tapestry users
Betreff: Re: Memory consumption in T4.1.2 - Hard data
Correct me if I am wrong 4.1.2 is not released as yet, and not in the main repository, so how else would I get to it?
Marcus.Schulte@bmw.ch wrote:
> But that POM does use snapshots.
> You shouldn't need the repository
> http://people.apache.org/repo/m2-snapshot-repository
> at all.
> Probably, there are no very significant differences between the latest
> 4.1.2 snapshot and the release. But nevertheless, for a productive
> environment, I'd always go with a released version.
>
>
>> -----Original Message-----
>> From: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
>> Sent: Wednesday, August 29, 2007 7:45 AM
>> To: Tapestry users
>> Subject: Re: Memory consumption in T4.1.2 - Hard data
>>
>> This is my POM:
>>
>> <repositories>
>> <repository>
>> <releases>
>> <updatePolicy>always</updatePolicy>
>> <checksumPolicy>warn</checksumPolicy>
>> </releases>
>> <snapshots>
>> <updatePolicy>always</updatePolicy>
>> <checksumPolicy>fail</checksumPolicy>
>> <repository>
>> <id>apache.snapshots</id>
>> <url>http://people.apache.org/repo/m2-snapshot-repository</url>
>> </repository>
>> </repositories>
>>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-framework</artifactId>
>> <version>4.1.2-SNAPSHOT</version>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-annotations</artifactId>
>> <version>4.1.2-SNAPSHOT</version>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-contrib</artifactId>
>> <version>4.1.2-SNAPSHOT</version>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-portlet</artifactId>
>> <version>4.1.2-SNAPSHOT</version>
>> <scope>test</scope>
>> </dependency>
>>
>> Marcus.Schulte@bmw.ch wrote:
>>
>>>
>>>
>>>
>>>> my configuration is as follows:
>>>>
>>>> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no
>>>> luck) Tomcat 5.5.20 Debian Linux (2.6.15.28 kernel) Tapestry
>>>> 4.1.2-SNAPSHOT with ognl 2.7
>>>>
>>>>
>>>>
>>> Hi Peter, 4.1.2-SNAPSHOT is a typo, I suppose?
>>>
>>>
>>>
>>>> regards,
>>>> Peter
>>>>
>>>> Jesse Kuhnert wrote:
>>>>
>>>>
>>>>> I'd downgrade then if I were you. Extensive profiling
>>>>>
>> with yourkit
>>
>>>>> hasn't shed any new light on whatever problems others are having.
>>>>> The OGNL classes indeed aren't being kept in the javassist
>>>>>
>>>>>
>>>> pool from
>>>>
>>>>
>>>>> what I can tell.
>>>>>
>>>>> I have another yourkit snapshot binary to look at so we'll
>>>>>
>>>>>
>>>> see what that says.
>>>>
>>>>
>>>>> I don't remember - have you filed any bug reports or reported any
>>>>> problems? Are you getting ognl exceptions during
>>>>>
>>>>>
>>>> development as well
>>>>
>>>>
>>>>> as production? Do these errors sound like
>>>>>
>>>>>
>>>> ummmmm....potential errors
>>>>
>>>>
>>>>> that should be looked at further?
>>>>>
>>>>> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Hi Jessie
>>>>>>
>>>>>> Any progress on this? .... sorry to bug you, but I have
>>>>>>
>> to take a
>>
>>>>>> decision soon, I have two production machines that will need to
>>>>>> upgrade or downgrade Tapestry.
>>>>>>
>>>>>> Best wishes,
>>>>>> Peter
>>>>>>
>>>>>> Jon Oakes wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi Bryan,
>>>>>>>
>>>>>>> I am a relative newbie but I was wondering if you are
>>>>>>>
>>>>>>>
>>>> running with
>>>>
>>>>
>>>>>>> caching enabled or disabled? It might make sense to
>>>>>>>
>> keep things
>>
>>>>>>> around when caching is disabled whereas I think it would
>>>>>>>
>>>>>>>
>>>> clearly be
>>>>
>>>>
>>>>>>> a bug to keep things around with it disabled.
>>>>>>>
>>>>>>> Jon Oakes
>>>>>>>
>>>>>>>
>>>>>>> Jesse Kuhnert wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hmmm...well, I don't think I like the sound of any of
>>>>>>>>
>>>>>>>>
>>>> that. I'm just
>>>>
>>>>
>>>>>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>>>>>
>>>>>>>> I had thought I was doing something special with the ognl
>>>>>>>> compilations that would cause its generated classes to
>>>>>>>>
>> not hang
>>
>>>>>>>> around afterwards in any pools.
>>>>>>>>
>>>>>>>> I'll take a look at things this weekend and am sure a fix will
>>>>>>>> appear in between now and Monday - if there is a
>>>>>>>>
>>>>>>>>
>>>> reasonable fix to be found.
>>>>
>>>>
>>>>>>>> (in 4.1.3 snapshot form)
>>>>>>>>
>>>>>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com>
>>>>>>>>
>>>>>>>>
>>>> <ma...@widen.com> wrote:
>>>>
>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I and another colleague of mine have been investigating
>>>>>>>>>
>>>>>>>>>
>>>> what seems
>>>>
>>>>
>>>>>>>>> to be a "memory leak" in our Tapestry application for about a
>>>>>>>>> month since we upgraded to T4.1.2. I won't bore you
>>>>>>>>>
>>>>>>>>>
>>>> with the saga
>>>>
>>>>
>>>>>>>>> of the last month, but I would like to present the data I've
>>>>>>>>> gathered and look to the list for a proposed solution. I was
>>>>>>>>> reading a recent thread in which Jesse said (08/09/2007):
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> "There is a map that grows as large as the system using it
>>>>>>>>> internally to javassist of various cached reflection
>>>>>>>>>
>>>>>>>>>
>>>> info - but it
>>>>
>>>>
>>>>>>>>> doesn't leak in any way."
>>>>>>>>>
>>>>>>>>> This is precisely what I've found in profiling our
>>>>>>>>>
>>>>>>>>>
>>>> application and
>>>>
>>>>
>>>>>>>>> it
>>>>>>>>> *appears* to be this map that is causing our applications to
>>>>>>>>> eventually run out of memory.
>>>>>>>>>
>>>>>>>>> The YourKit profiler shows me that, as time goes on,
>>>>>>>>>
>>>>>>>>>
>>>> there is an
>>>>
>>>>
>>>>>>>>> instance of HiveMindClassPool that grows and grows as class
>>>>>>>>> instances are created. This class extends from
>>>>>>>>> javassist.ClassPool and is the map that Jesse is
>>>>>>>>>
>>>>>>>>>
>>>> talking about in
>>>>
>>>>
>>>>>>>>> his quote above. And he's right, I wouldn't say that
>>>>>>>>>
>> the class
>>
>>>>>>>>> pool "leaks" either because it looks like it's designed
>>>>>>>>>
>>>>>>>>>
>>>> to retain
>>>>
>>>>
>>>>>>>>> that memory until the class pool itself is no longer needed.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>>>>>
>>>>>>>>> "Memory consumption memo:
>>>>>>>>> ClassPool objects hold all the CtClasses that have been
>>>>>>>>>
>>>>>>>>>
>>>> created so
>>>>
>>>>
>>>>>>>>> that the consistency among modified classes can be
>>>>>>>>>
>> guaranteed.
>>
>>>>>>>>> Thus if a large number of CtClasses are processed, the
>>>>>>>>>
>>>>>>>>>
>>>> ClassPool
>>>>
>>>>
>>>>>>>>> will consume a huge amount of memory. To avoid this, a
>>>>>>>>>
>>>>>>>>>
>>>> ClassPool
>>>>
>>>>
>>>>>>>>> object should be recreated, for example, every
>>>>>>>>>
>> hundred classes
>>
>>>>>>>>> processed. Note that
>>>>>>>>> getDefault() is a singleton factory. Otherwise, detach() in
>>>>>>>>> CtClass should be used to avoid huge memory consumption. "
>>>>>>>>>
>>>>>>>>> This huge memory consumption by the ClassPool is what I was
>>>>>>>>> seeing. In particular it is the ClassPool that is held
>>>>>>>>>
>>>>>>>>>
>>>> onto by OgnlRuntime.
>>>>
>>>>
>>>>>>>>> Inspecting this object in the profiler showed that it
>>>>>>>>>
>> has a map
>>
>>>>>>>>> containing about 45,000 classes. All of the keys
>>>>>>>>>
>> into this map
>>
>>>>>>>>> were things like: "ASTTest_11494aca9af" and
>>>>>>>>>
>>>>>>>>>
>>>> "ASTAnd_11494ace4fb"
>>>>
>>>>
>>>>>>>>> and the values are instances of javassist.CtNewClass.
>>>>>>>>>
>>>>>>>>>
>>>> Each entry
>>>>
>>>>
>>>>>>>>> in this map looks like it retains about 1,900 bytes,
>>>>>>>>>
>>>>>>>>>
>>>> for a grand
>>>>
>>>>
>>>>>>>>> total of about 90 MB of memory used.
>>>>>>>>>
>>>>>>>>> These numbers came from my staging deployment where I had the
>>>>>>>>> profiler attached, using some reflection tricks I was
>>>>>>>>>
>>>>>>>>>
>>>> able to look
>>>>
>>>>
>>>>>>>>> at a production site and found that it had about
>>>>>>>>>
>>>>>>>>>
>>>> 240,000 items in
>>>>
>>>>
>>>>>>>>> that class pool.. approximately 450 MB of memory.
>>>>>>>>>
>>>>>>>>> So I guess the questions in my mind are: Why are
>>>>>>>>>
>> there so many
>>
>>>>>>>>> classes in the pool? Why does the number only ever
>>>>>>>>>
>> go up? Do
>>
>>>>>>>>> those classes really need to stay in the pool forever?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>> --------------------------------------------------------------------
>>
>>>>
>>>>
>>>>>>> - To unsubscribe, e-mail:
>>>>>>>
>>>>>>>
>>>> users-unsubscribe@tapestry.apache.org For
>>>>
>>>>
>>>>>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>
>>>>
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>
>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>
>>>>
>>>>
>>>>
>>>
>> ---------------------------------------------------------------------
>>
>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
Correct me if I am wrong 4.1.2 is not released as yet, and not in the
main repository, so how else would I get to it?
Marcus.Schulte@bmw.ch wrote:
> But that POM does use snapshots.
> You shouldn't need the repository
> http://people.apache.org/repo/m2-snapshot-repository
> at all.
> Probably, there are no very significant differences between the latest
> 4.1.2 snapshot and the release. But nevertheless, for a productive
> environment, I'd always go with a released version.
>
>
>> -----Original Message-----
>> From: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
>> Sent: Wednesday, August 29, 2007 7:45 AM
>> To: Tapestry users
>> Subject: Re: Memory consumption in T4.1.2 - Hard data
>>
>> This is my POM:
>>
>> <repositories>
>> <repository>
>> <releases>
>> <updatePolicy>always</updatePolicy>
>> <checksumPolicy>warn</checksumPolicy>
>> </releases>
>> <snapshots>
>> <updatePolicy>always</updatePolicy>
>> <checksumPolicy>fail</checksumPolicy>
>> <repository>
>> <id>apache.snapshots</id>
>> <url>http://people.apache.org/repo/m2-snapshot-repository</url>
>> </repository>
>> </repositories>
>>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-framework</artifactId>
>> <version>4.1.2-SNAPSHOT</version>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-annotations</artifactId>
>> <version>4.1.2-SNAPSHOT</version>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-contrib</artifactId>
>> <version>4.1.2-SNAPSHOT</version>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.tapestry</groupId>
>> <artifactId>tapestry-portlet</artifactId>
>> <version>4.1.2-SNAPSHOT</version>
>> <scope>test</scope>
>> </dependency>
>>
>> Marcus.Schulte@bmw.ch wrote:
>>
>>>
>>>
>>>
>>>> my configuration is as follows:
>>>>
>>>> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no
>>>> luck) Tomcat 5.5.20 Debian Linux (2.6.15.28 kernel) Tapestry
>>>> 4.1.2-SNAPSHOT with ognl 2.7
>>>>
>>>>
>>>>
>>> Hi Peter, 4.1.2-SNAPSHOT is a typo, I suppose?
>>>
>>>
>>>
>>>> regards,
>>>> Peter
>>>>
>>>> Jesse Kuhnert wrote:
>>>>
>>>>
>>>>> I'd downgrade then if I were you. Extensive profiling
>>>>>
>> with yourkit
>>
>>>>> hasn't shed any new light on whatever problems others are having.
>>>>> The OGNL classes indeed aren't being kept in the javassist
>>>>>
>>>>>
>>>> pool from
>>>>
>>>>
>>>>> what I can tell.
>>>>>
>>>>> I have another yourkit snapshot binary to look at so we'll
>>>>>
>>>>>
>>>> see what that says.
>>>>
>>>>
>>>>> I don't remember - have you filed any bug reports or reported any
>>>>> problems? Are you getting ognl exceptions during
>>>>>
>>>>>
>>>> development as well
>>>>
>>>>
>>>>> as production? Do these errors sound like
>>>>>
>>>>>
>>>> ummmmm....potential errors
>>>>
>>>>
>>>>> that should be looked at further?
>>>>>
>>>>> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Hi Jessie
>>>>>>
>>>>>> Any progress on this? .... sorry to bug you, but I have
>>>>>>
>> to take a
>>
>>>>>> decision soon, I have two production machines that will need to
>>>>>> upgrade or downgrade Tapestry.
>>>>>>
>>>>>> Best wishes,
>>>>>> Peter
>>>>>>
>>>>>> Jon Oakes wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi Bryan,
>>>>>>>
>>>>>>> I am a relative newbie but I was wondering if you are
>>>>>>>
>>>>>>>
>>>> running with
>>>>
>>>>
>>>>>>> caching enabled or disabled? It might make sense to
>>>>>>>
>> keep things
>>
>>>>>>> around when caching is disabled whereas I think it would
>>>>>>>
>>>>>>>
>>>> clearly be
>>>>
>>>>
>>>>>>> a bug to keep things around with it disabled.
>>>>>>>
>>>>>>> Jon Oakes
>>>>>>>
>>>>>>>
>>>>>>> Jesse Kuhnert wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hmmm...well, I don't think I like the sound of any of
>>>>>>>>
>>>>>>>>
>>>> that. I'm just
>>>>
>>>>
>>>>>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>>>>>
>>>>>>>> I had thought I was doing something special with the ognl
>>>>>>>> compilations that would cause its generated classes to
>>>>>>>>
>> not hang
>>
>>>>>>>> around afterwards in any pools.
>>>>>>>>
>>>>>>>> I'll take a look at things this weekend and am sure a fix will
>>>>>>>> appear in between now and Monday - if there is a
>>>>>>>>
>>>>>>>>
>>>> reasonable fix to be found.
>>>>
>>>>
>>>>>>>> (in 4.1.3 snapshot form)
>>>>>>>>
>>>>>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com>
>>>>>>>>
>>>>>>>>
>>>> <ma...@widen.com> wrote:
>>>>
>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I and another colleague of mine have been investigating
>>>>>>>>>
>>>>>>>>>
>>>> what seems
>>>>
>>>>
>>>>>>>>> to be a "memory leak" in our Tapestry application for about a
>>>>>>>>> month since we upgraded to T4.1.2. I won't bore you
>>>>>>>>>
>>>>>>>>>
>>>> with the saga
>>>>
>>>>
>>>>>>>>> of the last month, but I would like to present the data I've
>>>>>>>>> gathered and look to the list for a proposed solution. I was
>>>>>>>>> reading a recent thread in which Jesse said (08/09/2007):
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> "There is a map that grows as large as the system using it
>>>>>>>>> internally to javassist of various cached reflection
>>>>>>>>>
>>>>>>>>>
>>>> info - but it
>>>>
>>>>
>>>>>>>>> doesn't leak in any way."
>>>>>>>>>
>>>>>>>>> This is precisely what I've found in profiling our
>>>>>>>>>
>>>>>>>>>
>>>> application and
>>>>
>>>>
>>>>>>>>> it
>>>>>>>>> *appears* to be this map that is causing our applications to
>>>>>>>>> eventually run out of memory.
>>>>>>>>>
>>>>>>>>> The YourKit profiler shows me that, as time goes on,
>>>>>>>>>
>>>>>>>>>
>>>> there is an
>>>>
>>>>
>>>>>>>>> instance of HiveMindClassPool that grows and grows as class
>>>>>>>>> instances are created. This class extends from
>>>>>>>>> javassist.ClassPool and is the map that Jesse is
>>>>>>>>>
>>>>>>>>>
>>>> talking about in
>>>>
>>>>
>>>>>>>>> his quote above. And he's right, I wouldn't say that
>>>>>>>>>
>> the class
>>
>>>>>>>>> pool "leaks" either because it looks like it's designed
>>>>>>>>>
>>>>>>>>>
>>>> to retain
>>>>
>>>>
>>>>>>>>> that memory until the class pool itself is no longer needed.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>>>>>
>>>>>>>>> "Memory consumption memo:
>>>>>>>>> ClassPool objects hold all the CtClasses that have been
>>>>>>>>>
>>>>>>>>>
>>>> created so
>>>>
>>>>
>>>>>>>>> that the consistency among modified classes can be
>>>>>>>>>
>> guaranteed.
>>
>>>>>>>>> Thus if a large number of CtClasses are processed, the
>>>>>>>>>
>>>>>>>>>
>>>> ClassPool
>>>>
>>>>
>>>>>>>>> will consume a huge amount of memory. To avoid this, a
>>>>>>>>>
>>>>>>>>>
>>>> ClassPool
>>>>
>>>>
>>>>>>>>> object should be recreated, for example, every
>>>>>>>>>
>> hundred classes
>>
>>>>>>>>> processed. Note that
>>>>>>>>> getDefault() is a singleton factory. Otherwise, detach() in
>>>>>>>>> CtClass should be used to avoid huge memory consumption. "
>>>>>>>>>
>>>>>>>>> This huge memory consumption by the ClassPool is what I was
>>>>>>>>> seeing. In particular it is the ClassPool that is held
>>>>>>>>>
>>>>>>>>>
>>>> onto by OgnlRuntime.
>>>>
>>>>
>>>>>>>>> Inspecting this object in the profiler showed that it
>>>>>>>>>
>> has a map
>>
>>>>>>>>> containing about 45,000 classes. All of the keys
>>>>>>>>>
>> into this map
>>
>>>>>>>>> were things like: "ASTTest_11494aca9af" and
>>>>>>>>>
>>>>>>>>>
>>>> "ASTAnd_11494ace4fb"
>>>>
>>>>
>>>>>>>>> and the values are instances of javassist.CtNewClass.
>>>>>>>>>
>>>>>>>>>
>>>> Each entry
>>>>
>>>>
>>>>>>>>> in this map looks like it retains about 1,900 bytes,
>>>>>>>>>
>>>>>>>>>
>>>> for a grand
>>>>
>>>>
>>>>>>>>> total of about 90 MB of memory used.
>>>>>>>>>
>>>>>>>>> These numbers came from my staging deployment where I had the
>>>>>>>>> profiler attached, using some reflection tricks I was
>>>>>>>>>
>>>>>>>>>
>>>> able to look
>>>>
>>>>
>>>>>>>>> at a production site and found that it had about
>>>>>>>>>
>>>>>>>>>
>>>> 240,000 items in
>>>>
>>>>
>>>>>>>>> that class pool.. approximately 450 MB of memory.
>>>>>>>>>
>>>>>>>>> So I guess the questions in my mind are: Why are
>>>>>>>>>
>> there so many
>>
>>>>>>>>> classes in the pool? Why does the number only ever
>>>>>>>>>
>> go up? Do
>>
>>>>>>>>> those classes really need to stay in the pool forever?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>> --------------------------------------------------------------------
>>
>>>>
>>>>
>>>>>>> - To unsubscribe, e-mail:
>>>>>>>
>>>>>>>
>>>> users-unsubscribe@tapestry.apache.org For
>>>>
>>>>
>>>>>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>
>>>>
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>
>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>
>>>>
>>>>
>>>>
>>>
>> ---------------------------------------------------------------------
>>
>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
RE: Memory consumption in T4.1.2 - Hard data
Posted by Ma...@bmw.ch.
But that POM does use snapshots.
You shouldn't need the repository
http://people.apache.org/repo/m2-snapshot-repository
at all.
Probably, there are no very significant differences between the latest
4.1.2 snapshot and the release. But nevertheless, for a productive
environment, I'd always go with a released version.
> -----Original Message-----
> From: Peter Stavrinides [mailto:p.stavrinides@albourne.com]
> Sent: Wednesday, August 29, 2007 7:45 AM
> To: Tapestry users
> Subject: Re: Memory consumption in T4.1.2 - Hard data
>
> This is my POM:
>
> <repositories>
> <repository>
> <releases>
> <updatePolicy>always</updatePolicy>
> <checksumPolicy>warn</checksumPolicy>
> </releases>
> <snapshots>
> <updatePolicy>always</updatePolicy>
> <checksumPolicy>fail</checksumPolicy>
> <repository>
> <id>apache.snapshots</id>
> <url>http://people.apache.org/repo/m2-snapshot-repository</url>
> </repository>
> </repositories>
>
> <dependency>
> <groupId>org.apache.tapestry</groupId>
> <artifactId>tapestry-framework</artifactId>
> <version>4.1.2-SNAPSHOT</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.tapestry</groupId>
> <artifactId>tapestry-annotations</artifactId>
> <version>4.1.2-SNAPSHOT</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.tapestry</groupId>
> <artifactId>tapestry-contrib</artifactId>
> <version>4.1.2-SNAPSHOT</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.tapestry</groupId>
> <artifactId>tapestry-portlet</artifactId>
> <version>4.1.2-SNAPSHOT</version>
> <scope>test</scope>
> </dependency>
>
> Marcus.Schulte@bmw.ch wrote:
> >
> >
> >> my configuration is as follows:
> >>
> >> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no
> >> luck) Tomcat 5.5.20 Debian Linux (2.6.15.28 kernel) Tapestry
> >> 4.1.2-SNAPSHOT with ognl 2.7
> >>
> >>
> >
> > Hi Peter, 4.1.2-SNAPSHOT is a typo, I suppose?
> >
> >
> >> regards,
> >> Peter
> >>
> >> Jesse Kuhnert wrote:
> >>
> >>> I'd downgrade then if I were you. Extensive profiling
> with yourkit
> >>> hasn't shed any new light on whatever problems others are having.
> >>> The OGNL classes indeed aren't being kept in the javassist
> >>>
> >> pool from
> >>
> >>> what I can tell.
> >>>
> >>> I have another yourkit snapshot binary to look at so we'll
> >>>
> >> see what that says.
> >>
> >>> I don't remember - have you filed any bug reports or reported any
> >>> problems? Are you getting ognl exceptions during
> >>>
> >> development as well
> >>
> >>> as production? Do these errors sound like
> >>>
> >> ummmmm....potential errors
> >>
> >>> that should be looked at further?
> >>>
> >>> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> >>>
> >>>
> >>>> Hi Jessie
> >>>>
> >>>> Any progress on this? .... sorry to bug you, but I have
> to take a
> >>>> decision soon, I have two production machines that will need to
> >>>> upgrade or downgrade Tapestry.
> >>>>
> >>>> Best wishes,
> >>>> Peter
> >>>>
> >>>> Jon Oakes wrote:
> >>>>
> >>>>
> >>>>> Hi Bryan,
> >>>>>
> >>>>> I am a relative newbie but I was wondering if you are
> >>>>>
> >> running with
> >>
> >>>>> caching enabled or disabled? It might make sense to
> keep things
> >>>>> around when caching is disabled whereas I think it would
> >>>>>
> >> clearly be
> >>
> >>>>> a bug to keep things around with it disabled.
> >>>>>
> >>>>> Jon Oakes
> >>>>>
> >>>>>
> >>>>> Jesse Kuhnert wrote:
> >>>>>
> >>>>>
> >>>>>> Hmmm...well, I don't think I like the sound of any of
> >>>>>>
> >> that. I'm just
> >>
> >>>>>> going to pretend this problem doesn't exist. (just kidding)
> >>>>>>
> >>>>>> I had thought I was doing something special with the ognl
> >>>>>> compilations that would cause its generated classes to
> not hang
> >>>>>> around afterwards in any pools.
> >>>>>>
> >>>>>> I'll take a look at things this weekend and am sure a fix will
> >>>>>> appear in between now and Monday - if there is a
> >>>>>>
> >> reasonable fix to be found.
> >>
> >>>>>> (in 4.1.3 snapshot form)
> >>>>>>
> >>>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com>
> >>>>>>
> >> <ma...@widen.com> wrote:
> >>
> >>>>>>
> >>>>>>
> >>>>>>> I and another colleague of mine have been investigating
> >>>>>>>
> >> what seems
> >>
> >>>>>>> to be a "memory leak" in our Tapestry application for about a
> >>>>>>> month since we upgraded to T4.1.2. I won't bore you
> >>>>>>>
> >> with the saga
> >>
> >>>>>>> of the last month, but I would like to present the data I've
> >>>>>>> gathered and look to the list for a proposed solution. I was
> >>>>>>> reading a recent thread in which Jesse said (08/09/2007):
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> "There is a map that grows as large as the system using it
> >>>>>>> internally to javassist of various cached reflection
> >>>>>>>
> >> info - but it
> >>
> >>>>>>> doesn't leak in any way."
> >>>>>>>
> >>>>>>> This is precisely what I've found in profiling our
> >>>>>>>
> >> application and
> >>
> >>>>>>> it
> >>>>>>> *appears* to be this map that is causing our applications to
> >>>>>>> eventually run out of memory.
> >>>>>>>
> >>>>>>> The YourKit profiler shows me that, as time goes on,
> >>>>>>>
> >> there is an
> >>
> >>>>>>> instance of HiveMindClassPool that grows and grows as class
> >>>>>>> instances are created. This class extends from
> >>>>>>> javassist.ClassPool and is the map that Jesse is
> >>>>>>>
> >> talking about in
> >>
> >>>>>>> his quote above. And he's right, I wouldn't say that
> the class
> >>>>>>> pool "leaks" either because it looks like it's designed
> >>>>>>>
> >> to retain
> >>
> >>>>>>> that memory until the class pool itself is no longer needed.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Take this quote from the javassist.ClassPool javadocs:
> >>>>>>>
> >>>>>>> "Memory consumption memo:
> >>>>>>> ClassPool objects hold all the CtClasses that have been
> >>>>>>>
> >> created so
> >>
> >>>>>>> that the consistency among modified classes can be
> guaranteed.
> >>>>>>> Thus if a large number of CtClasses are processed, the
> >>>>>>>
> >> ClassPool
> >>
> >>>>>>> will consume a huge amount of memory. To avoid this, a
> >>>>>>>
> >> ClassPool
> >>
> >>>>>>> object should be recreated, for example, every
> hundred classes
> >>>>>>> processed. Note that
> >>>>>>> getDefault() is a singleton factory. Otherwise, detach() in
> >>>>>>> CtClass should be used to avoid huge memory consumption. "
> >>>>>>>
> >>>>>>> This huge memory consumption by the ClassPool is what I was
> >>>>>>> seeing. In particular it is the ClassPool that is held
> >>>>>>>
> >> onto by OgnlRuntime.
> >>
> >>>>>>> Inspecting this object in the profiler showed that it
> has a map
> >>>>>>> containing about 45,000 classes. All of the keys
> into this map
> >>>>>>> were things like: "ASTTest_11494aca9af" and
> >>>>>>>
> >> "ASTAnd_11494ace4fb"
> >>
> >>>>>>> and the values are instances of javassist.CtNewClass.
> >>>>>>>
> >> Each entry
> >>
> >>>>>>> in this map looks like it retains about 1,900 bytes,
> >>>>>>>
> >> for a grand
> >>
> >>>>>>> total of about 90 MB of memory used.
> >>>>>>>
> >>>>>>> These numbers came from my staging deployment where I had the
> >>>>>>> profiler attached, using some reflection tricks I was
> >>>>>>>
> >> able to look
> >>
> >>>>>>> at a production site and found that it had about
> >>>>>>>
> >> 240,000 items in
> >>
> >>>>>>> that class pool.. approximately 450 MB of memory.
> >>>>>>>
> >>>>>>> So I guess the questions in my mind are: Why are
> there so many
> >>>>>>> classes in the pool? Why does the number only ever
> go up? Do
> >>>>>>> those classes really need to stay in the pool forever?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>
> --------------------------------------------------------------------
> >>
> >>>>> - To unsubscribe, e-mail:
> >>>>>
> >> users-unsubscribe@tapestry.apache.org For
> >>
> >>>>> additional commands, e-mail: users-help@tapestry.apache.org
> >>>>>
> >>>>>
> >>
> ---------------------------------------------------------------------
> >>
> >>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> >>>> For additional commands, e-mail: users-help@tapestry.apache.org
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> >> For additional commands, e-mail: users-help@tapestry.apache.org
> >>
> >>
> >>
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> > For additional commands, e-mail: users-help@tapestry.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
This is my POM:
<repositories>
<repository>
<releases>
<updatePolicy>always</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</releases>
<snapshots>
<updatePolicy>always</updatePolicy>
<checksumPolicy>fail</checksumPolicy>
<repository>
<id>apache.snapshots</id>
<url>http://people.apache.org/repo/m2-snapshot-repository</url>
</repository>
</repositories>
<dependency>
<groupId>org.apache.tapestry</groupId>
<artifactId>tapestry-framework</artifactId>
<version>4.1.2-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.tapestry</groupId>
<artifactId>tapestry-annotations</artifactId>
<version>4.1.2-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.tapestry</groupId>
<artifactId>tapestry-contrib</artifactId>
<version>4.1.2-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.tapestry</groupId>
<artifactId>tapestry-portlet</artifactId>
<version>4.1.2-SNAPSHOT</version>
<scope>test</scope>
</dependency>
Marcus.Schulte@bmw.ch wrote:
>
>
>> my configuration is as follows:
>>
>> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no
>> luck) Tomcat 5.5.20 Debian Linux (2.6.15.28 kernel) Tapestry
>> 4.1.2-SNAPSHOT with ognl 2.7
>>
>>
>
> Hi Peter, 4.1.2-SNAPSHOT is a typo, I suppose?
>
>
>> regards,
>> Peter
>>
>> Jesse Kuhnert wrote:
>>
>>> I'd downgrade then if I were you. Extensive profiling with yourkit
>>> hasn't shed any new light on whatever problems others are having.
>>> The OGNL classes indeed aren't being kept in the javassist
>>>
>> pool from
>>
>>> what I can tell.
>>>
>>> I have another yourkit snapshot binary to look at so we'll
>>>
>> see what that says.
>>
>>> I don't remember - have you filed any bug reports or reported any
>>> problems? Are you getting ognl exceptions during
>>>
>> development as well
>>
>>> as production? Do these errors sound like
>>>
>> ummmmm....potential errors
>>
>>> that should be looked at further?
>>>
>>> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>>>
>>>
>>>> Hi Jessie
>>>>
>>>> Any progress on this? .... sorry to bug you, but I have to take a
>>>> decision soon, I have two production machines that will need to
>>>> upgrade or downgrade Tapestry.
>>>>
>>>> Best wishes,
>>>> Peter
>>>>
>>>> Jon Oakes wrote:
>>>>
>>>>
>>>>> Hi Bryan,
>>>>>
>>>>> I am a relative newbie but I was wondering if you are
>>>>>
>> running with
>>
>>>>> caching enabled or disabled? It might make sense to keep things
>>>>> around when caching is disabled whereas I think it would
>>>>>
>> clearly be
>>
>>>>> a bug to keep things around with it disabled.
>>>>>
>>>>> Jon Oakes
>>>>>
>>>>>
>>>>> Jesse Kuhnert wrote:
>>>>>
>>>>>
>>>>>> Hmmm...well, I don't think I like the sound of any of
>>>>>>
>> that. I'm just
>>
>>>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>>>
>>>>>> I had thought I was doing something special with the ognl
>>>>>> compilations that would cause its generated classes to not hang
>>>>>> around afterwards in any pools.
>>>>>>
>>>>>> I'll take a look at things this weekend and am sure a fix will
>>>>>> appear in between now and Monday - if there is a
>>>>>>
>> reasonable fix to be found.
>>
>>>>>> (in 4.1.3 snapshot form)
>>>>>>
>>>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com>
>>>>>>
>> <ma...@widen.com> wrote:
>>
>>>>>>
>>>>>>
>>>>>>> I and another colleague of mine have been investigating
>>>>>>>
>> what seems
>>
>>>>>>> to be a "memory leak" in our Tapestry application for about a
>>>>>>> month since we upgraded to T4.1.2. I won't bore you
>>>>>>>
>> with the saga
>>
>>>>>>> of the last month, but I would like to present the data I've
>>>>>>> gathered and look to the list for a proposed solution. I was
>>>>>>> reading a recent thread in which Jesse said (08/09/2007):
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> "There is a map that grows as large as the system using it
>>>>>>> internally to javassist of various cached reflection
>>>>>>>
>> info - but it
>>
>>>>>>> doesn't leak in any way."
>>>>>>>
>>>>>>> This is precisely what I've found in profiling our
>>>>>>>
>> application and
>>
>>>>>>> it
>>>>>>> *appears* to be this map that is causing our applications to
>>>>>>> eventually run out of memory.
>>>>>>>
>>>>>>> The YourKit profiler shows me that, as time goes on,
>>>>>>>
>> there is an
>>
>>>>>>> instance of HiveMindClassPool that grows and grows as class
>>>>>>> instances are created. This class extends from
>>>>>>> javassist.ClassPool and is the map that Jesse is
>>>>>>>
>> talking about in
>>
>>>>>>> his quote above. And he's right, I wouldn't say that the class
>>>>>>> pool "leaks" either because it looks like it's designed
>>>>>>>
>> to retain
>>
>>>>>>> that memory until the class pool itself is no longer needed.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>>>
>>>>>>> "Memory consumption memo:
>>>>>>> ClassPool objects hold all the CtClasses that have been
>>>>>>>
>> created so
>>
>>>>>>> that the consistency among modified classes can be guaranteed.
>>>>>>> Thus if a large number of CtClasses are processed, the
>>>>>>>
>> ClassPool
>>
>>>>>>> will consume a huge amount of memory. To avoid this, a
>>>>>>>
>> ClassPool
>>
>>>>>>> object should be recreated, for example, every hundred classes
>>>>>>> processed. Note that
>>>>>>> getDefault() is a singleton factory. Otherwise, detach() in
>>>>>>> CtClass should be used to avoid huge memory consumption. "
>>>>>>>
>>>>>>> This huge memory consumption by the ClassPool is what I was
>>>>>>> seeing. In particular it is the ClassPool that is held
>>>>>>>
>> onto by OgnlRuntime.
>>
>>>>>>> Inspecting this object in the profiler showed that it has a map
>>>>>>> containing about 45,000 classes. All of the keys into this map
>>>>>>> were things like: "ASTTest_11494aca9af" and
>>>>>>>
>> "ASTAnd_11494ace4fb"
>>
>>>>>>> and the values are instances of javassist.CtNewClass.
>>>>>>>
>> Each entry
>>
>>>>>>> in this map looks like it retains about 1,900 bytes,
>>>>>>>
>> for a grand
>>
>>>>>>> total of about 90 MB of memory used.
>>>>>>>
>>>>>>> These numbers came from my staging deployment where I had the
>>>>>>> profiler attached, using some reflection tricks I was
>>>>>>>
>> able to look
>>
>>>>>>> at a production site and found that it had about
>>>>>>>
>> 240,000 items in
>>
>>>>>>> that class pool.. approximately 450 MB of memory.
>>>>>>>
>>>>>>> So I guess the questions in my mind are: Why are there so many
>>>>>>> classes in the pool? Why does the number only ever go up? Do
>>>>>>> those classes really need to stay in the pool forever?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>> --------------------------------------------------------------------
>>
>>>>> - To unsubscribe, e-mail:
>>>>>
>> users-unsubscribe@tapestry.apache.org For
>>
>>>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>
>>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
RE: Memory consumption in T4.1.2 - Hard data
Posted by Ma...@bmw.ch.
> my configuration is as follows:
>
> JDK 6.01 32bit JVM (I have also tested on a 64 bit with no
> luck) Tomcat 5.5.20 Debian Linux (2.6.15.28 kernel) Tapestry
> 4.1.2-SNAPSHOT with ognl 2.7
>
Hi Peter, 4.1.2-SNAPSHOT is a typo, I suppose?
> regards,
> Peter
>
> Jesse Kuhnert wrote:
> > I'd downgrade then if I were you. Extensive profiling with yourkit
> > hasn't shed any new light on whatever problems others are having.
> > The OGNL classes indeed aren't being kept in the javassist
> pool from
> > what I can tell.
> >
> > I have another yourkit snapshot binary to look at so we'll
> see what that says.
> >
> > I don't remember - have you filed any bug reports or reported any
> > problems? Are you getting ognl exceptions during
> development as well
> > as production? Do these errors sound like
> ummmmm....potential errors
> > that should be looked at further?
> >
> > On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> >
> >> Hi Jessie
> >>
> >> Any progress on this? .... sorry to bug you, but I have to take a
> >> decision soon, I have two production machines that will need to
> >> upgrade or downgrade Tapestry.
> >>
> >> Best wishes,
> >> Peter
> >>
> >> Jon Oakes wrote:
> >>
> >>> Hi Bryan,
> >>>
> >>> I am a relative newbie but I was wondering if you are
> running with
> >>> caching enabled or disabled? It might make sense to keep things
> >>> around when caching is disabled whereas I think it would
> clearly be
> >>> a bug to keep things around with it disabled.
> >>>
> >>> Jon Oakes
> >>>
> >>>
> >>> Jesse Kuhnert wrote:
> >>>
> >>>> Hmmm...well, I don't think I like the sound of any of
> that. I'm just
> >>>> going to pretend this problem doesn't exist. (just kidding)
> >>>>
> >>>> I had thought I was doing something special with the ognl
> >>>> compilations that would cause its generated classes to not hang
> >>>> around afterwards in any pools.
> >>>>
> >>>> I'll take a look at things this weekend and am sure a fix will
> >>>> appear in between now and Monday - if there is a
> reasonable fix to be found.
> >>>> (in 4.1.3 snapshot form)
> >>>>
> >>>> On 8/24/07, Bryan Dotzour <BD...@widen.com>
> <ma...@widen.com> wrote:
> >>>>
> >>>>
> >>>>> I and another colleague of mine have been investigating
> what seems
> >>>>> to be a "memory leak" in our Tapestry application for about a
> >>>>> month since we upgraded to T4.1.2. I won't bore you
> with the saga
> >>>>> of the last month, but I would like to present the data I've
> >>>>> gathered and look to the list for a proposed solution. I was
> >>>>> reading a recent thread in which Jesse said (08/09/2007):
> >>>>>
> >>>>>
> >>>>>
> >>>>> "There is a map that grows as large as the system using it
> >>>>> internally to javassist of various cached reflection
> info - but it
> >>>>> doesn't leak in any way."
> >>>>>
> >>>>> This is precisely what I've found in profiling our
> application and
> >>>>> it
> >>>>> *appears* to be this map that is causing our applications to
> >>>>> eventually run out of memory.
> >>>>>
> >>>>> The YourKit profiler shows me that, as time goes on,
> there is an
> >>>>> instance of HiveMindClassPool that grows and grows as class
> >>>>> instances are created. This class extends from
> >>>>> javassist.ClassPool and is the map that Jesse is
> talking about in
> >>>>> his quote above. And he's right, I wouldn't say that the class
> >>>>> pool "leaks" either because it looks like it's designed
> to retain
> >>>>> that memory until the class pool itself is no longer needed.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Take this quote from the javassist.ClassPool javadocs:
> >>>>>
> >>>>> "Memory consumption memo:
> >>>>> ClassPool objects hold all the CtClasses that have been
> created so
> >>>>> that the consistency among modified classes can be guaranteed.
> >>>>> Thus if a large number of CtClasses are processed, the
> ClassPool
> >>>>> will consume a huge amount of memory. To avoid this, a
> ClassPool
> >>>>> object should be recreated, for example, every hundred classes
> >>>>> processed. Note that
> >>>>> getDefault() is a singleton factory. Otherwise, detach() in
> >>>>> CtClass should be used to avoid huge memory consumption. "
> >>>>>
> >>>>> This huge memory consumption by the ClassPool is what I was
> >>>>> seeing. In particular it is the ClassPool that is held
> onto by OgnlRuntime.
> >>>>> Inspecting this object in the profiler showed that it has a map
> >>>>> containing about 45,000 classes. All of the keys into this map
> >>>>> were things like: "ASTTest_11494aca9af" and
> "ASTAnd_11494ace4fb"
> >>>>> and the values are instances of javassist.CtNewClass.
> Each entry
> >>>>> in this map looks like it retains about 1,900 bytes,
> for a grand
> >>>>> total of about 90 MB of memory used.
> >>>>>
> >>>>> These numbers came from my staging deployment where I had the
> >>>>> profiler attached, using some reflection tricks I was
> able to look
> >>>>> at a production site and found that it had about
> 240,000 items in
> >>>>> that class pool.. approximately 450 MB of memory.
> >>>>>
> >>>>> So I guess the questions in my mind are: Why are there so many
> >>>>> classes in the pool? Why does the number only ever go up? Do
> >>>>> those classes really need to stay in the pool forever?
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>
> --------------------------------------------------------------------
> >>> - To unsubscribe, e-mail:
> users-unsubscribe@tapestry.apache.org For
> >>> additional commands, e-mail: users-help@tapestry.apache.org
> >>>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> >> For additional commands, e-mail: users-help@tapestry.apache.org
> >>
> >>
> >>
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
Hi Jessie
I'm gonna try update my version of ognl, if that won't work I will
downgrade. I have not filed any bugs yet, but I sent a couple of emails
to the list. I don't get any exceptions during development, but that is
because its not running for 3 or 4 days. I strongly suggest you consider
looking into it, because I use the standard libraries provided by
Tapestry through Maven.
my configuration is as follows:
JDK 6.01 32bit JVM (I have also tested on a 64 bit with no luck)
Tomcat 5.5.20
Debian Linux (2.6.15.28 kernel)
Tapestry 4.1.2-SNAPSHOT with ognl 2.7
regards,
Peter
Jesse Kuhnert wrote:
> I'd downgrade then if I were you. Extensive profiling with yourkit
> hasn't shed any new light on whatever problems others are having.
> The OGNL classes indeed aren't being kept in the javassist pool from
> what I can tell.
>
> I have another yourkit snapshot binary to look at so we'll see what that says.
>
> I don't remember - have you filed any bug reports or reported any
> problems? Are you getting ognl exceptions during development as well
> as production? Do these errors sound like ummmmm....potential errors
> that should be looked at further?
>
> On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
>
>> Hi Jessie
>>
>> Any progress on this? .... sorry to bug you, but I have to take a
>> decision soon, I have two production machines that will need to upgrade
>> or downgrade Tapestry.
>>
>> Best wishes,
>> Peter
>>
>> Jon Oakes wrote:
>>
>>> Hi Bryan,
>>>
>>> I am a relative newbie but I was wondering if you are running with
>>> caching enabled or disabled? It might make sense to keep things
>>> around when caching is disabled whereas I think it would clearly be a
>>> bug to keep things around with it disabled.
>>>
>>> Jon Oakes
>>>
>>>
>>> Jesse Kuhnert wrote:
>>>
>>>> Hmmm...well, I don't think I like the sound of any of that. I'm just
>>>> going to pretend this problem doesn't exist. (just kidding)
>>>>
>>>> I had thought I was doing something special with the ognl compilations
>>>> that would cause its generated classes to not hang around afterwards
>>>> in any pools.
>>>>
>>>> I'll take a look at things this weekend and am sure a fix will appear
>>>> in between now and Monday - if there is a reasonable fix to be found.
>>>> (in 4.1.3 snapshot form)
>>>>
>>>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <ma...@widen.com> wrote:
>>>>
>>>>
>>>>> I and another colleague of mine have been investigating what seems to be
>>>>> a "memory leak" in our Tapestry application for about a month since we
>>>>> upgraded to T4.1.2. I won't bore you with the saga of the last month,
>>>>> but I would like to present the data I've gathered and look to the list
>>>>> for a proposed solution. I was reading a recent thread in which Jesse
>>>>> said (08/09/2007):
>>>>>
>>>>>
>>>>>
>>>>> "There is a map that grows as large as the system using it internally to
>>>>> javassist of various cached reflection info - but it doesn't leak in any
>>>>> way."
>>>>>
>>>>> This is precisely what I've found in profiling our application and it
>>>>> *appears* to be this map that is causing our applications to eventually
>>>>> run out of memory.
>>>>>
>>>>> The YourKit profiler shows me that, as time goes on, there is an
>>>>> instance of HiveMindClassPool that grows and grows as class instances
>>>>> are created. This class extends from javassist.ClassPool and is the map
>>>>> that Jesse is talking about in his quote above. And he's right, I
>>>>> wouldn't say that the class pool "leaks" either because it looks like
>>>>> it's designed to retain that memory until the class pool itself is no
>>>>> longer needed.
>>>>>
>>>>>
>>>>>
>>>>> Take this quote from the javassist.ClassPool javadocs:
>>>>>
>>>>> "Memory consumption memo:
>>>>> ClassPool objects hold all the CtClasses that have been created so that
>>>>> the consistency among modified classes can be guaranteed. Thus if a
>>>>> large number of CtClasses are processed, the ClassPool will consume a
>>>>> huge amount of memory. To avoid this, a ClassPool object should be
>>>>> recreated, for example, every hundred classes processed. Note that
>>>>> getDefault() is a singleton factory. Otherwise, detach() in CtClass
>>>>> should be used to avoid huge memory consumption. "
>>>>>
>>>>> This huge memory consumption by the ClassPool is what I was seeing. In
>>>>> particular it is the ClassPool that is held onto by OgnlRuntime.
>>>>> Inspecting this object in the profiler showed that it has a map
>>>>> containing about 45,000 classes. All of the keys into this map were
>>>>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
>>>>> values are instances of javassist.CtNewClass. Each entry in this map
>>>>> looks like it retains about 1,900 bytes, for a grand total of about 90
>>>>> MB of memory used.
>>>>>
>>>>> These numbers came from my staging deployment where I had the profiler
>>>>> attached, using some reflection tricks I was able to look at a
>>>>> production site and found that it had about 240,000 items in that class
>>>>> pool.. approximately 450 MB of memory.
>>>>>
>>>>> So I guess the questions in my mind are: Why are there so many classes
>>>>> in the pool? Why does the number only ever go up? Do those classes
>>>>> really need to stay in the pool forever?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
>>> additional commands, e-mail: users-help@tapestry.apache.org
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Jesse Kuhnert <jk...@gmail.com>.
I'd downgrade then if I were you. Extensive profiling with yourkit
hasn't shed any new light on whatever problems others are having.
The OGNL classes indeed aren't being kept in the javassist pool from
what I can tell.
I have another yourkit snapshot binary to look at so we'll see what that says.
I don't remember - have you filed any bug reports or reported any
problems? Are you getting ognl exceptions during development as well
as production? Do these errors sound like ummmmm....potential errors
that should be looked at further?
On 8/28/07, Peter Stavrinides <p....@albourne.com> wrote:
> Hi Jessie
>
> Any progress on this? .... sorry to bug you, but I have to take a
> decision soon, I have two production machines that will need to upgrade
> or downgrade Tapestry.
>
> Best wishes,
> Peter
>
> Jon Oakes wrote:
> > Hi Bryan,
> >
> > I am a relative newbie but I was wondering if you are running with
> > caching enabled or disabled? It might make sense to keep things
> > around when caching is disabled whereas I think it would clearly be a
> > bug to keep things around with it disabled.
> >
> > Jon Oakes
> >
> >
> > Jesse Kuhnert wrote:
> >> Hmmm...well, I don't think I like the sound of any of that. I'm just
> >> going to pretend this problem doesn't exist. (just kidding)
> >>
> >> I had thought I was doing something special with the ognl compilations
> >> that would cause its generated classes to not hang around afterwards
> >> in any pools.
> >>
> >> I'll take a look at things this weekend and am sure a fix will appear
> >> in between now and Monday - if there is a reasonable fix to be found.
> >> (in 4.1.3 snapshot form)
> >>
> >> On 8/24/07, Bryan Dotzour <BD...@widen.com> <ma...@widen.com> wrote:
> >>
> >>> I and another colleague of mine have been investigating what seems to be
> >>> a "memory leak" in our Tapestry application for about a month since we
> >>> upgraded to T4.1.2. I won't bore you with the saga of the last month,
> >>> but I would like to present the data I've gathered and look to the list
> >>> for a proposed solution. I was reading a recent thread in which Jesse
> >>> said (08/09/2007):
> >>>
> >>>
> >>>
> >>> "There is a map that grows as large as the system using it internally to
> >>> javassist of various cached reflection info - but it doesn't leak in any
> >>> way."
> >>>
> >>> This is precisely what I've found in profiling our application and it
> >>> *appears* to be this map that is causing our applications to eventually
> >>> run out of memory.
> >>>
> >>> The YourKit profiler shows me that, as time goes on, there is an
> >>> instance of HiveMindClassPool that grows and grows as class instances
> >>> are created. This class extends from javassist.ClassPool and is the map
> >>> that Jesse is talking about in his quote above. And he's right, I
> >>> wouldn't say that the class pool "leaks" either because it looks like
> >>> it's designed to retain that memory until the class pool itself is no
> >>> longer needed.
> >>>
> >>>
> >>>
> >>> Take this quote from the javassist.ClassPool javadocs:
> >>>
> >>> "Memory consumption memo:
> >>> ClassPool objects hold all the CtClasses that have been created so that
> >>> the consistency among modified classes can be guaranteed. Thus if a
> >>> large number of CtClasses are processed, the ClassPool will consume a
> >>> huge amount of memory. To avoid this, a ClassPool object should be
> >>> recreated, for example, every hundred classes processed. Note that
> >>> getDefault() is a singleton factory. Otherwise, detach() in CtClass
> >>> should be used to avoid huge memory consumption. "
> >>>
> >>> This huge memory consumption by the ClassPool is what I was seeing. In
> >>> particular it is the ClassPool that is held onto by OgnlRuntime.
> >>> Inspecting this object in the profiler showed that it has a map
> >>> containing about 45,000 classes. All of the keys into this map were
> >>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
> >>> values are instances of javassist.CtNewClass. Each entry in this map
> >>> looks like it retains about 1,900 bytes, for a grand total of about 90
> >>> MB of memory used.
> >>>
> >>> These numbers came from my staging deployment where I had the profiler
> >>> attached, using some reflection tricks I was able to look at a
> >>> production site and found that it had about 240,000 items in that class
> >>> pool.. approximately 450 MB of memory.
> >>>
> >>> So I guess the questions in my mind are: Why are there so many classes
> >>> in the pool? Why does the number only ever go up? Do those classes
> >>> really need to stay in the pool forever?
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
> > additional commands, e-mail: users-help@tapestry.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
--
Jesse Kuhnert
Tapestry/Dojo team member/developer
Open source based consulting work centered around
dojo/tapestry/tacos/hivemind. http://blog.opencomponentry.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Peter Stavrinides <p....@albourne.com>.
Hi Jessie
Any progress on this? .... sorry to bug you, but I have to take a
decision soon, I have two production machines that will need to upgrade
or downgrade Tapestry.
Best wishes,
Peter
Jon Oakes wrote:
> Hi Bryan,
>
> I am a relative newbie but I was wondering if you are running with
> caching enabled or disabled? It might make sense to keep things
> around when caching is disabled whereas I think it would clearly be a
> bug to keep things around with it disabled.
>
> Jon Oakes
>
>
> Jesse Kuhnert wrote:
>> Hmmm...well, I don't think I like the sound of any of that. I'm just
>> going to pretend this problem doesn't exist. (just kidding)
>>
>> I had thought I was doing something special with the ognl compilations
>> that would cause its generated classes to not hang around afterwards
>> in any pools.
>>
>> I'll take a look at things this weekend and am sure a fix will appear
>> in between now and Monday - if there is a reasonable fix to be found.
>> (in 4.1.3 snapshot form)
>>
>> On 8/24/07, Bryan Dotzour <BD...@widen.com> <ma...@widen.com> wrote:
>>
>>> I and another colleague of mine have been investigating what seems to be
>>> a "memory leak" in our Tapestry application for about a month since we
>>> upgraded to T4.1.2. I won't bore you with the saga of the last month,
>>> but I would like to present the data I've gathered and look to the list
>>> for a proposed solution. I was reading a recent thread in which Jesse
>>> said (08/09/2007):
>>>
>>>
>>>
>>> "There is a map that grows as large as the system using it internally to
>>> javassist of various cached reflection info - but it doesn't leak in any
>>> way."
>>>
>>> This is precisely what I've found in profiling our application and it
>>> *appears* to be this map that is causing our applications to eventually
>>> run out of memory.
>>>
>>> The YourKit profiler shows me that, as time goes on, there is an
>>> instance of HiveMindClassPool that grows and grows as class instances
>>> are created. This class extends from javassist.ClassPool and is the map
>>> that Jesse is talking about in his quote above. And he's right, I
>>> wouldn't say that the class pool "leaks" either because it looks like
>>> it's designed to retain that memory until the class pool itself is no
>>> longer needed.
>>>
>>>
>>>
>>> Take this quote from the javassist.ClassPool javadocs:
>>>
>>> "Memory consumption memo:
>>> ClassPool objects hold all the CtClasses that have been created so that
>>> the consistency among modified classes can be guaranteed. Thus if a
>>> large number of CtClasses are processed, the ClassPool will consume a
>>> huge amount of memory. To avoid this, a ClassPool object should be
>>> recreated, for example, every hundred classes processed. Note that
>>> getDefault() is a singleton factory. Otherwise, detach() in CtClass
>>> should be used to avoid huge memory consumption. "
>>>
>>> This huge memory consumption by the ClassPool is what I was seeing. In
>>> particular it is the ClassPool that is held onto by OgnlRuntime.
>>> Inspecting this object in the profiler showed that it has a map
>>> containing about 45,000 classes. All of the keys into this map were
>>> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
>>> values are instances of javassist.CtNewClass. Each entry in this map
>>> looks like it retains about 1,900 bytes, for a grand total of about 90
>>> MB of memory used.
>>>
>>> These numbers came from my staging deployment where I had the profiler
>>> attached, using some reflection tricks I was able to look at a
>>> production site and found that it had about 240,000 items in that class
>>> pool.. approximately 450 MB of memory.
>>>
>>> So I guess the questions in my mind are: Why are there so many classes
>>> in the pool? Why does the number only ever go up? Do those classes
>>> really need to stay in the pool forever?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org For
> additional commands, e-mail: users-help@tapestry.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Memory consumption in T4.1.2 - Hard data
Posted by Jesse Kuhnert <jk...@gmail.com>.
Hmmm...well, I don't think I like the sound of any of that. I'm just
going to pretend this problem doesn't exist. (just kidding)
I had thought I was doing something special with the ognl compilations
that would cause its generated classes to not hang around afterwards
in any pools.
I'll take a look at things this weekend and am sure a fix will appear
in between now and Monday - if there is a reasonable fix to be found.
(in 4.1.3 snapshot form)
On 8/24/07, Bryan Dotzour <BD...@widen.com> wrote:
> I and another colleague of mine have been investigating what seems to be
> a "memory leak" in our Tapestry application for about a month since we
> upgraded to T4.1.2. I won't bore you with the saga of the last month,
> but I would like to present the data I've gathered and look to the list
> for a proposed solution. I was reading a recent thread in which Jesse
> said (08/09/2007):
>
>
>
> "There is a map that grows as large as the system using it internally to
> javassist of various cached reflection info - but it doesn't leak in any
> way."
>
> This is precisely what I've found in profiling our application and it
> *appears* to be this map that is causing our applications to eventually
> run out of memory.
>
> The YourKit profiler shows me that, as time goes on, there is an
> instance of HiveMindClassPool that grows and grows as class instances
> are created. This class extends from javassist.ClassPool and is the map
> that Jesse is talking about in his quote above. And he's right, I
> wouldn't say that the class pool "leaks" either because it looks like
> it's designed to retain that memory until the class pool itself is no
> longer needed.
>
>
>
> Take this quote from the javassist.ClassPool javadocs:
>
> "Memory consumption memo:
> ClassPool objects hold all the CtClasses that have been created so that
> the consistency among modified classes can be guaranteed. Thus if a
> large number of CtClasses are processed, the ClassPool will consume a
> huge amount of memory. To avoid this, a ClassPool object should be
> recreated, for example, every hundred classes processed. Note that
> getDefault() is a singleton factory. Otherwise, detach() in CtClass
> should be used to avoid huge memory consumption. "
>
> This huge memory consumption by the ClassPool is what I was seeing. In
> particular it is the ClassPool that is held onto by OgnlRuntime.
> Inspecting this object in the profiler showed that it has a map
> containing about 45,000 classes. All of the keys into this map were
> things like: "ASTTest_11494aca9af" and "ASTAnd_11494ace4fb" and the
> values are instances of javassist.CtNewClass. Each entry in this map
> looks like it retains about 1,900 bytes, for a grand total of about 90
> MB of memory used.
>
> These numbers came from my staging deployment where I had the profiler
> attached, using some reflection tricks I was able to look at a
> production site and found that it had about 240,000 items in that class
> pool.. approximately 450 MB of memory.
>
> So I guess the questions in my mind are: Why are there so many classes
> in the pool? Why does the number only ever go up? Do those classes
> really need to stay in the pool forever?
>
>
>
>
>
>
--
Jesse Kuhnert
Tapestry/Dojo team member/developer
Open source based consulting work centered around
dojo/tapestry/tacos/hivemind. http://blog.opencomponentry.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org