You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Jeff Turner <je...@apache.org> on 2006/05/04 09:03:49 UTC
Memory leak? (issues.apache.org)
Hi,
For the last few months, issues.apache.org/jira has been running out of
memory about once a week. We've finally got it running in a profiler, and
are seeing most of the memory (eg. 486 of 572Mb) used up by char[]
buffers in BodyContentImpl. Here is a sample GC Root -> Object trace:
char[16777218]
cb of org.apache.jasper.runtime.BodyContentImpl
[0] of org.apache.jasper.runtime.BodyContentImpl[7]
outs of org.apache.jasper.runtime.PageContextImpl
[86] of java.lang.Object[101]
pool of org.apache.jasper.util.SimplePool
pool of org.apache.jasper.runtime.JspFactoryImpl
deflt of javax.servlet.jsp.JspFactory
[57] of java.lang.Object[641]
elementData of java.util.Vector
classes of org.apache.catalina.loader.StandardClassLoader [Other]
There seems to be a constantly increasing number of BodyContentImpl
objects in the system:
1 May: 93 Objects (126Mb)
2 May: 107 Objects (263Mb)
3 May: 492 Objects (486MB)
(the first two were taken directly after a full gc)
It appears to be a case of this bug:
http://issues.apache.org/bugzilla/show_bug.cgi?id=37793
Perhaps this bug affects the ASF JIRA in particular (and not most people)
because people occasionally request huge (20-30Mb) pages. There are 23
BodyContentImpls between 33Mb and 10Mb in the last dump, and due to the
pooling, these all stick around taking up memory.
Could anyone comment on this issue? Remy seemed to think it was 'as
intended', and the bug is marked WONTFIX. I'm happy to provide yourkit
memory dumps or access to the server if necessary. We're currently
running 5.5.16.
Thanks!
Jeff
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Jean-frederic Clere <jf...@gmail.com>.
Jeff Turner wrote:
>Hi,
>
>For the last few months, issues.apache.org/jira has been running out of
>memory about once a week. We've finally got it running in a profiler, and
>are seeing most of the memory (eg. 486 of 572Mb) used up by char[]
>buffers in BodyContentImpl. Here is a sample GC Root -> Object trace:
>
> char[16777218]
> cb of org.apache.jasper.runtime.BodyContentImpl
> [0] of org.apache.jasper.runtime.BodyContentImpl[7]
> outs of org.apache.jasper.runtime.PageContextImpl
> [86] of java.lang.Object[101]
> pool of org.apache.jasper.util.SimplePool
> pool of org.apache.jasper.runtime.JspFactoryImpl
> deflt of javax.servlet.jsp.JspFactory
> [57] of java.lang.Object[641]
> elementData of java.util.Vector
> classes of org.apache.catalina.loader.StandardClassLoader [Other]
>
>
>There seems to be a constantly increasing number of BodyContentImpl
>objects in the system:
>
>1 May: 93 Objects (126Mb)
>2 May: 107 Objects (263Mb)
>3 May: 492 Objects (486MB)
>
>(the first two were taken directly after a full gc)
>
>
>It appears to be a case of this bug:
>
>http://issues.apache.org/bugzilla/show_bug.cgi?id=37793
>
>Perhaps this bug affects the ASF JIRA in particular (and not most people)
>because people occasionally request huge (20-30Mb) pages. There are 23
>BodyContentImpls between 33Mb and 10Mb in the last dump, and due to the
>pooling, these all stick around taking up memory.
>
>Could anyone comment on this issue? Remy seemed to think it was 'as
>intended', and the bug is marked WONTFIX. I'm happy to provide yourkit
>memory dumps or access to the server if necessary. We're currently
>running 5.5.16.
>
>
Reopen the bug, there seem to be something really wrong...
Cheers
Jean-Frederic
>
>Thanks!
>
>Jeff
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Jean-frederic Clere <jf...@gmail.com>.
Jeff Turner wrote:
>Hi,
>
>For the last few months, issues.apache.org/jira has been running out of
>memory about once a week. We've finally got it running in a profiler, and
>are seeing most of the memory (eg. 486 of 572Mb) used up by char[]
>buffers in BodyContentImpl. Here is a sample GC Root -> Object trace:
>
> char[16777218]
> cb of org.apache.jasper.runtime.BodyContentImpl
> [0] of org.apache.jasper.runtime.BodyContentImpl[7]
> outs of org.apache.jasper.runtime.PageContextImpl
> [86] of java.lang.Object[101]
> pool of org.apache.jasper.util.SimplePool
> pool of org.apache.jasper.runtime.JspFactoryImpl
> deflt of javax.servlet.jsp.JspFactory
> [57] of java.lang.Object[641]
> elementData of java.util.Vector
> classes of org.apache.catalina.loader.StandardClassLoader [Other]
>
>
>There seems to be a constantly increasing number of BodyContentImpl
>objects in the system:
>
>1 May: 93 Objects (126Mb)
>2 May: 107 Objects (263Mb)
>3 May: 492 Objects (486MB)
>
>(the first two were taken directly after a full gc)
>
>
>It appears to be a case of this bug:
>
>http://issues.apache.org/bugzilla/show_bug.cgi?id=37793
>
>Perhaps this bug affects the ASF JIRA in particular (and not most people)
>because people occasionally request huge (20-30Mb) pages. There are 23
>BodyContentImpls between 33Mb and 10Mb in the last dump, and due to the
>pooling, these all stick around taking up memory.
>
>Could anyone comment on this issue? Remy seemed to think it was 'as
>intended', and the bug is marked WONTFIX. I'm happy to provide yourkit
>memory dumps or access to the server if necessary. We're currently
>running 5.5.16.
>
>
Have you try to set org.apache.jasper.runtime.JspFactoryImpl.USE_POOL to
false?
(adding -Dorg.apache.jasper.runtime.JspFactoryImpl.USE_POOL=false to
CATALINA_OPTS environment variable).
Cheers
Jean-Frederic
>
>Thanks!
>
>Jeff
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC 6: What needs to be done?
Posted by Jess Holle <je...@ptc.com>.
Costin Manolache wrote:
> I didn't know this - how does it find the defaults ? I'm not talking only
> about
> default values of a property, but also in the object hierarchy, tomcat
> creates a lot
> of components by default, if you don't specify a manager or logger, a
> default one
> will be created. Seems pretty specific to tomcat - can't see how
> XMLEncoder
> would
> guess that.
It can't guess. For each object that is being encoded it has to be able
to create another from scratch and then it basically diffs the two and
writes the differences.
Sorry for the poor explanation -- it's all I have time for at the
moment. It's probably possible to arrange things such that XMLEncoder
would capture some level of hierarchy aspects as differences and
defaults, but I'm not sure how workable this would be.
[I have a top-level MBean that keeps a list of top-level entities
(primarily other MBeans) that then are delegated to for everything below
that point, so the defaulting stuff only occurs below the top-level in
my case -- and the hierarchy below that point is empty by default...]
--
Jess Holle
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC 6: What needs to be done?
Posted by Costin Manolache <co...@gmail.com>.
On 5/5/06, Jess Holle <je...@ptc.com> wrote:
>
> Costin Manolache wrote:
> > On 5/5/06, Jess Holle <je...@ptc.com> wrote:
> > AFAIK XMLEncoder/XMLDecoder will save all the attributes, and lose
> > comments
> > in the round trip.
> Yep. I'm not trying to save comments in my case -- just configuration
> values.
>
> What I feel is missing is that I persist *values*. I cannot persist the
> notion that the value was obtained via $(propertyName) substitution in
> the XML, for example.
Yes, that's hard. It may be possible to preserve them when using DOM if the
user
didn't change them - but if the user sets a value the $() is lost.
> I'm not sure what you mean by "having only the modified elements".
>
> XMLEncoder only saves attributes that differ from their default values,
> if that's what you mean.
I didn't know this - how does it find the defaults ? I'm not talking only
about
default values of a property, but also in the object hierarchy, tomcat
creates a lot
of components by default, if you don't specify a manager or logger, a
default one
will be created. Seems pretty specific to tomcat - can't see how XMLEncoder
would
guess that.
Costin
Re: TC 6: What needs to be done?
Posted by Jess Holle <je...@ptc.com>.
Costin Manolache wrote:
> On 5/5/06, Jess Holle <je...@ptc.com> wrote:
> AFAIK XMLEncoder/XMLDecoder will save all the attributes, and lose
> comments
> in the round trip.
Yep. I'm not trying to save comments in my case -- just configuration
values.
What I feel is missing is that I persist *values*. I cannot persist the
notion that the value was obtained via $(propertyName) substitution in
the XML, for example.
> With DOM - you have a good round trip, and if all config happens via JMX,
> you know what attributes have changed.
True enough. One could also merge these in via XSLT.
> Apple and Mozilla configs are the models I'm considering, even if some of
> the structure is lost in the
> round trip, I think having only the modified elements is essential. I
> don't
> think you can do this with XMLEncoder,
> and our server.xml saving doesn't seem to support this either.
I'm not sure what you mean by "having only the modified elements".
XMLEncoder only saves attributes that differ from their default values,
if that's what you mean.
--
Jess Holle
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC 6: What needs to be done?
Posted by Costin Manolache <co...@gmail.com>.
On 5/5/06, Jess Holle <je...@ptc.com> wrote:
>
> Costin Manolache wrote:
> > On 5/5/06, Jess Holle <je...@ptc.com> wrote:
> >> Costin Manolache wrote:
> >> > On 5/5/06, Rainer Jung <to...@kippdata.de> wrote:
> >> >> 1) Configuration Management
> >> >> ===========================
> >> >>
> >> >> My impreesion is, that to much configuration is hard-coded in
> RuleSet
> >> >> classes. Of course everyone can easily add properties to existing
> >> >> components, but adding subcomponents nedds changes in core RuleSet
> >> >> classes. Am I wrong? Should we change that to allow more complex
> >> >> subsystems handling their configuration rules independently of the
> >> core?
> >> >> One such example is the current stable clusster module.
> >> > IMO the entire server.xml and RuleSet should be deprecated, and
> >> replaced
> >> > with JMX. We could keep current server.xml around, for compat - but
> at
> >> > least not
> >> > extend it.
> >> >
> >> > Even the very limited MLET syntax can support most of our needs.
> >> >
> >> > RuleSets are just a way to set attributes ( == jmx setters ) and
> >> > define hierarchy
> >> > of componets ( == can be done based on JMX names, in a more dynamic
> >> way )
> >> I'd generally tend to agree, but suggest that thought be given not just
> >> on elegant configuration but also on the ability to change things via
> >> JMX and then *save* the results as an updated configuration.
> > 'save' is on my list for about 2 years now :-)
> >
> > But this would be the second step - after we use JMX to set up tomcat
> > instead
> > of server.xml.
> I just bring it up as I have completed a JMX framework for our use with
> round trip persistence to XML.
>
> The biggest limitation in my case is that the configuration files are
> not (nearly) so elegant as something like Spring's -- though they're
> rather similar to a point (I use Java's built-in XMLEncoder/XMLDecoder
> stuff). This is primarily so that the original configuration file can
> be reproduced reliably with very little persistence architecture on my
> part. Trying to provide the richness of something like Spring and
> provide round-trip saving would seem to be the holy grail -- and
> possibly just as unattainable.
> > My old plan was to use something like MLET, and DOM to read/save ( so
> > most of comments/structure is preserved ).
> Use of XMLEncoder/XMLDecoder allows persistence of arbitrarily complex
> substructures, which is nice. The result is not extraordinarily
> reader-friendly, but it is quite general.
AFAIK XMLEncoder/XMLDecoder will save all the attributes, and lose comments
in the round trip.
With DOM - you have a good round trip, and if all config happens via JMX,
you know what attributes
have changed.
Apple and Mozilla configs are the models I'm considering, even if some of
the structure is lost in the
round trip, I think having only the modified elements is essential. I don't
think you can do this with XMLEncoder,
and our server.xml saving doesn't seem to support this either.
Costin
Re: TC 6: What needs to be done?
Posted by Jess Holle <je...@ptc.com>.
Costin Manolache wrote:
> On 5/5/06, Jess Holle <je...@ptc.com> wrote:
>> Costin Manolache wrote:
>> > On 5/5/06, Rainer Jung <to...@kippdata.de> wrote:
>> >> 1) Configuration Management
>> >> ===========================
>> >>
>> >> My impreesion is, that to much configuration is hard-coded in RuleSet
>> >> classes. Of course everyone can easily add properties to existing
>> >> components, but adding subcomponents nedds changes in core RuleSet
>> >> classes. Am I wrong? Should we change that to allow more complex
>> >> subsystems handling their configuration rules independently of the
>> core?
>> >> One such example is the current stable clusster module.
>> > IMO the entire server.xml and RuleSet should be deprecated, and
>> replaced
>> > with JMX. We could keep current server.xml around, for compat - but at
>> > least not
>> > extend it.
>> >
>> > Even the very limited MLET syntax can support most of our needs.
>> >
>> > RuleSets are just a way to set attributes ( == jmx setters ) and
>> > define hierarchy
>> > of componets ( == can be done based on JMX names, in a more dynamic
>> way )
>> I'd generally tend to agree, but suggest that thought be given not just
>> on elegant configuration but also on the ability to change things via
>> JMX and then *save* the results as an updated configuration.
> 'save' is on my list for about 2 years now :-)
>
> But this would be the second step - after we use JMX to set up tomcat
> instead
> of server.xml.
I just bring it up as I have completed a JMX framework for our use with
round trip persistence to XML.
The biggest limitation in my case is that the configuration files are
not (nearly) so elegant as something like Spring's -- though they're
rather similar to a point (I use Java's built-in XMLEncoder/XMLDecoder
stuff). This is primarily so that the original configuration file can
be reproduced reliably with very little persistence architecture on my
part. Trying to provide the richness of something like Spring and
provide round-trip saving would seem to be the holy grail -- and
possibly just as unattainable.
> My old plan was to use something like MLET, and DOM to read/save ( so
> most of comments/structure is preserved ).
Use of XMLEncoder/XMLDecoder allows persistence of arbitrarily complex
substructures, which is nice. The result is not extraordinarily
reader-friendly, but it is quite general.
--
Jess Holle
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC 6: What needs to be done?
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
Costin Manolache wrote:
>> There is a tradeoff here, though. Elegant configurations generally>
>> can't be reproduced via fairly simple save mechanisms, i.e.>
>> roundtripping of the data becomes an issue.
> That's true. At one end you have the windows registry, the other
> -custom config files thatare very unfriendly to programs. There is
> something in the middle -maybe apple-style configfiles, or XML.
> My old plan was to use something like MLET, and DOM to read/save (
> somost of comments/structure is preserved ).
completely unrelated topic, but ANT has a pretty neat way of injecting
pretty much anything anywhere in the build.xml file,
not that server.xml will be anything like build.xml, but the idea of
injecting sub components, is pretty neat.
so if I want to provide my custom connector, I could have sub XML
elements to the connector element.
Filip
>
>> >> 6) Timeline> >> ===========> >>> >> How much we will change for TC
>> 6 also depends on how much time we are> >> willing to give it. Is
>> there any time limit we should preserve, to> >> deliver a JEE 5 web
>> container not too far behind the standard approval?> >> I assujme
>> everyone wants TC 6 to show up before end of the year (beta),> >> but
>> how much earlier will it need to be available?> I'm more interested
>> in availability of full support for the new servlet> and JSP
>> standards in a fast, stable, scalable> Tomcat release than anything
>> else.
> +1
> Everything else can be done incrementally, in future releases.
> And it's even better if cluster, etc are separate module, withseparate
> release cycle - this way they won't block or add extraoverhead to
> tomcat release.
you'd still need to coordinate the releases, cause if the new tomcat
version is released and it breaks the cluster module, then you are
shipping a broken module, and I do believe that Tomcat should with each
release ship a default cluster implementation.
That is why I opted to have "ha" in the main tree, cause "ha" is an
extension to the tomcat components but swapping out few key components
to enable replication. "ha" itself doesn't contain any "cluster" logic,
it only enabled data to be serialized/deserialized, diffed etc,
something that is core to tomcat.
as always, I will listen to the majority.
> Costin
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC 6: What needs to be done?
Posted by Costin Manolache <co...@gmail.com>.
On 5/5/06, Jess Holle <je...@ptc.com> wrote:
> Costin Manolache wrote:
> > On 5/5/06, Rainer Jung <to...@kippdata.de> wrote:
> >> 1) Configuration Management
> >> ===========================
> >>
> >> My impreesion is, that to much configuration is hard-coded in RuleSet
> >> classes. Of course everyone can easily add properties to existing
> >> components, but adding subcomponents nedds changes in core RuleSet
> >> classes. Am I wrong? Should we change that to allow more complex
> >> subsystems handling their configuration rules independently of the core?
> >> One such example is the current stable clusster module.
> > IMO the entire server.xml and RuleSet should be deprecated, and replaced
> > with JMX. We could keep current server.xml around, for compat - but at
> > least not
> > extend it.
> >
> > Even the very limited MLET syntax can support most of our needs.
> >
> > RuleSets are just a way to set attributes ( == jmx setters ) and
> > define hierarchy
> > of componets ( == can be done based on JMX names, in a more dynamic way )
> I'd generally tend to agree, but suggest that thought be given not just
> on elegant configuration but also on the ability to change things via
> JMX and then *save* the results as an updated configuration.
'save' is on my list for about 2 years now :-)
But this would be the second step - after we use JMX to set up tomcat instead
of server.xml.
>
> There is a tradeoff here, though. Elegant configurations generally
> can't be reproduced via fairly simple save mechanisms, i.e.
> roundtripping of the data becomes an issue.
That's true. At one end you have the windows registry, the other -
custom config files that
are very unfriendly to programs. There is something in the middle -
maybe apple-style config
files, or XML.
My old plan was to use something like MLET, and DOM to read/save ( so
most of comments/structure is preserved ).
> >> 6) Timeline
> >> ===========
> >>
> >> How much we will change for TC 6 also depends on how much time we are
> >> willing to give it. Is there any time limit we should preserve, to
> >> deliver a JEE 5 web container not too far behind the standard approval?
> >> I assujme everyone wants TC 6 to show up before end of the year (beta),
> >> but how much earlier will it need to be available?
> I'm more interested in availability of full support for the new servlet
> and JSP standards in a fast, stable, scalable
> Tomcat release than anything else.
+1
Everything else can be done incrementally, in future releases.
And it's even better if cluster, etc are separate module, with
separate release cycle - this way they won't block or add extra
overhead to tomcat release.
Costin
Re: TC 6: What needs to be done?
Posted by Jess Holle <je...@ptc.com>.
Costin Manolache wrote:
> On 5/5/06, Rainer Jung <to...@kippdata.de> wrote:
>> 1) Configuration Management
>> ===========================
>>
>> My impreesion is, that to much configuration is hard-coded in RuleSet
>> classes. Of course everyone can easily add properties to existing
>> components, but adding subcomponents nedds changes in core RuleSet
>> classes. Am I wrong? Should we change that to allow more complex
>> subsystems handling their configuration rules independently of the core?
>> One such example is the current stable clusster module.
> IMO the entire server.xml and RuleSet should be deprecated, and replaced
> with JMX. We could keep current server.xml around, for compat - but at
> least not
> extend it.
>
> Even the very limited MLET syntax can support most of our needs.
>
> RuleSets are just a way to set attributes ( == jmx setters ) and
> define hierarchy
> of componets ( == can be done based on JMX names, in a more dynamic way )
I'd generally tend to agree, but suggest that thought be given not just
on elegant configuration but also on the ability to change things via
JMX and then *save* the results as an updated configuration.
There is a tradeoff here, though. Elegant configurations generally
can't be reproduced via fairly simple save mechanisms, i.e.
roundtripping of the data becomes an issue.
>> 6) Timeline
>> ===========
>>
>> How much we will change for TC 6 also depends on how much time we are
>> willing to give it. Is there any time limit we should preserve, to
>> deliver a JEE 5 web container not too far behind the standard approval?
>> I assujme everyone wants TC 6 to show up before end of the year (beta),
>> but how much earlier will it need to be available?
I'm more interested in availability of full support for the new servlet
and JSP standards in a fast, stable, scalable
Tomcat release than anything else.
--
Jess Holle
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC 6: What needs to be done?
Posted by Peter Rossbach <pr...@objektpark.de>.
Hi
Am 05.05.2006 um 17:21 schrieb Costin Manolache:
> On 5/5/06, Rainer Jung <to...@kippdata.de> wrote:
>> Hi,
>>
>> Remy started a thread talking about source tree reorg. It soon turned
>> into a discussion about various integration questions.
>>
>> I would be interested in discussing a couple of questions
>> concerning TC
>> 6, most of them already came up during the last week. I hope it's
>> not to
>> much shortly before the weekend :)
>>
>> My focus is: do other commiters also think some of these things
>> should
>> be improved, and who would be willing to care about some of these
>> topics.
>>
>> 1) Configuration Management
>> ===========================
>>
>> My impreesion is, that to much configuration is hard-coded in RuleSet
>> classes. Of course everyone can easily add properties to existing
>> components, but adding subcomponents nedds changes in core RuleSet
>> classes. Am I wrong? Should we change that to allow more complex
>> subsystems handling their configuration rules independently of the
>> core?
>> One such example is the current stable clusster module.
>
> IMO the entire server.xml and RuleSet should be deprecated, and
> replaced
> with JMX. We could keep current server.xml around, for compat - but at
> least not
> extend it.
>
Can you send me a dicussion link to this topic?
> Even the very limited MLET syntax can support most of our needs.
>
> RuleSets are just a way to set attributes ( == jmx setters ) and
> define hierarchy
> of componets ( == can be done based on JMX names, in a more dynamic
> way )
>
Look here at my very old patch to get a better Mlet support at
commons modeller.
http://issues.apache.org/bugzilla/show_bug.cgi?id=26493
>
>>
>> 2) Deployer
>> ===========
>>
>> Is it worth trying to implement a new deployer? I've no concrete
>> design
>> in my head now, but I had the impression, that although the current
>> deployer works, there is an option that a redesign would make it more
>> user friendly.
>>
>> 3) Manager
>> ==========
>>
>> There is a lot of code duplication between the different Manager
>> implementations. Some refactoring will help.
>>
>> 4) Core vs. Component Bundling, especially Cluster Module
>> =========================================================
>>
>> What should be included in the core download. I'm not really talking
>> about the code structure in subversion. It's the question of the
>> right
>> size of standard download. I think the current bundlings are OK,
>> except
>> that we have to decide how to handle the cluster modules. Any more
>> module like things coming up?
>>
>> Concerning the cluster modules: I would propose to make both of them
>> optional modules. For this to be easily usable one question will be,
>> what the user has to do, to integrate his cluster download into
>> the core
>> distribution. At the moment, the cluster config for the stable
>> version
>> is already contained in server.xml. Here the discussion has a close
>> relation to 1). I think it would help to be able to simply change the
>> cluster module used via config/system property inside a core
>> configuration and to configure the details of clustering itself
>> inside a
>> configuration object provided by the additional cluster module
>> download.
>
> +1 on making it optional. IMHO even things like JK could be an
> optional component.
>
+1
>
>
>> 5) Default Cluster
>> ==================
>>
>> Assuming that we will provide an easy way of switching between the
>> implementations, we should decide on the default cluster module, when
>> the time for the first 6.0 beta is coming closer. The main criterium
>> then should be code stability for the new module.
>>
>> In every case we need to support usage of the old module with TC
>> 6, so i
>> think it needs to stay compatible. but we should clearly state,
>> that the
>> old module will no longer be actively developped and we should even
>> deprecate it in TC 6, so we don't need to support it behind TC 6.
>>
>> 6) Timeline
>> ===========
>>
>> How much we will change for TC 6 also depends on how much time we are
>> willing to give it. Is there any time limit we should preserve, to
>> deliver a JEE 5 web container not too far behind the standard
>> approval?
>> I assujme everyone wants TC 6 to show up before end of the year
>> (beta),
>> but how much earlier will it need to be available?
>
> Not everything has to happen at once, and even less if we divert
> some optional
> modules like cluster in separate components.
>
> AFAIK the only requirement is the new API.
>
+1
>>
>> 7) Build, Test
>> ==============
>>
>> I think automated Gump builds should be helpful. Is there a need for
>> more automted testing?
>>
Yes, I think we must start to wrote junit testcase for better finding
strange component bugs
and easier integration testing. It was also nice to have separate
build script for check code
with PMD, checkstyle, findbugs,.... :-)
I have a lot of experience with CruiseControl that we can verify
checkings with the testsuite
and build code quality reports.
>> Mnny questions, not so many opinions in this mail. But I think
>> it's the
>> right time to discuss about these topics to form a consensus about
>> TC 6.
>>
>> - Rainer
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: dev-help@tomcat.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC 6: What needs to be done?
Posted by Costin Manolache <co...@gmail.com>.
On 5/5/06, Rainer Jung <to...@kippdata.de> wrote:
> Hi,
>
> Remy started a thread talking about source tree reorg. It soon turned
> into a discussion about various integration questions.
>
> I would be interested in discussing a couple of questions concerning TC
> 6, most of them already came up during the last week. I hope it's not to
> much shortly before the weekend :)
>
> My focus is: do other commiters also think some of these things should
> be improved, and who would be willing to care about some of these topics.
>
> 1) Configuration Management
> ===========================
>
> My impreesion is, that to much configuration is hard-coded in RuleSet
> classes. Of course everyone can easily add properties to existing
> components, but adding subcomponents nedds changes in core RuleSet
> classes. Am I wrong? Should we change that to allow more complex
> subsystems handling their configuration rules independently of the core?
> One such example is the current stable clusster module.
IMO the entire server.xml and RuleSet should be deprecated, and replaced
with JMX. We could keep current server.xml around, for compat - but at
least not
extend it.
Even the very limited MLET syntax can support most of our needs.
RuleSets are just a way to set attributes ( == jmx setters ) and
define hierarchy
of componets ( == can be done based on JMX names, in a more dynamic way )
>
> 2) Deployer
> ===========
>
> Is it worth trying to implement a new deployer? I've no concrete design
> in my head now, but I had the impression, that although the current
> deployer works, there is an option that a redesign would make it more
> user friendly.
>
> 3) Manager
> ==========
>
> There is a lot of code duplication between the different Manager
> implementations. Some refactoring will help.
>
> 4) Core vs. Component Bundling, especially Cluster Module
> =========================================================
>
> What should be included in the core download. I'm not really talking
> about the code structure in subversion. It's the question of the right
> size of standard download. I think the current bundlings are OK, except
> that we have to decide how to handle the cluster modules. Any more
> module like things coming up?
>
> Concerning the cluster modules: I would propose to make both of them
> optional modules. For this to be easily usable one question will be,
> what the user has to do, to integrate his cluster download into the core
> distribution. At the moment, the cluster config for the stable version
> is already contained in server.xml. Here the discussion has a close
> relation to 1). I think it would help to be able to simply change the
> cluster module used via config/system property inside a core
> configuration and to configure the details of clustering itself inside a
> configuration object provided by the additional cluster module download.
+1 on making it optional. IMHO even things like JK could be an
optional component.
> 5) Default Cluster
> ==================
>
> Assuming that we will provide an easy way of switching between the
> implementations, we should decide on the default cluster module, when
> the time for the first 6.0 beta is coming closer. The main criterium
> then should be code stability for the new module.
>
> In every case we need to support usage of the old module with TC 6, so i
> think it needs to stay compatible. but we should clearly state, that the
> old module will no longer be actively developped and we should even
> deprecate it in TC 6, so we don't need to support it behind TC 6.
>
> 6) Timeline
> ===========
>
> How much we will change for TC 6 also depends on how much time we are
> willing to give it. Is there any time limit we should preserve, to
> deliver a JEE 5 web container not too far behind the standard approval?
> I assujme everyone wants TC 6 to show up before end of the year (beta),
> but how much earlier will it need to be available?
Not everything has to happen at once, and even less if we divert some optional
modules like cluster in separate components.
AFAIK the only requirement is the new API.
>
> 7) Build, Test
> ==============
>
> I think automated Gump builds should be helpful. Is there a need for
> more automted testing?
>
> Mnny questions, not so many opinions in this mail. But I think it's the
> right time to discuss about these topics to form a consensus about TC 6.
>
> - Rainer
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
Re: TC 6: What needs to be done?
Posted by Remy Maucherat <re...@apache.org>.
Rainer Jung wrote:
> I assujme everyone wants TC 6 to show up before end of the year (beta),
> but how much earlier will it need to be available?
How about next month ? ;)
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
TC 6: What needs to be done?
Posted by Rainer Jung <to...@kippdata.de>.
Hi,
Remy started a thread talking about source tree reorg. It soon turned
into a discussion about various integration questions.
I would be interested in discussing a couple of questions concerning TC
6, most of them already came up during the last week. I hope it's not to
much shortly before the weekend :)
My focus is: do other commiters also think some of these things should
be improved, and who would be willing to care about some of these topics.
1) Configuration Management
===========================
My impreesion is, that to much configuration is hard-coded in RuleSet
classes. Of course everyone can easily add properties to existing
components, but adding subcomponents nedds changes in core RuleSet
classes. Am I wrong? Should we change that to allow more complex
subsystems handling their configuration rules independently of the core?
One such example is the current stable clusster module.
2) Deployer
===========
Is it worth trying to implement a new deployer? I've no concrete design
in my head now, but I had the impression, that although the current
deployer works, there is an option that a redesign would make it more
user friendly.
3) Manager
==========
There is a lot of code duplication between the different Manager
implementations. Some refactoring will help.
4) Core vs. Component Bundling, especially Cluster Module
=========================================================
What should be included in the core download. I'm not really talking
about the code structure in subversion. It's the question of the right
size of standard download. I think the current bundlings are OK, except
that we have to decide how to handle the cluster modules. Any more
module like things coming up?
Concerning the cluster modules: I would propose to make both of them
optional modules. For this to be easily usable one question will be,
what the user has to do, to integrate his cluster download into the core
distribution. At the moment, the cluster config for the stable version
is already contained in server.xml. Here the discussion has a close
relation to 1). I think it would help to be able to simply change the
cluster module used via config/system property inside a core
configuration and to configure the details of clustering itself inside a
configuration object provided by the additional cluster module download.
5) Default Cluster
==================
Assuming that we will provide an easy way of switching between the
implementations, we should decide on the default cluster module, when
the time for the first 6.0 beta is coming closer. The main criterium
then should be code stability for the new module.
In every case we need to support usage of the old module with TC 6, so i
think it needs to stay compatible. but we should clearly state, that the
old module will no longer be actively developped and we should even
deprecate it in TC 6, so we don't need to support it behind TC 6.
6) Timeline
===========
How much we will change for TC 6 also depends on how much time we are
willing to give it. Is there any time limit we should preserve, to
deliver a JEE 5 web container not too far behind the standard approval?
I assujme everyone wants TC 6 to show up before end of the year (beta),
but how much earlier will it need to be available?
7) Build, Test
==============
I think automated Gump builds should be helpful. Is there a need for
more automted testing?
Mnny questions, not so many opinions in this mail. But I think it's the
right time to discuss about these topics to form a consensus about TC 6.
- Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Jeff Turner <je...@apache.org>.
On Fri, May 05, 2006 at 06:07:26PM +0200, Rainer Jung wrote:
> ... and I definitely want to choose the size of one chunk not to small,
> e.g. one of the will be able to grow until 64KB, so that usually you
> will only need one of them. Only for jumbo pages we will start gluing
> them together...
>
> So I'll dig my head into BodyContentImpl ...
I'd be interested in trying whatever you come up with on
issues.apache.org. Most pages there are normal-sized, but occasionally
people request large (10Mb+) RSS/XML files.
--Jeff
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Rainer Jung <to...@kippdata.de>.
... and I definitely want to choose the size of one chunk not to small,
e.g. one of the will be able to grow until 64KB, so that usually you
will only need one of them. Only for jumbo pages we will start gluing
them together...
So I'll dig my head into BodyContentImpl ...
Filip Hanik - Dev Lists wrote:
> Remy Maucherat wrote:
>
>> Rainer Jung wrote:
>>
>>> I'm wondering if we should split the (possibly huge) char arrays in
>>> BodyContentImpl into smaller chunks of char arrays. Each chunk will
>>> be able to grow big enough to handle the usual cases efficiently
>>> (e.g. 64KB). Whenever a bigger size is needed we allocate more of
>>> these chunks from a pool. After using the BodyContentImpl we give
>>> back all chunks except for the first to the chunk pool.
>>>
>>> This way performance should not really suffer, but the char arrays
>>> can be efficiently shrinked for apps needing generating large
>>> responses every now and then.
>>
>>
>> This could be a good idea, but performance would suffer, I think: if
>> one request needs 100 buffers, then you'll have 100 synced operations
>> to retrieve them from the pool (only one to put them back, hopefully
>> ;)). It could then be cheaper to always allocate new objects.
>
> yes, but in TC6 we can use java.util.concurrent, and get a little bit
> more juice out of it, there is a pretty big difference between the lock
> free thread safe operations and the lock algo ones we use today
> (synchronized)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
Remy Maucherat wrote:
> Rainer Jung wrote:
>> I'm wondering if we should split the (possibly huge) char arrays in
>> BodyContentImpl into smaller chunks of char arrays. Each chunk will
>> be able to grow big enough to handle the usual cases efficiently
>> (e.g. 64KB). Whenever a bigger size is needed we allocate more of
>> these chunks from a pool. After using the BodyContentImpl we give
>> back all chunks except for the first to the chunk pool.
>>
>> This way performance should not really suffer, but the char arrays
>> can be efficiently shrinked for apps needing generating large
>> responses every now and then.
>
> This could be a good idea, but performance would suffer, I think: if
> one request needs 100 buffers, then you'll have 100 synced operations
> to retrieve them from the pool (only one to put them back, hopefully
> ;)). It could then be cheaper to always allocate new objects.
yes, but in TC6 we can use java.util.concurrent, and get a little bit
more juice out of it, there is a pretty big difference between the lock
free thread safe operations and the lock algo ones we use today
(synchronized)
Filip
>
>
> Rémy
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Remy Maucherat <re...@apache.org>.
Rainer Jung wrote:
> I'm wondering if we should split the (possibly huge) char arrays in
> BodyContentImpl into smaller chunks of char arrays. Each chunk will be
> able to grow big enough to handle the usual cases efficiently (e.g.
> 64KB). Whenever a bigger size is needed we allocate more of these chunks
> from a pool. After using the BodyContentImpl we give back all chunks
> except for the first to the chunk pool.
>
> This way performance should not really suffer, but the char arrays can
> be efficiently shrinked for apps needing generating large responses
> every now and then.
This could be a good idea, but performance would suffer, I think: if one
request needs 100 buffers, then you'll have 100 synced operations to
retrieve them from the pool (only one to put them back, hopefully ;)).
It could then be cheaper to always allocate new objects.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Rainer Jung <to...@kippdata.de>.
I'm wondering if we should split the (possibly huge) char arrays in
BodyContentImpl into smaller chunks of char arrays. Each chunk will be
able to grow big enough to handle the usual cases efficiently (e.g.
64KB). Whenever a bigger size is needed we allocate more of these chunks
from a pool. After using the BodyContentImpl we give back all chunks
except for the first to the chunk pool.
This way performance should not really suffer, but the char arrays can
be efficiently shrinked for apps needing generating large responses
every now and then.
The case in which this will be worse is, when most of the requests
generate large responses. For this case, maximum chunk size could be
configurable.
I didn't dig through the code, whether the change from a simple char
array to a flexible list of char arrays will be a difficult one, but I
will go through it and try to implement it, if there's no big warning
coming from the list.
Remy Maucherat wrote:
> Jeff Turner wrote:
>
>> 1 May: 93 Objects (126Mb)
>> 2 May: 107 Objects (263Mb)
>> 3 May: 492 Objects (486MB)
>
>
> BodyContentImpls are pooled and reused since it makes JSP processing
> significantly faster. If the application is evil, and uses body tags
> with huge bodies (it seems to be the case here), then there could be a
> problem.
>
> Large buffers may be discarded after usage, by setting the
> "org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER" system property
> to "true". Unfortunately, performance will go down and GC activity will
> go up.
>
> If you would like to limit the amount of resident memory that is used,
> you can disable pooling of buffer objects completely using the
> "org.apache.jasper.runtime.JspFactoryImpl.USE_POOL" system property set
> to "false". The performance will go down further due to object
> allocations and GC activity, but you may have a few CPU cycles to spare.
>
> BTW, you can set system properties in the catalina.properties file
> (although you may set them on the command line too).
>
> Rémy
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Jeff Turner <je...@apache.org>.
On Thu, May 04, 2006 at 11:10:20AM +0200, Remy Maucherat wrote:
> Jeff Turner wrote:
> > 1 May: 93 Objects (126Mb)
> > 2 May: 107 Objects (263Mb)
> > 3 May: 492 Objects (486MB)
>
> BodyContentImpls are pooled and reused since it makes JSP processing
> significantly faster. If the application is evil, and uses body tags
> with huge bodies (it seems to be the case here), then there could be a
> problem.
>
> Large buffers may be discarded after usage, by setting the
> "org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER" system property
> to "true". Unfortunately, performance will go down and GC activity will
> go up.
Thanks, I'll give that a try.
--Jeff
> If you would like to limit the amount of resident memory that is used,
> you can disable pooling of buffer objects completely using the
> "org.apache.jasper.runtime.JspFactoryImpl.USE_POOL" system property set
> to "false". The performance will go down further due to object
> allocations and GC activity, but you may have a few CPU cycles to spare.
>
> BTW, you can set system properties in the catalina.properties file
> (although you may set them on the command line too).
>
> Rémy
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Rainer Jung <ra...@kippdata.de>.
From my understanding PageContextImpls are pooled, but not the JSP
output (body). Those are written using JspWriterImpls that buffer only
8KB of output.
If you are using tags, then their content is kept in a char array inside
BodyContentImpls. Each (pooled) PageContextImpl has a stack of these and
the stack and the BodyContentImpl including the char array are not
resized after usage. The pooled PageContextImpl keeps references to those.
The LIMIT_BUFFER makes the BodyContentImpl resizing to 512 Bytes in case
it grew larger.
Jeff Turner wrote:
> Hi,
>
> On Thu, May 04, 2006 at 11:10:20AM +0200, Remy Maucherat wrote:
>> Jeff Turner wrote:
>>> 1 May: 93 Objects (126Mb)
>>> 2 May: 107 Objects (263Mb)
>>> 3 May: 492 Objects (486MB)
>> BodyContentImpls are pooled and reused since it makes JSP processing
>> significantly faster. If the application is evil, and uses body tags
>> with huge bodies (it seems to be the case here), then there could be a
>> problem.
>
> Just wondering - are all JSP bodies pooled, or only tag bodies? Eg.
> would this JSP's body be pooled:
>
> <%= for (int i=0; i<1000000; i++) { out.print("womble"); } %>
>
> Or only if it were wrapped in JSP tag.
>
>> Large buffers may be discarded after usage, by setting the
>> "org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER" system property
>> to "true". Unfortunately, performance will go down and GC activity will
>> go up.
>
> It appears to have fixed the leak on issues.a.o.
>
>
> Cheers,
> Jeff
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
--
kippdata informationstechnologie GmbH
Bornheimer Str. 33a
53111 Bonn
Tel.: 0228/98549-0
Fax: 0228/98549-50
www.kippdata.de
=======================
kippdata informationstechnologie GmbH
Bornheimer Str. 33a
D-53111 Bonn
Tel.: +49/0228/98549-0
Fax: +49/0228/98549-50
www.kippdata.de
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Jeff Turner <je...@apache.org>.
Hi,
On Thu, May 04, 2006 at 11:10:20AM +0200, Remy Maucherat wrote:
> Jeff Turner wrote:
> > 1 May: 93 Objects (126Mb)
> > 2 May: 107 Objects (263Mb)
> > 3 May: 492 Objects (486MB)
>
> BodyContentImpls are pooled and reused since it makes JSP processing
> significantly faster. If the application is evil, and uses body tags
> with huge bodies (it seems to be the case here), then there could be a
> problem.
Just wondering - are all JSP bodies pooled, or only tag bodies? Eg.
would this JSP's body be pooled:
<%= for (int i=0; i<1000000; i++) { out.print("womble"); } %>
Or only if it were wrapped in JSP tag.
> Large buffers may be discarded after usage, by setting the
> "org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER" system property
> to "true". Unfortunately, performance will go down and GC activity will
> go up.
It appears to have fixed the leak on issues.a.o.
Cheers,
Jeff
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Memory leak? (issues.apache.org)
Posted by Remy Maucherat <re...@apache.org>.
Jeff Turner wrote:
> 1 May: 93 Objects (126Mb)
> 2 May: 107 Objects (263Mb)
> 3 May: 492 Objects (486MB)
BodyContentImpls are pooled and reused since it makes JSP processing
significantly faster. If the application is evil, and uses body tags
with huge bodies (it seems to be the case here), then there could be a
problem.
Large buffers may be discarded after usage, by setting the
"org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER" system property
to "true". Unfortunately, performance will go down and GC activity will
go up.
If you would like to limit the amount of resident memory that is used,
you can disable pooling of buffer objects completely using the
"org.apache.jasper.runtime.JspFactoryImpl.USE_POOL" system property set
to "false". The performance will go down further due to object
allocations and GC activity, but you may have a few CPU cycles to spare.
BTW, you can set system properties in the catalina.properties file
(although you may set them on the command line too).
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org