You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-dev@axis.apache.org by Ann Robinson <ro...@us.ibm.com> on 2007/07/16 00:04:44 UTC
Proposal for fixing out-of-memory error JIRA axis2-2968
Hi, all,
I've been investigating an out-of-memory error that happens on the server
in some environments. In analyzing the java heap dumps, one of the biggest
consumers of memory is with the wsdl4j WSDLDefinition objects. The heap
dump indicates that the WSDLDefinition uses the xerces dom for underlying
support, particularly for schemas. This makes the WSDLDefinition very
heavy-weight.
The heap dump also shows that the WSDLDefinition objects of concern are the
ones being saved in the AxisService's ParameterInclude list. Comments in
the code (WSDL11ToAxisServiceBuilder) indicate that this is done so that,
if some component needs to utilize the WSDLDefinition, the component can
access it via a Parameter in the AxisService.
Is it possible to reduce the utilization of the WSDLDefinition?
Some ideas are:
(a) releasing it when it is no longer needed
- this might not be possible to determine
(b) putting a wrapper on the WSDLDefinition object
- so that the WSDLDefinition,or a portion of the WSDLDefinition,
can be released
- but if the WSDLDefinition is accessed after it was released,
the wrapper can reload the WSDLDefinition transparently to the
user
(c) create a layer for caching wsdl-related information
- this would allow for releasing memory based on some algorithm
and/or interface that could indicate what's no longer needed
Ann
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Amila Suriarachchi <am...@gmail.com>.
On 7/19/07, Ann Robinson <ro...@us.ibm.com> wrote:
>
> I have gone through all of the tests in the Axis2 build, including the
> simple WSDLClientTest.
> Are there some simple tests covering the variations for ?wsdl that are
> available?
>
No I think. but you can find lot of such wsdls from this site [1].
Please generate the Skelton for some wsdls see that. I will go through your
patch at the week end.
(now I am bit busy with fixing some issue and testing the axis2)
[1] http://131.107.72.15/ilab/
- Ann
>
>
>
> On 07/18/2007 12:15 AM, *"Amila Suriarachchi" <amilasuriarachchi@gmail.com
> >* wrote:
>
> Did you tried the ?wsdl for wsdls having
> 1. imported wsdl in it
> 2. imported and included schemas on it
> 3. inline schemas on it
> with useOriginalWSDL parameter set to true?
>
> And wsdl2java generated code for above senarios?
>
>
--
Amila Suriarachchi,
WSO2 Inc.
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Davanum Srinivas <da...@gmail.com>.
Please see my comments in JIRA. definitely NO for 1.3.
thanks,
dims
On 7/18/07, Ann Robinson <ro...@us.ibm.com> wrote:
>
>
> I have attached a patch to the JIRA axis2-2968 with the changes for the
> wrapper on the WSDL definition that is provided via the Parameter on the
> AxisService object. I would like some additional comments and tests before
> committing it.
>
> - Ann
>
> On 07/18/2007 03:26 PM, "Ann Robinson" <ro...@us.ibm.com> wrote:
>
> I have gone through all of the tests in the Axis2 build, including the
> simple WSDLClientTest.
> Are there some simple tests covering the variations for ?wsdl that are
> available?
>
> - Ann
>
> On 07/18/2007 12:15 AM, "Amila Suriarachchi" <am...@gmail.com>
> wrote:
>
> Did you tried the ?wsdl for wsdls having
> 1. imported wsdl in it
> 2. imported and included schemas on it
> 3. inline schemas on it
> with useOriginalWSDL parameter set to true?
>
> And wsdl2java generated code for above senarios?
>
>
>
>
>
>
>
--
Davanum Srinivas :: http://davanum.wordpress.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-dev-help@ws.apache.org
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Ann Robinson <ro...@us.ibm.com>.
I have attached a patch to the JIRA axis2-2968 with the changes for the
wrapper on the WSDL definition that is provided via the Parameter on the
AxisService object. I would like some additional comments and tests before
committing it.
- Ann
On 07/18/2007 03:26 PM, "Ann Robinson" <ro...@us.ibm.com> wrote:
I have gone through all of the tests in the Axis2 build, including the
simple WSDLClientTest.
Are there some simple tests covering the variations for ?wsdl that are
available?
- Ann
On 07/18/2007 12:15 AM, "Amila Suriarachchi" <am...@gmail.com>
wrote:
Did you tried the ?wsdl for wsdls having
1. imported wsdl in it
2. imported and included schemas on it
3. inline schemas on it
with useOriginalWSDL parameter set to true?
And wsdl2java generated code for above senarios?
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Ann Robinson <ro...@us.ibm.com>.
I have gone through all of the tests in the Axis2 build, including the
simple WSDLClientTest.
Are there some simple tests covering the variations for ?wsdl that are
available?
- Ann
On 07/18/2007 12:15 AM, "Amila Suriarachchi" <am...@gmail.com>
wrote:
Did you tried the ?wsdl for wsdls having
1. imported wsdl in it
2. imported and included schemas on it
3. inline schemas on it
with useOriginalWSDL parameter set to true?
And wsdl2java generated code for above senarios?
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Amila Suriarachchi <am...@gmail.com>.
On 7/17/07, Ann Robinson <ro...@us.ibm.com> wrote:
>
> The Out-Of-Memory (OOM) error happens on a server with a large number of
> applications. At deployment time, when the applications are loaded, the
> AxisService objects are created. For those applications where the loading
> ends up invoking the WSDL11ToAxisServiceBuilder, the wsdl is read in and
> saved as a Parameter on the AxisService object. So, the OOM happens before
> requests start flowing.
>
> I tried a solution where the WSDLDefinition object is saved via
> serialization, so that it could be re-loaded when needed after deployment
> time. The javax.wsdl.Definition interface is supposed to implement the
> java.io.Serializable interface. However, the WSDLDefinition object fails
> to serialize. I speculate that the WSDLDefinition implementation has some
> internal tables or structures that don't support the default java
> serialization.
>
> I tried another solution with a wrapper on the WSDLDefinition object. The
> wrapper, which is transparent to the user of the WSDLDefinition object,
> releases the heavyweight portions of the WSDLDefinition object. If a
> component accesses the WSDLDefinition via the Parameter from the AxisService
> object, the wrapper can reload the WSDLDefinition. This solution resolved
> the OOM problem - the server was able to load and start the applications
> successfully, and handle requests.
>
Did you tried the ?wsdl for wsdls having
1. imported wsdl in it
2. imported and included schemas on it
3. inline schemas on it
with useOriginalWSDL parameter set to true?
And wsdl2java generated code for above senarios?
I will create a patch with this solution so that it can be reviewed.
Ok. Please send a note to this thread as well.
> Ann
>
>
> On 07/16/2007 11:25 PM, *"Amila Suriarachchi" <amilasuriarachchi@gmail.com
> >* wrote:
>
> > On 7/16/07, *Sanjiva Weerawarana* <*s...@opensource.lk>>
> wrote:
> > This is used for when useOriginalWSDL is true right?
>
> yes.
>
>
> > I think the best thing is option (a) and have some policy for
> taking it out of the server
> > and re-loading it if it hasn't been hit for a while.
> >
> > I'm very much against option (c)- that's overkill for this problem
> IMO.
> >
> > In any case, why is it leading to an OOM error? The WSDL should be
> read
> > once and then hung onto ... so unless the number of services is
> growing
> > arbitrarily large there's really no reason for the memory usage to
> grow.
> > I'm not convinced this is the reason for the OOM (unless for some
> reason
> > we keep creating new Definitions and never release the old one).
>
>
> yes. this should not cause any OOM errors unless we load wsdl definition
> object per request. (then that is clearly an Axis2 bug). Ann are you getting
> the error at request processing time or deployment time?
>
> if it is at request processing time then we should creating wsdl
> definition objects per request and that should be an axis2 bug. What we
> supposed to do is to keep the original wsdl definition object and serialize
> it when user requested. But Here I think we can check for useOriginalWsdl
> parameter before storing that in memory to improve performance.
>
> if it is at deployment time none of above solutions would not solve the
> problem. Since we have to create the wsdlDefinition object at least once to
> create the axis object structure.
>
> thanks,
> Amila.
>
> >> On 15/07/07, Ann Robinson <*r...@us.ibm.com>>
> wrote:
> >> >
> >> >
> >> > Hi, all,
> >> > I've been investigating an out-of-memory error that happens
> on the server
> >> > in some environments. In analyzing the java heap dumps, one
> of the biggest
> >> > consumers of memory is with the wsdl4j WSDLDefinition objects.
> The heap
> >> > dump indicates that the WSDLDefinition uses the xerces dom for
> underlying
> >> > support, particularly for schemas. This makes the
> WSDLDefinition very
> >> > heavy-weight.
> >> >
> >> > The heap dump also shows that the WSDLDefinition objects of
> concern are the
> >> > ones being saved in the AxisService's ParameterInclude list.
> Comments in
> >> > the code (WSDL11ToAxisServiceBuilder) indicate that this is
> done so that, if
> >> > some component needs to utilize the WSDLDefinition, the
> component can access
> >> > it via a Parameter in the AxisService.
> >> >
> >> > Is it possible to reduce the utilization of the
> WSDLDefinition?
> >> > Some ideas are:
> >> > (a) releasing it when it is no longer needed
> >> > - this might not be possible to determine
> >> >
> >> > (b) putting a wrapper on the WSDLDefinition object
> >> > - so that the WSDLDefinition,or a portion of the
> WSDLDefinition,
> >> > can be released
> >> > - but if the WSDLDefinition is accessed after it was
> released
> >> > the wrapper can reload the WSDLDefinition
> transparently to
> >> > user
> >> >
> >> > (c) create a layer for caching wsdl-related information
> >> > - this would allow for releasing memory based on
> some algorithm
> >> > and/or interface that could indicate what's no
> longer needed
> >> >
> >> > Ann
>
>
--
Amila Suriarachchi,
WSO2 Inc.
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Ann Robinson <ro...@us.ibm.com>.
The Out-Of-Memory (OOM) error happens on a server with a large number of
applications. At deployment time, when the applications are loaded, the
AxisService objects are created. For those applications where the loading
ends up invoking the WSDL11ToAxisServiceBuilder, the wsdl is read in and
saved as a Parameter on the AxisService object. So, the OOM happens before
requests start flowing.
I tried a solution where the WSDLDefinition object is saved via
serialization, so that it could be re-loaded when needed after deployment
time. The javax.wsdl.Definition interface is supposed to implement the
java.io.Serializable interface. However, the WSDLDefinition object fails to
serialize. I speculate that the WSDLDefinition implementation has some
internal tables or structures that don't support the default java
serialization.
I tried another solution with a wrapper on the WSDLDefinition object. The
wrapper, which is transparent to the user of the WSDLDefinition object,
releases the heavyweight portions of the WSDLDefinition object. If a
component accesses the WSDLDefinition via the Parameter from the
AxisService object, the wrapper can reload the WSDLDefinition. This
solution resolved the OOM problem - the server was able to load and start
the applications successfully, and handle requests.
I will create a patch with this solution so that it can be reviewed.
Ann
On 07/16/2007 11:25 PM, "Amila Suriarachchi" <am...@gmail.com>
wrote:
> On 7/16/07, Sanjiva Weerawarana <sa...@opensource.lk> wrote:
> This is used for when useOriginalWSDL is true right?
yes.
> I think the best thing is option (a) and have some policy for taking it
out of the server
> and re-loading it if it hasn't been hit for a while.
>
> I'm very much against option (c)- that's overkill for this problem IMO.
>
> In any case, why is it leading to an OOM error? The WSDL should be read
> once and then hung onto ... so unless the number of services is growing
> arbitrarily large there's really no reason for the memory usage to
grow.
> I'm not convinced this is the reason for the OOM (unless for some
reason
> we keep creating new Definitions and never release the old one).
yes. this should not cause any OOM errors unless we load wsdl definition
object per request. (then that is clearly an Axis2 bug). Ann are you
getting the error at request processing time or deployment time?
if it is at request processing time then we should creating wsdl definition
objects per request and that should be an axis2 bug. What we supposed to do
is to keep the original wsdl definition object and serialize it when user
requested. But Here I think we can check for useOriginalWsdl parameter
before storing that in memory to improve performance.
if it is at deployment time none of above solutions would not solve the
problem. Since we have to create the wsdlDefinition object at least once to
create the axis object structure.
thanks,
Amila.
>> On 15/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
>> >
>> >
>> > Hi, all,
>> > I've been investigating an out-of-memory error that happens on the
server
>> > in some environments. In analyzing the java heap dumps, one of the
biggest
>> > consumers of memory is with the wsdl4j WSDLDefinition objects. The
heap
>> > dump indicates that the WSDLDefinition uses the xerces dom for
underlying
>> > support, particularly for schemas. This makes the WSDLDefinition
very
>> > heavy-weight.
>> >
>> > The heap dump also shows that the WSDLDefinition objects of
concern are the
>> > ones being saved in the AxisService's ParameterInclude list.
Comments in
>> > the code (WSDL11ToAxisServiceBuilder) indicate that this is done so
that, if
>> > some component needs to utilize the WSDLDefinition, the component
can access
>> > it via a Parameter in the AxisService.
>> >
>> > Is it possible to reduce the utilization of the WSDLDefinition?
>> > Some ideas are:
>> > (a) releasing it when it is no longer needed
>> > - this might not be possible to determine
>> >
>> > (b) putting a wrapper on the WSDLDefinition object
>> > - so that the WSDLDefinition,or a portion of the
WSDLDefinition,
>> > can be released
>> > - but if the WSDLDefinition is accessed after it was
released
>> > the wrapper can reload the WSDLDefinition transparently
to
>> > user
>> >
>> > (c) create a layer for caching wsdl-related information
>> > - this would allow for releasing memory based on some
algorithm
>> > and/or interface that could indicate what's no longer
needed
>> >
>> > Ann
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Amila Suriarachchi <am...@gmail.com>.
On 7/16/07, Sanjiva Weerawarana <sa...@opensource.lk> wrote:
>
> This is used for when useOriginalWSDL is true right?
yes.
I think the best
> thing is option (a) and have some policy for taking it out of the server
> and re-loading it if it hasn't been hit for a while.
>
> I'm very much against option (c)- that's overkill for this problem IMO.
>
> In any case, why is it leading to an OOM error? The WSDL should be read
> once and then hung onto ... so unless the number of services is growing
> arbitrarily large there's really no reason for the memory usage to grow.
> I'm not convinced this is the reason for the OOM (unless for some reason
> we keep creating new Definitions and never release the old one).
yes. this should not cause any OOM errors unless we load wsdl definition
object per request. (then that is clearly an Axis2 bug). Ann are you getting
the error at request processing time or deployment time?
if it is at request processing time then we should creating wsdl definition
objects per request and that should be an axis2 bug. What we supposed to do
is to keep the original wsdl definition object and serialize it when user
requested. But Here I think we can check for useOriginalWsdl parameter
before storing that in memory to improve performance.
if it is at deployment time none of above solutions would not solve the
problem. Since we have to create the wsdlDefinition object at least once to
create the axis object structure.
thanks,
Amila.
Sanjiva.
>
> David Illsley wrote:
> > While in general I agree that deprecating interfaces is the way to go,
> > for something which feels like such an implementation detail I think
> > we should consider just fixing it straight off.
> >
> > Anyone else have any thoughts? Anyone using this outside of Axis2?
> > Cheers,
> > David
> >
> > On 16/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
> >>
> >>
> >> Hi, David,
> >> I agree that (c) would be the best solution for the long term. As I
> >> look
> >> at this problem more, I think it will be necessary to do (b) in the
> >> short-term because the WSDLDefinition can be accessed via a Parameter
> >> on the
> >> AxisService - this is effectively an interface and I want to avoid
> >> changing
> >> interfaces. Doing (b) would allow this interface to continue to be
> >> supported
> >> until we can deprecate it and replace it with (c).
> >> Ann
> >>
> >> On 07/16/2007 04:07 AM "David Illsley" <da...@gmail.com> wrote
> >>
> >>
> >> (c) sounds like the best long term solution to me. It looks like the
> >> objects are just used to serve up ?wsdl and MEX requests, which only
> >> requires the host/port to be modified. We'd probably get better
> >> performance if we just wrote the definitions to a temporary file and
> >> used a templating solution to modify the address on each request.
> >> David
> >>
> >> On 15/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
> >> >
> >> >
> >> > Hi, all,
> >> > I've been investigating an out-of-memory error that happens on the
> >> server
> >> > in some environments. In analyzing the java heap dumps, one of the
> >> biggest
> >> > consumers of memory is with the wsdl4j WSDLDefinition objects.
> >> The heap
> >> > dump indicates that the WSDLDefinition uses the xerces dom for
> >> underlying
> >> > support, particularly for schemas. This makes the WSDLDefinition
> very
> >> > heavy-weight.
> >> >
> >> > The heap dump also shows that the WSDLDefinition objects of
> >> concern are
> >> the
> >> > ones being saved in the AxisService's ParameterInclude list.
> >> Comments in
> >> > the code (WSDL11ToAxisServiceBuilder) indicate that this is done so
> >> that,
> >> if
> >> > some component needs to utilize the WSDLDefinition, the component
> can
> >> access
> >> > it via a Parameter in the AxisService.
> >> >
> >> > Is it possible to reduce the utilization of the WSDLDefinition?
> >> > Some ideas are:
> >> > (a) releasing it when it is no longer needed
> >> > - this might not be possible to determine
> >> >
> >> > (b) putting a wrapper on the WSDLDefinition object
> >> >
> >> > - so that the WSDLDefinition,or a portion of the
> >> WSDLDefinition,
> >> > can be released
> >> > - but if the WSDLDefinition is accessed after it was
> >> released,
> >> > the wrapper can reload the WSDLDefinition transparently
> to
> >> the
> >> > user
> >> >
> >> > (c) create a layer for caching wsdl-related information
> >> > - this would allow for releasing memory based on some
> >> algorithm
> >> > and/or interface that could indicate what's no longer
> >> needed
> >> >
> >> > Ann
> >>
> >>
> >> --
> >> David Illsley - IBM Web Services Development
> >>
> >>
> >>
> >>
> >
> >
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder & Director; Lanka Software Foundation; http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Director; Open Source Initiative; http://www.opensource.org/
> Member; Apache Software Foundation; http://www.apache.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-dev-help@ws.apache.org
>
>
--
Amila Suriarachchi,
WSO2 Inc.
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by David Illsley <da...@gmail.com>.
I'm not sure a 'caching layer' needs to be heavyweight - we can simply
serialise the WSDL to temporary storage (or use the orifginal!) and do
a quick XSLT when serving up the ?wsdl rather than serialising from
the WSDL4J object.
I'm not pushing this hard, it just seems reasonable in my head.
David
On 17/07/07, Sanjiva Weerawarana <sa...@opensource.lk> wrote:
> This is used for when useOriginalWSDL is true right? I think the best
> thing is option (a) and have some policy for taking it out of the server
> and re-loading it if it hasn't been hit for a while.
>
> I'm very much against option (c)- that's overkill for this problem IMO.
>
> In any case, why is it leading to an OOM error? The WSDL should be read
> once and then hung onto ... so unless the number of services is growing
> arbitrarily large there's really no reason for the memory usage to grow.
> I'm not convinced this is the reason for the OOM (unless for some reason
> we keep creating new Definitions and never release the old one).
>
> Sanjiva.
>
> David Illsley wrote:
> > While in general I agree that deprecating interfaces is the way to go,
> > for something which feels like such an implementation detail I think
> > we should consider just fixing it straight off.
> >
> > Anyone else have any thoughts? Anyone using this outside of Axis2?
> > Cheers,
> > David
> >
> > On 16/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
> >>
> >>
> >> Hi, David,
> >> I agree that (c) would be the best solution for the long term. As I
> >> look
> >> at this problem more, I think it will be necessary to do (b) in the
> >> short-term because the WSDLDefinition can be accessed via a Parameter
> >> on the
> >> AxisService - this is effectively an interface and I want to avoid
> >> changing
> >> interfaces. Doing (b) would allow this interface to continue to be
> >> supported
> >> until we can deprecate it and replace it with (c).
> >> Ann
> >>
> >> On 07/16/2007 04:07 AM "David Illsley" <da...@gmail.com> wrote
> >>
> >>
> >> (c) sounds like the best long term solution to me. It looks like the
> >> objects are just used to serve up ?wsdl and MEX requests, which only
> >> requires the host/port to be modified. We'd probably get better
> >> performance if we just wrote the definitions to a temporary file and
> >> used a templating solution to modify the address on each request.
> >> David
> >>
> >> On 15/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
> >> >
> >> >
> >> > Hi, all,
> >> > I've been investigating an out-of-memory error that happens on the
> >> server
> >> > in some environments. In analyzing the java heap dumps, one of the
> >> biggest
> >> > consumers of memory is with the wsdl4j WSDLDefinition objects.
> >> The heap
> >> > dump indicates that the WSDLDefinition uses the xerces dom for
> >> underlying
> >> > support, particularly for schemas. This makes the WSDLDefinition very
> >> > heavy-weight.
> >> >
> >> > The heap dump also shows that the WSDLDefinition objects of
> >> concern are
> >> the
> >> > ones being saved in the AxisService's ParameterInclude list.
> >> Comments in
> >> > the code (WSDL11ToAxisServiceBuilder) indicate that this is done so
> >> that,
> >> if
> >> > some component needs to utilize the WSDLDefinition, the component can
> >> access
> >> > it via a Parameter in the AxisService.
> >> >
> >> > Is it possible to reduce the utilization of the WSDLDefinition?
> >> > Some ideas are:
> >> > (a) releasing it when it is no longer needed
> >> > - this might not be possible to determine
> >> >
> >> > (b) putting a wrapper on the WSDLDefinition object
> >> >
> >> > - so that the WSDLDefinition,or a portion of the
> >> WSDLDefinition,
> >> > can be released
> >> > - but if the WSDLDefinition is accessed after it was
> >> released,
> >> > the wrapper can reload the WSDLDefinition transparently to
> >> the
> >> > user
> >> >
> >> > (c) create a layer for caching wsdl-related information
> >> > - this would allow for releasing memory based on some
> >> algorithm
> >> > and/or interface that could indicate what's no longer
> >> needed
> >> >
> >> > Ann
> >>
> >>
> >> --
> >> David Illsley - IBM Web Services Development
> >>
> >>
> >>
> >>
> >
> >
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder & Director; Lanka Software Foundation; http://www.opensource.lk/
> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> Director; Open Source Initiative; http://www.opensource.org/
> Member; Apache Software Foundation; http://www.apache.org/
> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-dev-help@ws.apache.org
>
>
--
David Illsley - IBM Web Services Development
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-dev-help@ws.apache.org
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Sanjiva Weerawarana <sa...@opensource.lk>.
This is used for when useOriginalWSDL is true right? I think the best
thing is option (a) and have some policy for taking it out of the server
and re-loading it if it hasn't been hit for a while.
I'm very much against option (c)- that's overkill for this problem IMO.
In any case, why is it leading to an OOM error? The WSDL should be read
once and then hung onto ... so unless the number of services is growing
arbitrarily large there's really no reason for the memory usage to grow.
I'm not convinced this is the reason for the OOM (unless for some reason
we keep creating new Definitions and never release the old one).
Sanjiva.
David Illsley wrote:
> While in general I agree that deprecating interfaces is the way to go,
> for something which feels like such an implementation detail I think
> we should consider just fixing it straight off.
>
> Anyone else have any thoughts? Anyone using this outside of Axis2?
> Cheers,
> David
>
> On 16/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
>>
>>
>> Hi, David,
>> I agree that (c) would be the best solution for the long term. As I
>> look
>> at this problem more, I think it will be necessary to do (b) in the
>> short-term because the WSDLDefinition can be accessed via a Parameter
>> on the
>> AxisService - this is effectively an interface and I want to avoid
>> changing
>> interfaces. Doing (b) would allow this interface to continue to be
>> supported
>> until we can deprecate it and replace it with (c).
>> Ann
>>
>> On 07/16/2007 04:07 AM "David Illsley" <da...@gmail.com> wrote
>>
>>
>> (c) sounds like the best long term solution to me. It looks like the
>> objects are just used to serve up ?wsdl and MEX requests, which only
>> requires the host/port to be modified. We'd probably get better
>> performance if we just wrote the definitions to a temporary file and
>> used a templating solution to modify the address on each request.
>> David
>>
>> On 15/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
>> >
>> >
>> > Hi, all,
>> > I've been investigating an out-of-memory error that happens on the
>> server
>> > in some environments. In analyzing the java heap dumps, one of the
>> biggest
>> > consumers of memory is with the wsdl4j WSDLDefinition objects.
>> The heap
>> > dump indicates that the WSDLDefinition uses the xerces dom for
>> underlying
>> > support, particularly for schemas. This makes the WSDLDefinition very
>> > heavy-weight.
>> >
>> > The heap dump also shows that the WSDLDefinition objects of
>> concern are
>> the
>> > ones being saved in the AxisService's ParameterInclude list.
>> Comments in
>> > the code (WSDL11ToAxisServiceBuilder) indicate that this is done so
>> that,
>> if
>> > some component needs to utilize the WSDLDefinition, the component can
>> access
>> > it via a Parameter in the AxisService.
>> >
>> > Is it possible to reduce the utilization of the WSDLDefinition?
>> > Some ideas are:
>> > (a) releasing it when it is no longer needed
>> > - this might not be possible to determine
>> >
>> > (b) putting a wrapper on the WSDLDefinition object
>> >
>> > - so that the WSDLDefinition,or a portion of the
>> WSDLDefinition,
>> > can be released
>> > - but if the WSDLDefinition is accessed after it was
>> released,
>> > the wrapper can reload the WSDLDefinition transparently to
>> the
>> > user
>> >
>> > (c) create a layer for caching wsdl-related information
>> > - this would allow for releasing memory based on some
>> algorithm
>> > and/or interface that could indicate what's no longer
>> needed
>> >
>> > Ann
>>
>>
>> --
>> David Illsley - IBM Web Services Development
>>
>>
>>
>>
>
>
--
Sanjiva Weerawarana, Ph.D.
Founder & Director; Lanka Software Foundation; http://www.opensource.lk/
Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
Director; Open Source Initiative; http://www.opensource.org/
Member; Apache Software Foundation; http://www.apache.org/
Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-dev-help@ws.apache.org
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by David Illsley <da...@gmail.com>.
While in general I agree that deprecating interfaces is the way to go,
for something which feels like such an implementation detail I think
we should consider just fixing it straight off.
Anyone else have any thoughts? Anyone using this outside of Axis2?
Cheers,
David
On 16/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
>
>
> Hi, David,
> I agree that (c) would be the best solution for the long term. As I look
> at this problem more, I think it will be necessary to do (b) in the
> short-term because the WSDLDefinition can be accessed via a Parameter on the
> AxisService - this is effectively an interface and I want to avoid changing
> interfaces. Doing (b) would allow this interface to continue to be supported
> until we can deprecate it and replace it with (c).
> Ann
>
> On 07/16/2007 04:07 AM "David Illsley" <da...@gmail.com> wrote
>
>
> (c) sounds like the best long term solution to me. It looks like the
> objects are just used to serve up ?wsdl and MEX requests, which only
> requires the host/port to be modified. We'd probably get better
> performance if we just wrote the definitions to a temporary file and
> used a templating solution to modify the address on each request.
> David
>
> On 15/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
> >
> >
> > Hi, all,
> > I've been investigating an out-of-memory error that happens on the
> server
> > in some environments. In analyzing the java heap dumps, one of the
> biggest
> > consumers of memory is with the wsdl4j WSDLDefinition objects. The heap
> > dump indicates that the WSDLDefinition uses the xerces dom for underlying
> > support, particularly for schemas. This makes the WSDLDefinition very
> > heavy-weight.
> >
> > The heap dump also shows that the WSDLDefinition objects of concern are
> the
> > ones being saved in the AxisService's ParameterInclude list. Comments in
> > the code (WSDL11ToAxisServiceBuilder) indicate that this is done so that,
> if
> > some component needs to utilize the WSDLDefinition, the component can
> access
> > it via a Parameter in the AxisService.
> >
> > Is it possible to reduce the utilization of the WSDLDefinition?
> > Some ideas are:
> > (a) releasing it when it is no longer needed
> > - this might not be possible to determine
> >
> > (b) putting a wrapper on the WSDLDefinition object
> >
> > - so that the WSDLDefinition,or a portion of the
> WSDLDefinition,
> > can be released
> > - but if the WSDLDefinition is accessed after it was released,
> > the wrapper can reload the WSDLDefinition transparently to
> the
> > user
> >
> > (c) create a layer for caching wsdl-related information
> > - this would allow for releasing memory based on some algorithm
> > and/or interface that could indicate what's no longer needed
> >
> > Ann
>
>
> --
> David Illsley - IBM Web Services Development
>
>
>
>
--
David Illsley - IBM Web Services Development
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-dev-help@ws.apache.org
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by Ann Robinson <ro...@us.ibm.com>.
Hi, David,
I agree that (c) would be the best solution for the long term. As I look
at this problem more, I think it will be necessary to do (b) in the
short-term because the WSDLDefinition can be accessed via a Parameter on
the AxisService - this is effectively an interface and I want to avoid
changing interfaces. Doing (b) would allow this interface to continue to be
supported until we can deprecate it and replace it with (c).
Ann
On 07/16/2007 04:07 AM "David Illsley" <da...@gmail.com> wrote
(c) sounds like the best long term solution to me. It looks like the
objects are just used to serve up ?wsdl and MEX requests, which only
requires the host/port to be modified. We'd probably get better
performance if we just wrote the definitions to a temporary file and
used a templating solution to modify the address on each request.
David
On 15/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
>
>
> Hi, all,
> I've been investigating an out-of-memory error that happens on the
server
> in some environments. In analyzing the java heap dumps, one of the
biggest
> consumers of memory is with the wsdl4j WSDLDefinition objects. The heap
> dump indicates that the WSDLDefinition uses the xerces dom for underlying
> support, particularly for schemas. This makes the WSDLDefinition very
> heavy-weight.
>
> The heap dump also shows that the WSDLDefinition objects of concern are
the
> ones being saved in the AxisService's ParameterInclude list. Comments in
> the code (WSDL11ToAxisServiceBuilder) indicate that this is done so that,
if
> some component needs to utilize the WSDLDefinition, the component can
access
> it via a Parameter in the AxisService.
>
> Is it possible to reduce the utilization of the WSDLDefinition?
> Some ideas are:
> (a) releasing it when it is no longer needed
> - this might not be possible to determine
>
> (b) putting a wrapper on the WSDLDefinition object
>
> - so that the WSDLDefinition,or a portion of the
WSDLDefinition,
> can be released
> - but if the WSDLDefinition is accessed after it was released,
> the wrapper can reload the WSDLDefinition transparently to
the
> user
>
> (c) create a layer for caching wsdl-related information
> - this would allow for releasing memory based on some algorithm
> and/or interface that could indicate what's no longer needed
>
> Ann
--
David Illsley - IBM Web Services Development
Re: Proposal for fixing out-of-memory error JIRA axis2-2968
Posted by David Illsley <da...@gmail.com>.
(c) sounds like the best long term solution to me. It looks like the
objects are just used to serve up ?wsdl and MEX requests, which only
requires the host/port to be modified. We'd probably get better
performance if we just wrote the definitions to a temporary file and
used a templating solution to modify the address on each request.
David
On 15/07/07, Ann Robinson <ro...@us.ibm.com> wrote:
>
>
> Hi, all,
> I've been investigating an out-of-memory error that happens on the server
> in some environments. In analyzing the java heap dumps, one of the biggest
> consumers of memory is with the wsdl4j WSDLDefinition objects. The heap
> dump indicates that the WSDLDefinition uses the xerces dom for underlying
> support, particularly for schemas. This makes the WSDLDefinition very
> heavy-weight.
>
> The heap dump also shows that the WSDLDefinition objects of concern are the
> ones being saved in the AxisService's ParameterInclude list. Comments in
> the code (WSDL11ToAxisServiceBuilder) indicate that this is done so that, if
> some component needs to utilize the WSDLDefinition, the component can access
> it via a Parameter in the AxisService.
>
> Is it possible to reduce the utilization of the WSDLDefinition?
> Some ideas are:
> (a) releasing it when it is no longer needed
> - this might not be possible to determine
>
> (b) putting a wrapper on the WSDLDefinition object
>
> - so that the WSDLDefinition,or a portion of the WSDLDefinition,
> can be released
> - but if the WSDLDefinition is accessed after it was released,
> the wrapper can reload the WSDLDefinition transparently to the
> user
>
> (c) create a layer for caching wsdl-related information
> - this would allow for releasing memory based on some algorithm
> and/or interface that could indicate what's no longer needed
>
> Ann
--
David Illsley - IBM Web Services Development
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-dev-help@ws.apache.org