You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by Sergey Beryozkin <sb...@gmail.com> on 2010/08/06 12:33:03 UTC
Re: REST web service loading many classes for each request CXF 2.2.6
and jaxb-impl 2.1.5
Hi
On Fri, Aug 6, 2010 at 11:02 AM, Oliver Schmitz-Hennemann <
oli99sc@googlemail.com> wrote:
> Hi Everybody,
>
> we have implemented a REST webservice using CXF 2.2.6 and jaxb 2.1.5
> running
> in a tomcat 6.0.10.
> We are experiencing some serious problems concerning perm gen memory usage
> induced by continuously loading classes.
> Having traced around through the code we now know that the problem is
> induced by the fact, that the JaxbContext and within that some Injector
> implementations are only stored in WeakReferences. This each GC removes
> both. So, each Request after a GC will create a lot of classes.
>
I think there may've been some issue fixed in CXF 2.2.6/7 related to a
possible leak.
Dan pointed out the other day that WeakRefs don't make a difference in this
case given that JAXBContexts link to the Classes, so that should not cause
perm gen issues;
> Looking at the AbstractJAXBProvider in org.apache.cxf.jaxrs.provider, it
> seams possible to supply an JAXB Context somehow, but I could not figure
> out, how to do that.
>
>
> protected JAXBContext getJAXBContext(Class<?> type, Type genericType)
> throws JAXBException {
> if (mc != null) {
> ContextResolver<JAXBContext> resolver =
> mc.getResolver(ContextResolver.class, JAXBContext.class);
> if (resolver != null) {
> JAXBContext customContext = resolver.getContext(type);
> if (customContext != null) {
> return customContext;
> }
> }
> }
>
>
> Anybody any idea if this is the rigth solution, and how to make use of it?
>
> Sure, you can register a custom ContextResolver as a regular provider.
Alternatively, you can try to register a custom provider extending
JAXBElementProvider and override one of its createJAXBContext methods. Also
please try 2.2.7 or later and see if the issue has been resolved there
cheers, Sergey
> Oli
>
Re: REST web service loading many classes for each request CXF 2.2.6 and jaxb-impl 2.1.5
Posted by Daniel Kulp <dk...@apache.org>.
On Friday 13 August 2010 12:02:30 pm Sergey Beryozkin wrote:
> thanks, Dan applied it
Yea. I started trying to experiment to see if it would make sense to make the
maps non-static, but I decided getting 2.2.10 out was more important.
What I was hoping to accomplish is to have the maps non-static so the
JAXBElementProvider itself would hold onto them. Thus, if a war or something
is undeployed, the JAXBElementProvider would hopefully get GC'd and the
contexts cleared out and such. However, that only works if the
JAXBElementProvider itself isn't also held onto staticly, which, by default,
it is. :-( I'd like to get it so the JAXRS Client/Server stuff would each
get their own unique copy of the provider list so if they go out of scope,
they would get cleaned up. Something for another day.......
Dan
>
> cheers, Sergey
>
> On Thu, Aug 12, 2010 at 10:33 PM, Stefan Schubert <ma...@ste-net.de> wrote:
> > I added a simple HashMap patch in Jira
> > (https://issues.apache.org/jira/browse/CXF-2939).
> >
> > Cheers
> > Stefan
> > --
> > View this message in context:
> > http://cxf.547215.n5.nabble.com/REST-web-service-loading-many-classes-for
> > -each-request-CXF-2-2-6-and-jaxb-impl-2-1-5-tp2266472p2473688.html Sent
> > from the cxf-user mailing list archive at Nabble.com.
--
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog
Re: REST web service loading many classes for each request CXF 2.2.6
and jaxb-impl 2.1.5
Posted by Sergey Beryozkin <sb...@gmail.com>.
thanks, Dan applied it
cheers, Sergey
On Thu, Aug 12, 2010 at 10:33 PM, Stefan Schubert <ma...@ste-net.de> wrote:
>
> I added a simple HashMap patch in Jira
> (https://issues.apache.org/jira/browse/CXF-2939).
>
> Cheers
> Stefan
> --
> View this message in context:
> http://cxf.547215.n5.nabble.com/REST-web-service-loading-many-classes-for-each-request-CXF-2-2-6-and-jaxb-impl-2-1-5-tp2266472p2473688.html
> Sent from the cxf-user mailing list archive at Nabble.com.
>
[ANN]VTD-XML 2.9
Posted by Jimmy Zhang <cr...@comcast.net>.
VTD-XML 2.9, the next generation XML Processing API for SOA and Cloud computing, has been released. Please visit https://sourceforge.net/projects/vtd-xml/files/ to download the latest version.
a.. Strict Conformance
a.. VTD-XML now fully conforms to XML namespace 1.0 spec
b.. Performance Improvement
a.. Significantly improved parsing performance for small XML files
c.. Expand Core VTD-XML API
a.. Adds getPrefixString(), and toNormalizedString2()
d.. Cutting/Splitting
a.. Adds getSiblingElementFragment()
e.. A number of bug fixes and code enhancement including:
a.. Fixes a bug for reading very large XML documents on some platforms
b.. Fixes a bug in parsing processing instruction
c.. Fixes a bug in outputAndReparse()
Re: REST web service loading many classes for each request CXF
2.2.6 and jaxb-impl 2.1.5
Posted by Stefan Schubert <ma...@ste-net.de>.
I added a simple HashMap patch in Jira
(https://issues.apache.org/jira/browse/CXF-2939).
Cheers
Stefan
--
View this message in context: http://cxf.547215.n5.nabble.com/REST-web-service-loading-many-classes-for-each-request-CXF-2-2-6-and-jaxb-impl-2-1-5-tp2266472p2473688.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: REST web service loading many classes for each request CXF 2.2.6
and jaxb-impl 2.1.5
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi
On Thu, Aug 12, 2010 at 4:15 PM, Stefan Schubert <ma...@ste-net.de> wrote:
>
> Hi,
>
>
> Sergey Beryozkin-5 wrote:
> >
> > On Fri, Aug 6, 2010 at 11:02 AM, Oliver Schmitz-Hennemann <
> > oli99sc@googlemail.com> wrote:
> >
> >> we have implemented a REST webservice using CXF 2.2.6 and jaxb 2.1.5
> >> running
> >> in a tomcat 6.0.10.
> >> We are experiencing some serious problems concerning perm gen memory
> >> usage
> >> induced by continuously loading classes.
> >> Having traced around through the code we now know that the problem is
> >> induced by the fact, that the JaxbContext and within that some Injector
> >> implementations are only stored in WeakReferences. This each GC removes
> >> both. So, each Request after a GC will create a lot of classes.
> >>
> >
> > I think there may've been some issue fixed in CXF 2.2.6/7 related to a
> > possible leak.
> > Dan pointed out the other day that WeakRefs don't make a difference in
> > this
> > case given that JAXBContexts link to the Classes, so that should not
> cause
> > perm gen issues;
> >
>
> yes, of course WeakRefs make a difference. They do when the underlying
> technology (JAXB) has a memory leak as well.
>
> Let me clarify the problem:
> If a GC occurs, WeakRefs throw away the JAXBContext. So on the next
> occasion
> (where occasion could be one of billions of calls to a CXF/JAXB api) the
> JAXBContext has to be rebuilt from scratch.
>
JAXBElementResolver has two maps, one for keeping package contexts, another
one for class-specific contexts.
In the second one weak-refs do not work given that JAXBContexts reverse link
to Class keys, but yeah I can see now package contexts can be released
often, I thought they were keyed by classes too.
> JAXB (specifically
> com.sun.xml.bind.v2.runtime.reflect.opt.AccessorInjector)
> calls com.sun.xml.bind.v2.runtime.reflect.opt.Injector#find to try, if it
> already created a field or method accessor class. If not it injects a new
> one into the class loader via
> com.sun.xml.bind.v2.runtime.reflect.opt.Injector#inject.
> The problem is now that #inject caches the generated accessor in the
> Injector and find only looks into the Injector (and never again into the
> class loader, where it used to define the class). But once a GC occurs the
> Injector throws itself away, too (caching itself in some weak map).
>
> So it happens that each REST api call after a GC occurred a hundred new
> classes are created by JAXB - because JAXB on low-level forgets the classes
> it defines (and keeps them on high-level) and CXF forgets the JAXBContext
> so
> that the high-level memory of JAXB is erased as well.
>
> There are three possible solutions:
> - Fix the Injector: Actually very easy and my favorite solution. Just let
> the Injector look into the class loader as well when it cannot find the
> class in its own memory. BUT to have a bug fixed in JAXB actually sounds
> scary, how long would that take to get into a version?)
>
Not realistic I guess.
> - Enable CXF to keep only one JAXB context (and not to throw it away) - do
> not know how to do that
>
It is possible but may be problematic due to JAXRS allowing root resources
returning Object subresources so it is not possible to reliably introspect
all the service classes and build a single context. Perhaps there could be
an option there anyway for doing such an introspection. In fact you can do
it right now, use DataBindingProvider and delegate to CXF JAXB DataBinding,
please check the docs, there should be a link to the test showing how to do
it.
> - Use a workaround to disable bytecode generation by JAXB
> (-Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true) --> disables
> the whole bytecode magic Injector stuff and just does reflection - probably
> with a slight performance impact
>
it is a possibility
> - Do not use JAXB with CXF on a website with significant load
>
>
two more options :
- get rid of WeakHashMaps and use plain HashMaps
- register a custom JAXBElementProvider, override methods to do with
creating JAXB contexts and keep them in hash maps
If I can do anything to help resolving this, please let me know.
>
> would you consider supplying a patch to do with changing WeakHashMaps to
HashMaps in AbstractJaxbProvider ? This could be a good intermediate
solution ?
thanks, Sergey
> Cheers
> Stefan
> --
> View this message in context:
> http://cxf.547215.n5.nabble.com/REST-web-service-loading-many-classes-for-each-request-CXF-2-2-6-and-jaxb-impl-2-1-5-tp2266472p2473327.html
> Sent from the cxf-user mailing list archive at Nabble.com.
>
Re: REST web service loading many classes for each request CXF
2.2.6 and jaxb-impl 2.1.5
Posted by Stefan Schubert <ma...@ste-net.de>.
Hi,
Sergey Beryozkin-5 wrote:
>
> On Fri, Aug 6, 2010 at 11:02 AM, Oliver Schmitz-Hennemann <
> oli99sc@googlemail.com> wrote:
>
>> we have implemented a REST webservice using CXF 2.2.6 and jaxb 2.1.5
>> running
>> in a tomcat 6.0.10.
>> We are experiencing some serious problems concerning perm gen memory
>> usage
>> induced by continuously loading classes.
>> Having traced around through the code we now know that the problem is
>> induced by the fact, that the JaxbContext and within that some Injector
>> implementations are only stored in WeakReferences. This each GC removes
>> both. So, each Request after a GC will create a lot of classes.
>>
>
> I think there may've been some issue fixed in CXF 2.2.6/7 related to a
> possible leak.
> Dan pointed out the other day that WeakRefs don't make a difference in
> this
> case given that JAXBContexts link to the Classes, so that should not cause
> perm gen issues;
>
yes, of course WeakRefs make a difference. They do when the underlying
technology (JAXB) has a memory leak as well.
Let me clarify the problem:
If a GC occurs, WeakRefs throw away the JAXBContext. So on the next occasion
(where occasion could be one of billions of calls to a CXF/JAXB api) the
JAXBContext has to be rebuilt from scratch.
JAXB (specifically com.sun.xml.bind.v2.runtime.reflect.opt.AccessorInjector)
calls com.sun.xml.bind.v2.runtime.reflect.opt.Injector#find to try, if it
already created a field or method accessor class. If not it injects a new
one into the class loader via
com.sun.xml.bind.v2.runtime.reflect.opt.Injector#inject.
The problem is now that #inject caches the generated accessor in the
Injector and find only looks into the Injector (and never again into the
class loader, where it used to define the class). But once a GC occurs the
Injector throws itself away, too (caching itself in some weak map).
So it happens that each REST api call after a GC occurred a hundred new
classes are created by JAXB - because JAXB on low-level forgets the classes
it defines (and keeps them on high-level) and CXF forgets the JAXBContext so
that the high-level memory of JAXB is erased as well.
There are three possible solutions:
- Fix the Injector: Actually very easy and my favorite solution. Just let
the Injector look into the class loader as well when it cannot find the
class in its own memory. BUT to have a bug fixed in JAXB actually sounds
scary, how long would that take to get into a version?)
- Enable CXF to keep only one JAXB context (and not to throw it away) - do
not know how to do that
- Use a workaround to disable bytecode generation by JAXB
(-Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true) --> disables
the whole bytecode magic Injector stuff and just does reflection - probably
with a slight performance impact
- Do not use JAXB with CXF on a website with significant load
If I can do anything to help resolving this, please let me know.
Cheers
Stefan
--
View this message in context: http://cxf.547215.n5.nabble.com/REST-web-service-loading-many-classes-for-each-request-CXF-2-2-6-and-jaxb-impl-2-1-5-tp2266472p2473327.html
Sent from the cxf-user mailing list archive at Nabble.com.