You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by "ranadeep.sharma@gmail.com" <ra...@gmail.com> on 2016/06/14 17:38:27 UTC
Unwanted bunch of Bus Provider objects in HashMap occupying large
volumes of heap memory
*CXF Component/s:* Bus, JAX-RS
*Affects Version/s:* 3.1.0, 3.1.6
*Environment:* Redhat Enterprise Linux (Santiago), OpenJDK 7, Tomcat 7
We have an application with REST client components for making calls to
Backend web services. During our routine performance test, JProfiler tool
shows lots of Bus property entries (with keys named "bus.providers.set.")
populated while creating instances of ClientProviderFactory.
*public final class ClientProviderFactory extends ProviderFactory {
public static ClientProviderFactory createInstance(Bus bus) {
...
factory.setBusProviders();
...
}*
These Bus property entries seem to stay in heap for the whole duration of
the 6 hour run. In fact, around 100,000 entries occupying 13 MB of heap. In
short, GC doesn't seem to happening frequently enough to keep the heap usage
within limits.
Is this some sort of a bug or, lack of necessary configuration in CXF to
optimize the creation/cleanup of these objects?
--
View this message in context: http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: Unwanted bunch of Bus Provider objects in HashMap occupying large
volumes of heap memory
Posted by Sergey Beryozkin <sb...@gmail.com>.
I've removed the code setting this bus property. When it was initially
introduced few years back I believe a single factory was supporting
multiple endpoints, hence it was needed at a time.
Thanks, Sergey
On 20/06/16 20:49, Sergey Beryozkin wrote:
> Setting providers directly on a bus is a CXF optimization which was done
> mainly to help in cases where there are many jaxrs:server endpoints
> needing to share the same providers. It is not required to set the
> providers on the bus. Unless you have 3+ or even more jaxrs endpoints
> then the optimization is minimalistic. Simply refer to the same
> providers from the endpoints and the clients directly.
>
> The fact that the same bus is also used for per-request clients causes
> side-effects (this indeed will need to be addressed) and as I said you
> can have jaxrs:clients using their own Buses, while still referring to
> the same shared providers from within jaxrs:client/jaxrs:providers.
> Also rather than creating a new client per every request you can inject
> a thread-safe client and use this single client.
>
> The client interceptor code looks ok if it is what you prefer to do.
>
> Sergey
> On 20/06/16 19:41, ranadeep.sharma@gmail.com wrote:
>> Yes Sergey. You are right. Client is created for every request. And this
>> being the case, a new marker entry (factory hashCoded) is added
>> everytime,
>> although they are not used or, referred elsewhere. It looks like a CXF
>> defect.
>>
>> Regarding the providers, we need the providers on both hosted services as
>> well as clients (i.e. shared). So, your suggestion of registering the
>> providers only for clients does not help.
>>
>> If you feel the mentioned issue is a defect, could you suggest a
>> work-around
>> solution for now?
>>
>> Although I have tested a custom interceptor to do the clean-up of these
>> entries, I need your review to see if it would impact any part of the CXF
>> framework. Please find the source code below.
>>
>> *Java code*
>>
>> /public class PurgeFactoryHashcodeInterceptor extends
>> AbstractPhaseInterceptor<Message> {
>>
>> public PurgeFactoryHashcodeInterceptor() {
>>
>> super(Phase.PREPARE_SEND);
>> addBefore(MessageSenderInterceptor.class.getName());
>> }
>>
>> @Override
>> public void handleMessage(Message message) throws Fault {
>>
>> cleanUpBusProperties();
>> }
>>
>> @Override
>> public void handleFault(Message message) throws Fault {
>>
>> cleanUpBusProperties();
>> }
>>
>> private void cleanUpBusProperties() {
>>
>> Bus bus =
>> SpringContextProvider.getApplicationContext().getBean(Bus.class);
>> Map<String,Object> origBusPropMap = bus.getProperties();
>>
>> for(Map.Entry<String,Object> busPropEntry :
>> origBusPropMap.entrySet()) {
>> if(busPropEntry.getKey().startsWith("bus.providers.set.")) {
>> origBusPropMap.remove(busPropEntry.getKey());
>> }
>> }
>> }
>> }/
>>
>> *Spring configuration*
>>
>> /<bean id="purgeFtryHashcodeInterceptor"
>> class="com.mycompany.ws.interceptor.PurgeFactoryHashcodeInterceptor" />
>>
>> <cxf:bus>
>> <cxf:properties>
>> <entry key="org.apache.cxf.jaxrs.bus.providers">
>> <list>
>> <ref
>> bean="throwableExceptionMapper" />
>> <ref bean="customFilter1" />
>> </list>
>> </entry>
>> </cxf:properties>
>> <cxf:outInterceptors>
>> <ref bean="purgeFtryHashcodeInterceptor" />
>> </cxf:outInterceptors>
>> </cxf:bus>/
>>
>>
>>
>> --
>> View this message in context:
>> http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515p5769607.html
>>
>> Sent from the cxf-user mailing list archive at Nabble.com.
>>
>
>
--
Sergey Beryozkin
Talend Community Coders
http://coders.talend.com/
Re: Unwanted bunch of Bus Provider objects in HashMap occupying large
volumes of heap memory
Posted by Sergey Beryozkin <sb...@gmail.com>.
Setting providers directly on a bus is a CXF optimization which was done
mainly to help in cases where there are many jaxrs:server endpoints
needing to share the same providers. It is not required to set the
providers on the bus. Unless you have 3+ or even more jaxrs endpoints
then the optimization is minimalistic. Simply refer to the same
providers from the endpoints and the clients directly.
The fact that the same bus is also used for per-request clients causes
side-effects (this indeed will need to be addressed) and as I said you
can have jaxrs:clients using their own Buses, while still referring to
the same shared providers from within jaxrs:client/jaxrs:providers.
Also rather than creating a new client per every request you can inject
a thread-safe client and use this single client.
The client interceptor code looks ok if it is what you prefer to do.
Sergey
On 20/06/16 19:41, ranadeep.sharma@gmail.com wrote:
> Yes Sergey. You are right. Client is created for every request. And this
> being the case, a new marker entry (factory hashCoded) is added everytime,
> although they are not used or, referred elsewhere. It looks like a CXF
> defect.
>
> Regarding the providers, we need the providers on both hosted services as
> well as clients (i.e. shared). So, your suggestion of registering the
> providers only for clients does not help.
>
> If you feel the mentioned issue is a defect, could you suggest a work-around
> solution for now?
>
> Although I have tested a custom interceptor to do the clean-up of these
> entries, I need your review to see if it would impact any part of the CXF
> framework. Please find the source code below.
>
> *Java code*
>
> /public class PurgeFactoryHashcodeInterceptor extends
> AbstractPhaseInterceptor<Message> {
>
> public PurgeFactoryHashcodeInterceptor() {
>
> super(Phase.PREPARE_SEND);
> addBefore(MessageSenderInterceptor.class.getName());
> }
>
> @Override
> public void handleMessage(Message message) throws Fault {
>
> cleanUpBusProperties();
> }
>
> @Override
> public void handleFault(Message message) throws Fault {
>
> cleanUpBusProperties();
> }
>
> private void cleanUpBusProperties() {
>
> Bus bus =
> SpringContextProvider.getApplicationContext().getBean(Bus.class);
> Map<String,Object> origBusPropMap = bus.getProperties();
>
> for(Map.Entry<String,Object> busPropEntry :
> origBusPropMap.entrySet()) {
> if(busPropEntry.getKey().startsWith("bus.providers.set.")) {
> origBusPropMap.remove(busPropEntry.getKey());
> }
> }
> }
> }/
>
> *Spring configuration*
>
> /<bean id="purgeFtryHashcodeInterceptor"
> class="com.mycompany.ws.interceptor.PurgeFactoryHashcodeInterceptor" />
>
> <cxf:bus>
> <cxf:properties>
> <entry key="org.apache.cxf.jaxrs.bus.providers">
> <list>
> <ref bean="throwableExceptionMapper" />
> <ref bean="customFilter1" />
> </list>
> </entry>
> </cxf:properties>
> <cxf:outInterceptors>
> <ref bean="purgeFtryHashcodeInterceptor" />
> </cxf:outInterceptors>
> </cxf:bus>/
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515p5769607.html
> Sent from the cxf-user mailing list archive at Nabble.com.
>
--
Sergey Beryozkin
Talend Community Coders
http://coders.talend.com/
Re: Unwanted bunch of Bus Provider objects in HashMap occupying
large volumes of heap memory
Posted by "ranadeep.sharma@gmail.com" <ra...@gmail.com>.
Yes Sergey. You are right. Client is created for every request. And this
being the case, a new marker entry (factory hashCoded) is added everytime,
although they are not used or, referred elsewhere. It looks like a CXF
defect.
Regarding the providers, we need the providers on both hosted services as
well as clients (i.e. shared). So, your suggestion of registering the
providers only for clients does not help.
If you feel the mentioned issue is a defect, could you suggest a work-around
solution for now?
Although I have tested a custom interceptor to do the clean-up of these
entries, I need your review to see if it would impact any part of the CXF
framework. Please find the source code below.
*Java code*
/public class PurgeFactoryHashcodeInterceptor extends
AbstractPhaseInterceptor<Message> {
public PurgeFactoryHashcodeInterceptor() {
super(Phase.PREPARE_SEND);
addBefore(MessageSenderInterceptor.class.getName());
}
@Override
public void handleMessage(Message message) throws Fault {
cleanUpBusProperties();
}
@Override
public void handleFault(Message message) throws Fault {
cleanUpBusProperties();
}
private void cleanUpBusProperties() {
Bus bus =
SpringContextProvider.getApplicationContext().getBean(Bus.class);
Map<String,Object> origBusPropMap = bus.getProperties();
for(Map.Entry<String,Object> busPropEntry :
origBusPropMap.entrySet()) {
if(busPropEntry.getKey().startsWith("bus.providers.set.")) {
origBusPropMap.remove(busPropEntry.getKey());
}
}
}
}/
*Spring configuration*
/<bean id="purgeFtryHashcodeInterceptor"
class="com.mycompany.ws.interceptor.PurgeFactoryHashcodeInterceptor" />
<cxf:bus>
<cxf:properties>
<entry key="org.apache.cxf.jaxrs.bus.providers">
<list>
<ref bean="throwableExceptionMapper" />
<ref bean="customFilter1" />
</list>
</entry>
</cxf:properties>
<cxf:outInterceptors>
<ref bean="purgeFtryHashcodeInterceptor" />
</cxf:outInterceptors>
</cxf:bus>/
--
View this message in context: http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515p5769607.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: Unwanted bunch of Bus Provider objects in HashMap occupying large
volumes of heap memory
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi
Do you create a new client per every request ?
What might be happening is that in this case, when you have the
providers registered on the bus, every time a client is created it gets
a new ClientProviderFactory which in turn will mark on the current bus
that the bus providers have already been registered, using a factory
hash code - I think this may be the cause of the problem.
For now consider avoiding registering the providers on the bus - do it
on the client only. If the same bus (with the providers) is shared
between the clients and the server then create a client specific bus.
I'll reopen the issue next week
Cheers, Sergey
On 15/06/16 15:25, ranadeep.sharma@gmail.com wrote:
> Thanks Alexey.
>
> While waiting for answers to my earlier questions from the masters, I am
> thinking of adopting a CXF interceptor (as a work around) to nullify the
> contextCache reference, hoping that these unwanted map entries gets GCed
> more frequently.
>
> Your quick help is appreciated.
>
> Regards,
> Ranadeep.
> On Jun 15, 2016 3:09 AM, "Alexey Markevich [via CXF]" <
> ml-node+s547215n5769525h46@n5.nabble.com> wrote:
>
>> https://issues.apache.org/jira/browse/CXF-6823
>>
>> On Tue, 14 Jun 2016 20:38:27 +0300, [hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=5769525&i=0>
>> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=5769525&i=1>>
>> wrote:
>>
>>> *CXF Component/s:* Bus, JAX-RS
>>>
>>> *Affects Version/s:* 3.1.0, 3.1.6
>>>
>>> *Environment:* Redhat Enterprise Linux (Santiago), OpenJDK 7, Tomcat 7
>>>
>>> We have an application with REST client components for making calls to
>>> Backend web services. During our routine performance test, JProfiler
>> tool
>>> shows lots of Bus property entries (with keys named
>> "bus.providers.set.")
>>> populated while creating instances of ClientProviderFactory.
>>>
>>> *public final class ClientProviderFactory extends ProviderFactory {
>>>
>>> public static ClientProviderFactory createInstance(Bus bus) {
>>> ...
>>> factory.setBusProviders();
>>> ...
>>> }*
>>>
>>> These Bus property entries seem to stay in heap for the whole duration
>> of
>>> the 6 hour run. In fact, around 100,000 entries occupying 13 MB of heap.
>>
>>> In
>>> short, GC doesn't seem to happening frequently enough to keep the heap
>>> usage
>>> within limits.
>>>
>>> Is this some sort of a bug or, lack of necessary configuration in CXF to
>>> optimize the creation/cleanup of these objects?
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>>
>> http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515.html
>>> Sent from the cxf-user mailing list archive at Nabble.com.
>>
>>
>> --
>> Regards, Alexey.
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515p5769525.html
>> To unsubscribe from Unwanted bunch of Bus Provider objects in HashMap
>> occupying large volumes of heap memory, click here
>> <http://cxf.547215.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5769515&code=cmFuYWRlZXAuc2hhcm1hQGdtYWlsLmNvbXw1NzY5NTE1fC0xOTAyNzcwMDYz>
>> .
>> NAML
>> <http://cxf.547215.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515p5769533.html
> Sent from the cxf-user mailing list archive at Nabble.com.
>
--
Sergey Beryozkin
Talend Community Coders
http://coders.talend.com/
Re: Unwanted bunch of Bus Provider objects in HashMap occupying
large volumes of heap memory
Posted by "ranadeep.sharma@gmail.com" <ra...@gmail.com>.
Thanks Alexey.
While waiting for answers to my earlier questions from the masters, I am
thinking of adopting a CXF interceptor (as a work around) to nullify the
contextCache reference, hoping that these unwanted map entries gets GCed
more frequently.
Your quick help is appreciated.
Regards,
Ranadeep.
On Jun 15, 2016 3:09 AM, "Alexey Markevich [via CXF]" <
ml-node+s547215n5769525h46@n5.nabble.com> wrote:
> https://issues.apache.org/jira/browse/CXF-6823
>
> On Tue, 14 Jun 2016 20:38:27 +0300, [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=5769525&i=0>
> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=5769525&i=1>>
> wrote:
>
> > *CXF Component/s:* Bus, JAX-RS
> >
> > *Affects Version/s:* 3.1.0, 3.1.6
> >
> > *Environment:* Redhat Enterprise Linux (Santiago), OpenJDK 7, Tomcat 7
> >
> > We have an application with REST client components for making calls to
> > Backend web services. During our routine performance test, JProfiler
> tool
> > shows lots of Bus property entries (with keys named
> "bus.providers.set.")
> > populated while creating instances of ClientProviderFactory.
> >
> > *public final class ClientProviderFactory extends ProviderFactory {
> >
> > public static ClientProviderFactory createInstance(Bus bus) {
> > ...
> > factory.setBusProviders();
> > ...
> > }*
> >
> > These Bus property entries seem to stay in heap for the whole duration
> of
> > the 6 hour run. In fact, around 100,000 entries occupying 13 MB of heap.
>
> > In
> > short, GC doesn't seem to happening frequently enough to keep the heap
> > usage
> > within limits.
> >
> > Is this some sort of a bug or, lack of necessary configuration in CXF to
> > optimize the creation/cleanup of these objects?
> >
> >
> >
> > --
> > View this message in context:
> >
> http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515.html
> > Sent from the cxf-user mailing list archive at Nabble.com.
>
>
> --
> Regards, Alexey.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515p5769525.html
> To unsubscribe from Unwanted bunch of Bus Provider objects in HashMap
> occupying large volumes of heap memory, click here
> <http://cxf.547215.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5769515&code=cmFuYWRlZXAuc2hhcm1hQGdtYWlsLmNvbXw1NzY5NTE1fC0xOTAyNzcwMDYz>
> .
> NAML
> <http://cxf.547215.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515p5769533.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: Unwanted bunch of Bus Provider objects in HashMap occupying large
volumes of heap memory
Posted by Alexey Markevich <am...@talend.com>.
https://issues.apache.org/jira/browse/CXF-6823
On Tue, 14 Jun 2016 20:38:27 +0300, ranadeep.sharma@gmail.com
<ra...@gmail.com> wrote:
> *CXF Component/s:* Bus, JAX-RS
>
> *Affects Version/s:* 3.1.0, 3.1.6
>
> *Environment:* Redhat Enterprise Linux (Santiago), OpenJDK 7, Tomcat 7
>
> We have an application with REST client components for making calls to
> Backend web services. During our routine performance test, JProfiler tool
> shows lots of Bus property entries (with keys named "bus.providers.set.")
> populated while creating instances of ClientProviderFactory.
>
> *public final class ClientProviderFactory extends ProviderFactory {
>
> public static ClientProviderFactory createInstance(Bus bus) {
> ...
> factory.setBusProviders();
> ...
> }*
>
> These Bus property entries seem to stay in heap for the whole duration of
> the 6 hour run. In fact, around 100,000 entries occupying 13 MB of heap.
> In
> short, GC doesn't seem to happening frequently enough to keep the heap
> usage
> within limits.
>
> Is this some sort of a bug or, lack of necessary configuration in CXF to
> optimize the creation/cleanup of these objects?
>
>
>
> --
> View this message in context:
> http://cxf.547215.n5.nabble.com/Unwanted-bunch-of-Bus-Provider-objects-in-HashMap-occupying-large-volumes-of-heap-memory-tp5769515.html
> Sent from the cxf-user mailing list archive at Nabble.com.
--
Regards, Alexey.