BusFactory.localBus, ThreadLocal, spring, & permgen out of memory error

[Ben Dean <Be...@OntarioSystems.com>]

Hello CXF users,

I've been  working on figuring out why our JBoss servers keep going down with permgen out of memory exceptions when we redeploy our war files. To do this I had been using a profiler to inspect the WebAppClassloader to find out what was keeping it from being garbage collected. One such culprit was the class org.apache.cxf.BusFactory

The BusFactory has a ThreadLocal in which it stores a copy of CXFBusImpl. However, this isn't getting cleaned up properly when the war is undeployed. I noticed that CXFBusImpl has a shutdown method that calls BusFactory.setDefaultBus(null) which in turn sets the value stored in the ThreadLocal to null. However, this doesn't seem to be getting called.

The way we are using CXF from spring is with the following WEB-INF/services.xml file in our war:
  <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:amq="http://activemq.org/config/1.0" xmlns:jaxws="http://cxf.apache.org/jaxws"
    xmlns:http-conf="http://cxf.apache.org/transports/http/configuration"
    xsi:schemaLocation="
      http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
      http://cxf.apache.org/transports/http/configuration http://cxf.apache.org/schemas/configuration/http-conf.xsd
      http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd
      http://activemq.org/config/1.0 http://activemq.apache.org/schema/core/activemq-core-5.0.0.xsd"
    default-autowire="byType">

    <import resource="classpath:META-INF/cxf/cxf.xml" />
    <import resource="classpath:META-INF/cxf/cxf-extension-soap.xml" />
    <import resource="classpath:META-INF/cxf/cxf-servlet.xml" />

    <jaxws:endpoint id="helloService" implementor="#helloSvc" address="/HelloService" />
  </beans>

I'm not really sure why this Bus.shutdown() isn't getting called or even who is responsible for calling it. I tried doing something along the lines of:
  WebApplicationContext webContext = WebApplicationContextUtils.getRequiredWebApplicationContext(context);
  if (webContext.containsBean("cxf")) {
    CXFBusImpl cxfBus = (CXFBusImpl) webContext.getBean("cxf");
    cxfBus.shutdown(true);
  }

But that didn't work. What I eventually ended up doing was to have the following hack in the shutdown sequence of our webapp:
  Field field = org.apache.cxf.BussFactory.class.getDeclaredField("localBus");
  field.setAccessible(true);
  ThreadLocal<?> localBus = (ThreadLocal<?>) field.get(null);
  localBus.remove();

This did work but obviously it is a bit of an ugly hack. Is there something that needs to be included in our service.xml file to tell spring how to cleanup CXF? Is this maybe a bug in CXF that the CXFBusImpl.shutdown(Boolean) just isn't getting called at all or maybe that in BusFactory.setThreadDefaultBus(Bus) that there should be some check such as:
  if (bus == null) {
    localBus.remove();
  } else {
    localBus.set(bus);
  }

Or maybe some combination of all of these. I'm reasonably new to CXF, Spring, and all this stuff so I apologize ahead of time if there is some obvious solution that I just didn't come across but I've been trying to scour the CXF and Spring documentation for some idea of the correct way to do this and why BusFactory is keeping its ThreadLocal but I can't figure it out.

Thanks!

Ben Dean
Software Engineer
Ontario Systems, LLC


Attention: This message and all attachments are private and may contain
information that is confidential and privileged. If you received this
message in error, please notify the sender by reply email and delete
the message immediately.