You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by jian wu <he...@gmail.com> on 2008/10/29 18:23:49 UTC

Generated CXF Client is Blocked in Tomcat Container, is this a possible deadlock?

Hi,

I'm running a WSDL2JAVA generated CXF WS Client in Tomcat 6.0.10 to
call external WS
for the incoming request.

I'm using one Serivce Object to create one new port object for each WS
Call per thread,
it seems that, eventually all the http threads are blocked at:
  org.apache.cxf.jaxb.JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530)

I'm using CXF 2.1.1,  the Tomcat Server is running on Linux with IBM
64bit JVM. The
following is the thread dump snippet:

3LKMONOBJECT
org/apache/cxf/common/util/CacheMap@00000080D941B830/00000080D941B848:
Flat locked by "http-7101-2" (0x000000001129F300), entry count 1
3LKWAITERQ            Waiting to enter:
3LKWAITER                "http-7101-1" (0x0000000010870F00)
3LKWAITER                "http-7101-3" (0x00000000112A5C00)
3LKWAITER                "http-7101-4" (0x00000000128C4300)
3LKWAITER                "http-7101-5" (0x00000000137B9D00)
3LKWAITER                "http-7101-6" (0x00000000137CAF00)
3LKWAITER                "http-7101-7" (0x00000000137DF000)
3LKWAITER                "http-7101-8" (0x00000000137EF200)
3LKWAITER                "http-7101-9" (0x0000000013800300)
3LKWAITER                "http-7101-10" (0x000000001335A600)
(... ...)
3LKWAITER                "http-7101-60" (0x0000000014797300)

3XMTHREADINFO      "http-7101-2" (TID:0x000000001129F300,
sys_thread_t:0x0000000010862110, state:CW, native
ID:0x00000000000057CA) prio=5
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/TypeInfoSetImpl.getElementInfo(TypeInfoSetImpl.java:275(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeTypeInfoSetImpl.getElementInfo(RuntimeTypeInfoSetImpl.java:102)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeTypeInfoSetImpl.getElementInfo(RuntimeTypeInfoSetImpl.java:59)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.addGenericElement(ReferencePropertyInfoImpl.java:196)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.calcTypes(ReferencePropertyInfoImpl.java:158(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.getElements(ReferencePropertyInfoImpl.java:112)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.getElements(RuntimeReferencePropertyInfoImpl.java:67)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.ref(ReferencePropertyInfoImpl.java:103)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.ref(RuntimeReferencePropertyInfoImpl.java:71)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.ref(RuntimeReferencePropertyInfoImpl.java:51)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:255(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RegistryInfoImpl.<init>(RegistryInfoImpl.java:99)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.addRegistry(ModelBuilder.java:357(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:218(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/TypeRefImpl.calcRef(TypeRefImpl.java:92)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/TypeRefImpl.getTarget(TypeRefImpl.java:69)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeTypeRefImpl.getTarget(RuntimeTypeRefImpl.java:58)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeTypeRefImpl.getTarget(RuntimeTypeRefImpl.java:51)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ElementPropertyInfoImpl$1.get(ElementPropertyInfoImpl.java:74)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ElementPropertyInfoImpl$1.get(ElementPropertyInfoImpl.java:77)
4XESTACKTRACE          at
java/util/AbstractList$Itr.next(AbstractList.java:437(Compiled Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:255(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
Code))
4XESTACKTRACE          at
com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:330)
4XESTACKTRACE          at
com/sun/xml/bind/v2/runtime/JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:432)
4XESTACKTRACE          at
com/sun/xml/bind/v2/runtime/JAXBContextImpl.<init>(JAXBContextImpl.java:286)
4XESTACKTRACE          at
com/sun/xml/bind/v2/ContextFactory.createContext(ContextFactory.java:139)
4XESTACKTRACE          at
com/sun/xml/bind/v2/ContextFactory.createContext(ContextFactory.java:117)
4XESTACKTRACE          at
sun/reflect/NativeMethodAccessorImpl.invoke0(Native Method)
4XESTACKTRACE          at
sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64(Compiled
Code))
4XESTACKTRACE          at
sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43(Compiled
Code))
4XESTACKTRACE          at
java/lang/reflect/Method.invoke(Method.java:615(Compiled Code))
4XESTACKTRACE          at
javax/xml/bind/ContextFinder.newInstance(ContextFinder.java:211)
4XESTACKTRACE          at
javax/xml/bind/ContextFinder.find(ContextFinder.java:372)
4XESTACKTRACE          at
javax/xml/bind/JAXBContext.newInstance(JAXBContext.java:574)
4XESTACKTRACE          at
org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:532)
4XESTACKTRACE          at
org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
4XESTACKTRACE          at
org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
4XESTACKTRACE          at
org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
4XESTACKTRACE          at
org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
4XESTACKTRACE          at
org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
4XESTACKTRACE          at
org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)

3XMTHREADINFO      "http-7101-3" (TID:0x00000000112A5C00,
sys_thread_t:0x00000000108625E0, state:B, native
ID:0x00000000000057CB) prio=5
4XESTACKTRACE          at
org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530)
4XESTACKTRACE          at
org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
4XESTACKTRACE          at
org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
4XESTACKTRACE          at
org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
4XESTACKTRACE          at
org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
4XESTACKTRACE          at
org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
4XESTACKTRACE          at
org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)

(... ...)

"http-7101-30" (TID:0x00000000131A6400,
sys_thread_t:0x000000001442D4A0, state:B, native
ID:0x0000000000004808) prio=5
4XESTACKTRACE          at
org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530(Compiled
Code))
4XESTACKTRACE          at
org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
4XESTACKTRACE          at
org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
4XESTACKTRACE          at
org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
4XESTACKTRACE          at
org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
4XESTACKTRACE          at
org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
4XESTACKTRACE          at
org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)

(... ...)

"http-7101-60" (TID:0x0000000014797300,
sys_thread_t:0x0000000014782D40, state:B, native
ID:0x0000000000004C25) prio=5
4XESTACKTRACE          at
org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530(Compiled
Code))
4XESTACKTRACE          at
org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
4XESTACKTRACE          at
org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
4XESTACKTRACE          at
org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
4XESTACKTRACE          at
org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
4XESTACKTRACE          at
org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
4XESTACKTRACE          at
org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
4XESTACKTRACE          at
org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
4XESTACKTRACE          at
org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
4XESTACKTRACE          at
com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)


Is this a bug in CXF? Or I have to create new Service Object per Thread?

Any suggestion or tip is really appreciated.

Thanks a lot!

Jian

Re: Generated CXF Client is Blocked in Tomcat Container, is this a possible deadlock?

Posted by jian wu <he...@gmail.com>.
Hi Willem,

Thanks a lot for your quick reply!

> How did you get the all this threads blocked?

Since there are concurrent requests coming in

> For your stack trace , it looks like there are some concurrent
>  issues in the CacheMap.

Yes, I want to check whether someone else also run into the
similar issue and whether this is an issue of CXF.

> I think you need add some lock in your proxy object creating code, to
> create the proxy one by one for each thread.

This will have bad impact on performance, I hope that should not be
necessary. CXF should allow multi-threaded clients to work well.

Will one service object per thread resolve this problem?

Thanks again,

Jian


On Thu, Oct 30, 2008 at 12:07 AM, Willem Jiang <wi...@gmail.com> wrote:
> Hi,
> How did you get the all this threads blocked?
> Did you try to use one client to send one request to your server?
>
> For your stack trace , it looks like there are some concurrent issues in
>  the CacheMap.
>
> I think you need add some lock in your proxy object creating code, to
> create the proxy one by one for each thread.
>
> Willem
>
> jian wu wrote:
>> Hi,
>>
>> I'm running a WSDL2JAVA generated CXF WS Client in Tomcat 6.0.10 to
>> call external WS
>> for the incoming request.
>>
>> I'm using one Serivce Object to create one new port object for each WS
>> Call per thread,
>> it seems that, eventually all the http threads are blocked at:
>>   org.apache.cxf.jaxb.JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530)
>>
>> I'm using CXF 2.1.1,  the Tomcat Server is running on Linux with IBM
>> 64bit JVM. The
>> following is the thread dump snippet:
>>
>> 3LKMONOBJECT
>> org/apache/cxf/common/util/CacheMap@00000080D941B830/00000080D941B848:
>> Flat locked by "http-7101-2" (0x000000001129F300), entry count 1
>> 3LKWAITERQ            Waiting to enter:
>> 3LKWAITER                "http-7101-1" (0x0000000010870F00)
>> 3LKWAITER                "http-7101-3" (0x00000000112A5C00)
>> 3LKWAITER                "http-7101-4" (0x00000000128C4300)
>> 3LKWAITER                "http-7101-5" (0x00000000137B9D00)
>> 3LKWAITER                "http-7101-6" (0x00000000137CAF00)
>> 3LKWAITER                "http-7101-7" (0x00000000137DF000)
>> 3LKWAITER                "http-7101-8" (0x00000000137EF200)
>> 3LKWAITER                "http-7101-9" (0x0000000013800300)
>> 3LKWAITER                "http-7101-10" (0x000000001335A600)
>> (... ...)
>> 3LKWAITER                "http-7101-60" (0x0000000014797300)
>>
>> 3XMTHREADINFO      "http-7101-2" (TID:0x000000001129F300,
>> sys_thread_t:0x0000000010862110, state:CW, native
>> ID:0x00000000000057CA) prio=5
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/TypeInfoSetImpl.getElementInfo(TypeInfoSetImpl.java:275(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeTypeInfoSetImpl.getElementInfo(RuntimeTypeInfoSetImpl.java:102)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeTypeInfoSetImpl.getElementInfo(RuntimeTypeInfoSetImpl.java:59)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.addGenericElement(ReferencePropertyInfoImpl.java:196)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.calcTypes(ReferencePropertyInfoImpl.java:158(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.getElements(ReferencePropertyInfoImpl.java:112)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.getElements(RuntimeReferencePropertyInfoImpl.java:67)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.ref(ReferencePropertyInfoImpl.java:103)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.ref(RuntimeReferencePropertyInfoImpl.java:71)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.ref(RuntimeReferencePropertyInfoImpl.java:51)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:255(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RegistryInfoImpl.<init>(RegistryInfoImpl.java:99)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.addRegistry(ModelBuilder.java:357(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:218(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/TypeRefImpl.calcRef(TypeRefImpl.java:92)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/TypeRefImpl.getTarget(TypeRefImpl.java:69)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeTypeRefImpl.getTarget(RuntimeTypeRefImpl.java:58)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeTypeRefImpl.getTarget(RuntimeTypeRefImpl.java:51)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ElementPropertyInfoImpl$1.get(ElementPropertyInfoImpl.java:74)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ElementPropertyInfoImpl$1.get(ElementPropertyInfoImpl.java:77)
>> 4XESTACKTRACE          at
>> java/util/AbstractList$Itr.next(AbstractList.java:437(Compiled Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:255(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:330)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/runtime/JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:432)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/runtime/JAXBContextImpl.<init>(JAXBContextImpl.java:286)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/ContextFactory.createContext(ContextFactory.java:139)
>> 4XESTACKTRACE          at
>> com/sun/xml/bind/v2/ContextFactory.createContext(ContextFactory.java:117)
>> 4XESTACKTRACE          at
>> sun/reflect/NativeMethodAccessorImpl.invoke0(Native Method)
>> 4XESTACKTRACE          at
>> sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> java/lang/reflect/Method.invoke(Method.java:615(Compiled Code))
>> 4XESTACKTRACE          at
>> javax/xml/bind/ContextFinder.newInstance(ContextFinder.java:211)
>> 4XESTACKTRACE          at
>> javax/xml/bind/ContextFinder.find(ContextFinder.java:372)
>> 4XESTACKTRACE          at
>> javax/xml/bind/JAXBContext.newInstance(JAXBContext.java:574)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:532)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
>> 4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)
>>
>> 3XMTHREADINFO      "http-7101-3" (TID:0x00000000112A5C00,
>> sys_thread_t:0x00000000108625E0, state:B, native
>> ID:0x00000000000057CB) prio=5
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
>> 4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)
>>
>> (... ...)
>>
>> "http-7101-30" (TID:0x00000000131A6400,
>> sys_thread_t:0x000000001442D4A0, state:B, native
>> ID:0x0000000000004808) prio=5
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
>> 4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)
>>
>> (... ...)
>>
>> "http-7101-60" (TID:0x0000000014797300,
>> sys_thread_t:0x0000000014782D40, state:B, native
>> ID:0x0000000000004C25) prio=5
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530(Compiled
>> Code))
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
>> 4XESTACKTRACE          at
>> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
>> 4XESTACKTRACE          at
>> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
>> 4XESTACKTRACE          at
>> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
>> 4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
>> 4XESTACKTRACE          at
>> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)
>>
>>
>> Is this a bug in CXF? Or I have to create new Service Object per Thread?
>>
>> Any suggestion or tip is really appreciated.
>>
>> Thanks a lot!
>>
>> Jian
>>
>
>

Re: Generated CXF Client is Blocked in Tomcat Container, is this a possible deadlock?

Posted by Willem Jiang <wi...@gmail.com>.
Hi,
How did you get the all this threads blocked?
Did you try to use one client to send one request to your server?

For your stack trace , it looks like there are some concurrent issues in
  the CacheMap.

I think you need add some lock in your proxy object creating code, to
create the proxy one by one for each thread.

Willem

jian wu wrote:
> Hi,
> 
> I'm running a WSDL2JAVA generated CXF WS Client in Tomcat 6.0.10 to
> call external WS
> for the incoming request.
> 
> I'm using one Serivce Object to create one new port object for each WS
> Call per thread,
> it seems that, eventually all the http threads are blocked at:
>   org.apache.cxf.jaxb.JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530)
> 
> I'm using CXF 2.1.1,  the Tomcat Server is running on Linux with IBM
> 64bit JVM. The
> following is the thread dump snippet:
> 
> 3LKMONOBJECT
> org/apache/cxf/common/util/CacheMap@00000080D941B830/00000080D941B848:
> Flat locked by "http-7101-2" (0x000000001129F300), entry count 1
> 3LKWAITERQ            Waiting to enter:
> 3LKWAITER                "http-7101-1" (0x0000000010870F00)
> 3LKWAITER                "http-7101-3" (0x00000000112A5C00)
> 3LKWAITER                "http-7101-4" (0x00000000128C4300)
> 3LKWAITER                "http-7101-5" (0x00000000137B9D00)
> 3LKWAITER                "http-7101-6" (0x00000000137CAF00)
> 3LKWAITER                "http-7101-7" (0x00000000137DF000)
> 3LKWAITER                "http-7101-8" (0x00000000137EF200)
> 3LKWAITER                "http-7101-9" (0x0000000013800300)
> 3LKWAITER                "http-7101-10" (0x000000001335A600)
> (... ...)
> 3LKWAITER                "http-7101-60" (0x0000000014797300)
> 
> 3XMTHREADINFO      "http-7101-2" (TID:0x000000001129F300,
> sys_thread_t:0x0000000010862110, state:CW, native
> ID:0x00000000000057CA) prio=5
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/TypeInfoSetImpl.getElementInfo(TypeInfoSetImpl.java:275(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeTypeInfoSetImpl.getElementInfo(RuntimeTypeInfoSetImpl.java:102)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeTypeInfoSetImpl.getElementInfo(RuntimeTypeInfoSetImpl.java:59)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.addGenericElement(ReferencePropertyInfoImpl.java:196)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.calcTypes(ReferencePropertyInfoImpl.java:158(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.getElements(ReferencePropertyInfoImpl.java:112)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.getElements(RuntimeReferencePropertyInfoImpl.java:67)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.ref(ReferencePropertyInfoImpl.java:103)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.ref(RuntimeReferencePropertyInfoImpl.java:71)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.ref(RuntimeReferencePropertyInfoImpl.java:51)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:255(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RegistryInfoImpl.<init>(RegistryInfoImpl.java:99)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.addRegistry(ModelBuilder.java:357(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:218(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/TypeRefImpl.calcRef(TypeRefImpl.java:92)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/TypeRefImpl.getTarget(TypeRefImpl.java:69)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeTypeRefImpl.getTarget(RuntimeTypeRefImpl.java:58)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeTypeRefImpl.getTarget(RuntimeTypeRefImpl.java:51)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ElementPropertyInfoImpl$1.get(ElementPropertyInfoImpl.java:74)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ElementPropertyInfoImpl$1.get(ElementPropertyInfoImpl.java:77)
> 4XESTACKTRACE          at
> java/util/AbstractList$Itr.next(AbstractList.java:437(Compiled Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:255(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled
> Code))
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:330)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/runtime/JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:432)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/runtime/JAXBContextImpl.<init>(JAXBContextImpl.java:286)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/ContextFactory.createContext(ContextFactory.java:139)
> 4XESTACKTRACE          at
> com/sun/xml/bind/v2/ContextFactory.createContext(ContextFactory.java:117)
> 4XESTACKTRACE          at
> sun/reflect/NativeMethodAccessorImpl.invoke0(Native Method)
> 4XESTACKTRACE          at
> sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64(Compiled
> Code))
> 4XESTACKTRACE          at
> sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43(Compiled
> Code))
> 4XESTACKTRACE          at
> java/lang/reflect/Method.invoke(Method.java:615(Compiled Code))
> 4XESTACKTRACE          at
> javax/xml/bind/ContextFinder.newInstance(ContextFinder.java:211)
> 4XESTACKTRACE          at
> javax/xml/bind/ContextFinder.find(ContextFinder.java:372)
> 4XESTACKTRACE          at
> javax/xml/bind/JAXBContext.newInstance(JAXBContext.java:574)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:532)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
> 4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)
> 
> 3XMTHREADINFO      "http-7101-3" (TID:0x00000000112A5C00,
> sys_thread_t:0x00000000108625E0, state:B, native
> ID:0x00000000000057CB) prio=5
> 4XESTACKTRACE          at
> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
> 4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)
> 
> (... ...)
> 
> "http-7101-30" (TID:0x00000000131A6400,
> sys_thread_t:0x000000001442D4A0, state:B, native
> ID:0x0000000000004808) prio=5
> 4XESTACKTRACE          at
> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530(Compiled
> Code))
> 4XESTACKTRACE          at
> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
> 4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)
> 
> (... ...)
> 
> "http-7101-60" (TID:0x0000000014797300,
> sys_thread_t:0x0000000014782D40, state:B, native
> ID:0x0000000000004C25) prio=5
> 4XESTACKTRACE          at
> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530(Compiled
> Code))
> 4XESTACKTRACE          at
> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394)
> 4XESTACKTRACE          at
> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51)
> 4XESTACKTRACE          at
> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247)
> 4XESTACKTRACE          at
> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234)
> 4XESTACKTRACE          at javax/xml/ws/Service.getPort(Service.java:40)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87)
> 4XESTACKTRACE          at
> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91)
> 
> 
> Is this a bug in CXF? Or I have to create new Service Object per Thread?
> 
> Any suggestion or tip is really appreciated.
> 
> Thanks a lot!
> 
> Jian
>