[jira] [Commented] (CXF-7135) ConcurrentModificationException in MessageImpl.calcContextCache() when using JMS Transport and JAXRS Client

["Sergey Beryozkin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CXF-7135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15683097#comment-15683097 ] 

Sergey Beryozkin commented on CXF-7135:
---------------------------------------

I've started with updating ConnectionFactoryFeature.

> ConcurrentModificationException in MessageImpl.calcContextCache() when using JMS Transport and JAXRS Client
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: CXF-7135
>                 URL: https://issues.apache.org/jira/browse/CXF-7135
>             Project: CXF
>          Issue Type: Bug
>          Components: JAX-RS, JMS
>    Affects Versions: 3.0.11
>         Environment: Mac OSX 10.11.6. 
> Java JDK 1.8.0_71 (64 bit Server)
>            Reporter: Dan Salt
>         Attachments: cxf-rsjms-testcase.zip
>
>
> Our platform allows the user to switch transports for a particular proxy based on configuration. We currently allow HTTP and JMS as possible options.
> During recent performance testing, we've hit a bug which causes JAXRS Clients over JMS to fail with a ConcurrentModificationException:
> A test case is attached that reproduces our problem.
> Our test executes requests across multiple concurrent threads, switching between JMS and HTTP for both JAXWS and JAXRS Client Proxies. In the test case:
> - JAXWS over both HTTP and JMS works fine, both in single and multi-threaded tests.
> - JAXRS works fine across single and multi-threaded tests over HTTP with the 'ThreadSafe' flag set on the Client Factory. 
> - JAXRS over JMS fails in both single-threaded and multi-threaded mode with error (1) below. In multi-threaded, also fails with (2) below. 
> 1) The ConnectionFactoryFeature fails to properly set the JMS Connection Factory on the Conduit and results in an error at send-time because no Connection Factory is set. This is because in JAXRS Clients, the InterceptorProvider is neither a Client or Server, it is a ClientConfiguration, so the JMS ConnectionFactoryFeature does not fire either of it's methods:
> public void initialize(Client client, Bus bus) 
> public void initialize(Server server, Bus bus)
> We have worked around this in our code by sub-classing and overriding the method:
> public void initialize(InterceptorProvider interceptorProvider, Bus bus)
> .. and adding code similar to that in the ConnectionFactory feature. This resolves the missing ConnectionFactory problem and JAXRS-over-JMS works fine in single-threaded mode.
> 2) Under multi-threaded conditions, the JAXRS Clients fail with the following exception:
> javax.ws.rs.client.ResponseProcessingException: Problem with reading the data, class java.lang.String, ContentType: text/plain.
> 	at org.apache.cxf.jaxrs.impl.ResponseImpl.reportMessageHandlerProblem(ResponseImpl.java:438)
> 	at org.apache.cxf.jaxrs.impl.ResponseImpl.doReadEntity(ResponseImpl.java:378)
> 	at org.apache.cxf.jaxrs.client.AbstractClient.readBody(AbstractClient.java:521)
> 	at org.apache.cxf.jaxrs.client.ClientProxyImpl.handleResponse(ClientProxyImpl.java:817)
> 	at org.apache.cxf.jaxrs.client.ClientProxyImpl.doChainedInvocation(ClientProxyImpl.java:760)
> 	at org.apache.cxf.jaxrs.client.ClientProxyImpl.invoke(ClientProxyImpl.java:228)
> 	at com.sun.proxy.$Proxy24.doSimpleMethod(Unknown Source)
> 	at com.ge.testcase.ThreadingTest$ThreadedRSJMSTest.run(ThreadingTest.java:132)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.ConcurrentModificationException
> 	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
> 	at java.util.HashMap$EntryIterator.next(HashMap.java:1463)
> 	at java.util.HashMap$EntryIterator.next(HashMap.java:1461)
> 	at java.util.HashMap.putMapEntries(HashMap.java:511)
> 	at java.util.HashMap.putAll(HashMap.java:784)
> 	at org.apache.cxf.message.MessageImpl$1.putAll(MessageImpl.java:188)
> 	at org.apache.cxf.message.MessageImpl.calcContextCache(MessageImpl.java:213)
> 	at org.apache.cxf.message.MessageImpl.getContextualProperty(MessageImpl.java:174)
> 	at org.apache.cxf.jaxrs.impl.HttpHeadersImpl.getRequestHeaders(HttpHeadersImpl.java:172)
> 	at org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBodyReader(JAXRSUtils.java:1345)
> 	at org.apache.cxf.jaxrs.impl.ResponseImpl.doReadEntity(ResponseImpl.java:369)
> 	... 7 more
> This occurs even though the 'ThreadSafe' property is set via:
> rsClientFactory.setThreadSafe(true);
> Also, interestingly, this exception occurs even if I put the entire Proxy Client invocation in a synchronized block (which should effectively render the Clients single-threaded). 
> Finally, one important factor is that this error only happens on the latest 3.0.x release (3.0.11). When I switch my test case to 3.1.8, it does NOT occur. Not sure if this equates to a fix not being back-ported, or a fundamental difference in functionality. Sadly, we're not able to yet move to 3.1.x, because of running inside the Karaf container, and our dependencies on Camel and Spring 3.x.
> I've done quite a large amount of digging through the code to try and narrow down the scope of the problem, and the results above is as far as I could get, unfortunately. My guess/assumption is that there are some JMS-specific objects that are being added to the Request/Response context which are being added/removed in a non-thread safe way. Purely a guess, though.
> Thanks. Please let me know if you need any other information.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)