You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by "Gino Bustelo (JIRA)" <de...@uima.apache.org> on 2012/05/09 20:37:50 UTC
[jira] [Created] (UIMA-2401) BaseUIMAAsynchronousEngine_impl does
not protect access to SharedConnection in a multi-instance, multi-threaded
scenario. Possible javax.jms.IllegalStateException.
Gino Bustelo created UIMA-2401:
----------------------------------
Summary: BaseUIMAAsynchronousEngine_impl does not protect access to SharedConnection in a multi-instance, multi-threaded scenario. Possible javax.jms.IllegalStateException.
Key: UIMA-2401
URL: https://issues.apache.org/jira/browse/UIMA-2401
Project: UIMA
Issue Type: Bug
Components: Async Scaleout
Affects Versions: 2.3.1AS
Environment: RedHat Enterprise Linux 6.0, Mac OSX Lion 10.7.3
Reporter: Gino Bustelo
We see this problem sporadically when running one instance of BaseUIMAAsynchronousEngine_impl per Thread. This means that each request Thread will instantiate its own instance of BaseUIMAAsynchronousEngine_impl to submit a CAS async. We have noticed in our logs the following exception:
org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl WARN 2012-05-04 00:10:46,710 [BaseUIMAAsynchronousEngine_impl.803] - Top Level Controller Initialization Exception.
javax.jms.IllegalStateException: The Session is closed
at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:722)
at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1101)
at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1060)
at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:973)
at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:946)
at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initializeConsumer(BaseUIMAAsynchronousEngine_impl.java:523)
at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initialize(BaseUIMAAsynchronousEngine_impl.java:687)
Under further inspection, it seems that the BaseUIMAAsynchronousEngine_impl.initialize does not properly protects the access to the SharedConnection object. Given the following scenario:
Thread 1 is in the middle of a call to UimaAsynchronousEngine.initialize.
Thread 2 is finishing up and calling UimaAsynchronousEngine.stop.
Thread 1 can be schedule out just before trying to create a Consumer (BaseUIMAAsynchronousEngine_impl.java:523). Thread 2 will call stop() and due to client count, cause the Connection to get closed. When Thread 1 comes back, the Connection and Session instances that it has reference to are now invalid.
I've attached a test case that forces the exception to prove that there is no proper mutex around access to the SharedConnection between initialize and stop. I believe that the code in question is in BaseUIMAAsynchronousEngine_impl.initialize between lines 682 and 700.
createSharedConnection(brokerURI);
synchronized (connectionMux) {
// Reuse existing JMS connection if available
if (sharedConnection != null) {
initializeProducer(brokerURI, endpoint, sharedConnection.getConnection());
initializeConsumer(brokerURI, sharedConnection.getConnection());
} else {
initializeProducer(brokerURI, endpoint);
initializeConsumer(brokerURI);
}
// Increment number of client instances. SharedConnection object is a static
// and is used to share a single JMS connection. The connection is closed
// when the last client finishes processing and calls stop().
if (sharedConnection != null) {
sharedConnection.registerClient(this);
}
}
That entire block should be protected with the sharedConnectionSemaphore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (UIMA-2401) BaseUIMAAsynchronousEngine_impl does
not protect access to SharedConnection in a multi-instance, multi-threaded
scenario. Possible javax.jms.IllegalStateException.
Posted by "Jerry Cwiklik (JIRA)" <de...@uima.apache.org>.
[ https://issues.apache.org/jira/browse/UIMA-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerry Cwiklik closed UIMA-2401.
-------------------------------
Resolution: Fixed
Fix Version/s: 2.4.0AS
Assignee: Jerry Cwiklik
Replicated the problem with supplied test case. Modified BaseUIMAAsynchronousEngine_impl to synchronize access to SharedConnection object during initialization and shutdown of the uima-as client to prevent concurrent access by multiple threads. After the change, the test case completes with no exceptions.
> BaseUIMAAsynchronousEngine_impl does not protect access to SharedConnection in a multi-instance, multi-threaded scenario. Possible javax.jms.IllegalStateException.
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: UIMA-2401
> URL: https://issues.apache.org/jira/browse/UIMA-2401
> Project: UIMA
> Issue Type: Bug
> Components: Async Scaleout
> Affects Versions: 2.3.1AS
> Environment: RedHat Enterprise Linux 6.0, Mac OSX Lion 10.7.3
> Reporter: Gino Bustelo
> Assignee: Jerry Cwiklik
> Labels: ibm
> Fix For: 2.4.0AS
>
> Attachments: IllegalStateTimingDefect.zip
>
>
> We see this problem sporadically when running one instance of BaseUIMAAsynchronousEngine_impl per Thread. This means that each request Thread will instantiate its own instance of BaseUIMAAsynchronousEngine_impl to submit a CAS async. We have noticed in our logs the following exception:
> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl WARN 2012-05-04 00:10:46,710 [BaseUIMAAsynchronousEngine_impl.803] - Top Level Controller Initialization Exception.
> javax.jms.IllegalStateException: The Session is closed
> at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:722)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1101)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1060)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:973)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:946)
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initializeConsumer(BaseUIMAAsynchronousEngine_impl.java:523)
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initialize(BaseUIMAAsynchronousEngine_impl.java:687)
> Under further inspection, it seems that the BaseUIMAAsynchronousEngine_impl.initialize does not properly protects the access to the SharedConnection object. Given the following scenario:
> Thread 1 is in the middle of a call to UimaAsynchronousEngine.initialize.
> Thread 2 is finishing up and calling UimaAsynchronousEngine.stop.
> Thread 1 can be schedule out just before trying to create a Consumer (BaseUIMAAsynchronousEngine_impl.java:523). Thread 2 will call stop() and due to client count, cause the Connection to get closed. When Thread 1 comes back, the Connection and Session instances that it has reference to are now invalid.
> I've attached a test case that forces the exception to prove that there is no proper mutex around access to the SharedConnection between initialize and stop. I believe that the code in question is in BaseUIMAAsynchronousEngine_impl.initialize between lines 682 and 700.
> createSharedConnection(brokerURI);
> synchronized (connectionMux) {
> // Reuse existing JMS connection if available
> if (sharedConnection != null) {
> initializeProducer(brokerURI, endpoint, sharedConnection.getConnection());
> initializeConsumer(brokerURI, sharedConnection.getConnection());
> } else {
> initializeProducer(brokerURI, endpoint);
> initializeConsumer(brokerURI);
> }
> // Increment number of client instances. SharedConnection object is a static
> // and is used to share a single JMS connection. The connection is closed
> // when the last client finishes processing and calls stop().
> if (sharedConnection != null) {
> sharedConnection.registerClient(this);
> }
> }
> That entire block should be protected with the sharedConnectionSemaphore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (UIMA-2401) BaseUIMAAsynchronousEngine_impl does
not protect access to SharedConnection in a multi-instance, multi-threaded
scenario. Possible javax.jms.IllegalStateException.
Posted by "Jerry Cwiklik (JIRA)" <de...@uima.apache.org>.
[ https://issues.apache.org/jira/browse/UIMA-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285788#comment-13285788 ]
Jerry Cwiklik commented on UIMA-2401:
-------------------------------------
In addition to the above mentioned changes, the uima-as client code (BaseUIMAAsynchronousEngine_impl.java) has been modified to change ConnectionMux object scope to static. This change guards the connection object from concurrent access.
> BaseUIMAAsynchronousEngine_impl does not protect access to SharedConnection in a multi-instance, multi-threaded scenario. Possible javax.jms.IllegalStateException.
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: UIMA-2401
> URL: https://issues.apache.org/jira/browse/UIMA-2401
> Project: UIMA
> Issue Type: Bug
> Components: Async Scaleout
> Affects Versions: 2.3.1AS
> Environment: RedHat Enterprise Linux 6.0, Mac OSX Lion 10.7.3
> Reporter: Gino Bustelo
> Assignee: Jerry Cwiklik
> Labels: ibm
> Fix For: 2.4.0AS
>
> Attachments: IllegalStateTimingDefect.zip
>
>
> We see this problem sporadically when running one instance of BaseUIMAAsynchronousEngine_impl per Thread. This means that each request Thread will instantiate its own instance of BaseUIMAAsynchronousEngine_impl to submit a CAS async. We have noticed in our logs the following exception:
> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl WARN 2012-05-04 00:10:46,710 [BaseUIMAAsynchronousEngine_impl.803] - Top Level Controller Initialization Exception.
> javax.jms.IllegalStateException: The Session is closed
> at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:722)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1101)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1060)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:973)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:946)
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initializeConsumer(BaseUIMAAsynchronousEngine_impl.java:523)
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initialize(BaseUIMAAsynchronousEngine_impl.java:687)
> Under further inspection, it seems that the BaseUIMAAsynchronousEngine_impl.initialize does not properly protects the access to the SharedConnection object. Given the following scenario:
> Thread 1 is in the middle of a call to UimaAsynchronousEngine.initialize.
> Thread 2 is finishing up and calling UimaAsynchronousEngine.stop.
> Thread 1 can be schedule out just before trying to create a Consumer (BaseUIMAAsynchronousEngine_impl.java:523). Thread 2 will call stop() and due to client count, cause the Connection to get closed. When Thread 1 comes back, the Connection and Session instances that it has reference to are now invalid.
> I've attached a test case that forces the exception to prove that there is no proper mutex around access to the SharedConnection between initialize and stop. I believe that the code in question is in BaseUIMAAsynchronousEngine_impl.initialize between lines 682 and 700.
> createSharedConnection(brokerURI);
> synchronized (connectionMux) {
> // Reuse existing JMS connection if available
> if (sharedConnection != null) {
> initializeProducer(brokerURI, endpoint, sharedConnection.getConnection());
> initializeConsumer(brokerURI, sharedConnection.getConnection());
> } else {
> initializeProducer(brokerURI, endpoint);
> initializeConsumer(brokerURI);
> }
> // Increment number of client instances. SharedConnection object is a static
> // and is used to share a single JMS connection. The connection is closed
> // when the last client finishes processing and calls stop().
> if (sharedConnection != null) {
> sharedConnection.registerClient(this);
> }
> }
> That entire block should be protected with the sharedConnectionSemaphore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (UIMA-2401) BaseUIMAAsynchronousEngine_impl does
not protect access to SharedConnection in a multi-instance, multi-threaded
scenario. Possible javax.jms.IllegalStateException.
Posted by "Gino Bustelo (JIRA)" <de...@uima.apache.org>.
[ https://issues.apache.org/jira/browse/UIMA-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gino Bustelo updated UIMA-2401:
-------------------------------
Attachment: IllegalStateTimingDefect.zip
Eclipse Project with test case. Refer to README.
> BaseUIMAAsynchronousEngine_impl does not protect access to SharedConnection in a multi-instance, multi-threaded scenario. Possible javax.jms.IllegalStateException.
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: UIMA-2401
> URL: https://issues.apache.org/jira/browse/UIMA-2401
> Project: UIMA
> Issue Type: Bug
> Components: Async Scaleout
> Affects Versions: 2.3.1AS
> Environment: RedHat Enterprise Linux 6.0, Mac OSX Lion 10.7.3
> Reporter: Gino Bustelo
> Labels: ibm
> Attachments: IllegalStateTimingDefect.zip
>
>
> We see this problem sporadically when running one instance of BaseUIMAAsynchronousEngine_impl per Thread. This means that each request Thread will instantiate its own instance of BaseUIMAAsynchronousEngine_impl to submit a CAS async. We have noticed in our logs the following exception:
> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl WARN 2012-05-04 00:10:46,710 [BaseUIMAAsynchronousEngine_impl.803] - Top Level Controller Initialization Exception.
> javax.jms.IllegalStateException: The Session is closed
> at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:722)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1101)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1060)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:973)
> at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:946)
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initializeConsumer(BaseUIMAAsynchronousEngine_impl.java:523)
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initialize(BaseUIMAAsynchronousEngine_impl.java:687)
> Under further inspection, it seems that the BaseUIMAAsynchronousEngine_impl.initialize does not properly protects the access to the SharedConnection object. Given the following scenario:
> Thread 1 is in the middle of a call to UimaAsynchronousEngine.initialize.
> Thread 2 is finishing up and calling UimaAsynchronousEngine.stop.
> Thread 1 can be schedule out just before trying to create a Consumer (BaseUIMAAsynchronousEngine_impl.java:523). Thread 2 will call stop() and due to client count, cause the Connection to get closed. When Thread 1 comes back, the Connection and Session instances that it has reference to are now invalid.
> I've attached a test case that forces the exception to prove that there is no proper mutex around access to the SharedConnection between initialize and stop. I believe that the code in question is in BaseUIMAAsynchronousEngine_impl.initialize between lines 682 and 700.
> createSharedConnection(brokerURI);
> synchronized (connectionMux) {
> // Reuse existing JMS connection if available
> if (sharedConnection != null) {
> initializeProducer(brokerURI, endpoint, sharedConnection.getConnection());
> initializeConsumer(brokerURI, sharedConnection.getConnection());
> } else {
> initializeProducer(brokerURI, endpoint);
> initializeConsumer(brokerURI);
> }
> // Increment number of client instances. SharedConnection object is a static
> // and is used to share a single JMS connection. The connection is closed
> // when the last client finishes processing and calls stop().
> if (sharedConnection != null) {
> sharedConnection.registerClient(this);
> }
> }
> That entire block should be protected with the sharedConnectionSemaphore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira