[jira] [Closed] (UIMA-2392) UIMA-AS CAS multiplier hangs fetching empty CAS after client timeouts

["Jerry Cwiklik (JIRA)" <de...@uima.apache.org>]

     [ https://issues.apache.org/jira/browse/UIMA-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jerry Cwiklik closed UIMA-2392.
-------------------------------

       Resolution: Fixed
    Fix Version/s: 2.4.0AS
         Assignee: Jerry Cwiklik

In this scenario there are multiple uima-as client instances (same jvm - each client in a different thread) sending CASes and stopping due to simulated timeouts. All of them share the same Connection object (by design) but have different(dedicated) JMS Sessions, Consumers and temp reply queues. The Connection is shared by all clients to optimize the broker. This reduces number of threads, context switching, etc.
When any of the clients is stopped, its temp reply queue is not being removed from the Broker because the Connection is still open. The uima-as closes the Connection when all clients are stopped. This leads to two problems: temp queue build-up/leak and possible CM hang as described in this JIRA. The hang is caused by the fact that the service sends a CAS to a temp reply queue with no listener (client was stopped).

Modified uima-as client to delete its temp reply queue while cleaning up during stop. The AMQ connection class has an API - deleteTempDestination() which enables temp queue removal. Now, when a client is stopped, the service is unable to deliver a CAS to the client's temp queue (it no longer exists) and releases it back to the CM's cas pool.

Tested the change numerous times with supplied test case with no hang   
                
> UIMA-AS CAS multiplier hangs fetching empty CAS after client timeouts
> ---------------------------------------------------------------------
>
>                 Key: UIMA-2392
>                 URL: https://issues.apache.org/jira/browse/UIMA-2392
>             Project: UIMA
>          Issue Type: Bug
>          Components: Async Scaleout
>    Affects Versions: 2.3.1AS
>         Environment: RedHat Enterprise Linux 6.0, Mac OSX Lion 10.7.3
>            Reporter: Peter Parente
>            Assignee: Jerry Cwiklik
>              Labels: ibm
>             Fix For: 2.4.0AS
>
>         Attachments: logs.zip, uima-hang.zip
>
>
> I'm attaching a test case that reproduces the hang. See the README inside the zip about how to execute it.
> The annotator (ForwardJCas) consists of a simple CAS multiplier with delays in it scaled out in its own thread within 2 levels of aggregates. It receives a CAS, sleeps for 3 seconds, gets a new empty CAS, copies information into it, sleeps again, and then returns the CAS. The unit test (UimaAsTest) initializes 40 UIMA-AS clients with a timeout of 8 seconds, send 1 CAS from each to the service, and then waits. After 1-2 runs, the CAS multiplier winds up hanging on the getEmptyCas() call in the scaled out annotator. 
> What appears to be happening is that free CAS messages are not being sent from the client to the service in all cases when the timeouts occur. A similar defect (https://issues.apache.org/jira/browse/UIMA-1786) was fixed in 2.3.1 with slightly different symptoms.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira