You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by "Jerry Cwiklik (JIRA)" <de...@uima.apache.org> on 2011/02/25 22:39:22 UTC

[jira] Created: (UIMA-2078) Fix intermittent problem with CAS release while running scaled up AE in UIMA AS

Fix intermittent problem with CAS release while running scaled up AE in UIMA AS 
--------------------------------------------------------------------------------

                 Key: UIMA-2078
                 URL: https://issues.apache.org/jira/browse/UIMA-2078
             Project: UIMA
          Issue Type: Bug
          Components: Async Scaleout, Core Java Framework
            Reporter: Jerry Cwiklik
            Assignee: Jerry Cwiklik
             Fix For: 2.3.1AS


While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:

org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
	at java.lang.Thread.run(Thread.java:680)\
Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\


The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 

Postponing fix beyond 2.3.1 release

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Re: [jira] [Commented] (UIMA-2078) Fix intermittent problem with CAS release while running scaled up AE in UIMA AS

Posted by Marshall Schor <ms...@schor.com>.
I wrote a test case which should produce the error message

org.apache.uima.analysis_engine.AnalysisEngineProcessException:
AnalysisComponent "/" requested more CASes (2) than defined in its
getCasInstancesRequired() method (1).  It is possible that the AnalysisComponent
is not properly releasing CASes when it encounters an error.

This is due to the original code for multiprocessingAE sharing the top level
UIMA Context.  However, to my surprise, the test ran.  Careful inspection showed
that if the top level UIMA Context is the same for the various "instances" of an
aggregate, the lower level UIMA "Child" contexts are *not* shared.  So if your
Cas Multiplier is within an aggregate, the cas pooling for it will work correctly.

If you put the Cas Multiplier at the top level (not in a primitive), though, you
do get this error.

This error is being fixed by *not* sharing the top level UIMA-Context in this
case, while sharing the two objects contained within it that ought to be shared:
the ResourceManager and the ConfigurationManager.

-Marshall


On 3/26/2011 9:48 PM, Marshall Schor wrote:
> I've got this design mostly working:
>
> The basic structure of things:
>
> A Resource (e.g., an AnalysisEngine, or a MultiprocessingAnalysisEngine)
>
>     -> UIMAContext
>
>         -> ResourceManger
>         -> ConfigurationManager
>
> The redesign is setup so that the MultiprocessingAE Resource has its own
> UIMAContext, and the individual instances in the AE pool each have their own,
> unique UIMAContexts.
>
> These unique UIMAContexts, though, share a common ResourceManager.
>
> The ConfigurationManager has the info about the configuration (e.g., parameter
> settings).  It seems to me that all of the instances in the AE pool (plus the
> main MultiprocessingAE) might want to share this,
>
> There is a test case, currently, which
>
> a) sets a parameter in the MultiprocessingAE instance, using
> mae.setConfigParameterValue("StringParam", "Test2"); and then
> b) does a reconfigure call on the MultiprocessingAE, and expects that to
> reconfigure the params in the individual AEs. 
>
> The reconfigure method does indeed forward the reconfigure call to all the AE's
> in the pool, but they don't see the changed parameters, since they have their
> own ConfigurationManager data, I think.
>
> So it seems we either set things up to share this (recognizing the need for
> thread safety), or we change the reconfigure impl of MultiprocessingAE to also
> propagate the changes in the ConfigurationManager to the other
> ConfigurationManagers.
>
> Does anyone see a downside to sharing the ConfigurationManager in this case?
>
> -Marshall
>
>
> On 3/25/2011 12:12 PM, Marshall Schor (JIRA) wrote:
>>     [ https://issues.apache.org/jira/browse/UIMA-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011255#comment-13011255 ] 
>>
>> Marshall Schor commented on UIMA-2078:
>> --------------------------------------
>>
>> After some discussion on the mailing list, the right fix seems to be to have non-shared UIMA-Contexts (to make Cas Multipliers work) but to have a shared Resource Manager (to preserve the capability to share large external resources among the multiple instances).  With this design, each instance ought to have its own cas pool (for cas Multipliers).
>>
>>> Fix intermittent problem with CAS release while running scaled up AE in UIMA AS 
>>> --------------------------------------------------------------------------------
>>>
>>>                 Key: UIMA-2078
>>>                 URL: https://issues.apache.org/jira/browse/UIMA-2078
>>>             Project: UIMA
>>>          Issue Type: Bug
>>>          Components: Async Scaleout, Core Java Framework
>>>            Reporter: Jerry Cwiklik
>>>            Assignee: Marshall Schor
>>>             Fix For: 2.3.2SDK
>>>
>>>
>>> While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:
>>> org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
>>> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
>>> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
>>> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
>>> 	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
>>> 	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
>>> 	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
>>> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
>>> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
>>> 	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
>>> 	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
>>> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
>>> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
>>> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
>>> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
>>> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
>>> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
>>> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
>>> 	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
>>> 	at java.lang.Thread.run(Thread.java:680)\
>>> Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
>>> 	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
>>> 	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
>>> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
>>> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
>>> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\
>>> The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 
>>> Further diagnosis points toward an issue with the UIMA Java SDK, for use cases where multiple instances of an aggregate (containing a Cas Multiplier) are created and run using a shared common UIMA Context / Resource Manager.  Several parts of the Resource Manager are not thread safe.  
>>> Figure out where things like getEmptyCas need to be synchronized.
>>> Come up with a better test case (that isn't so intermittent).
>> --
>> This message is automatically generated by JIRA.
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>

Re: [jira] [Commented] (UIMA-2078) Fix intermittent problem with CAS release while running scaled up AE in UIMA AS

Posted by Marshall Schor <ms...@schor.com>.
I've got this design mostly working:

The basic structure of things:

A Resource (e.g., an AnalysisEngine, or a MultiprocessingAnalysisEngine)

    -> UIMAContext

        -> ResourceManger
        -> ConfigurationManager

The redesign is setup so that the MultiprocessingAE Resource has its own
UIMAContext, and the individual instances in the AE pool each have their own,
unique UIMAContexts.

These unique UIMAContexts, though, share a common ResourceManager.

The ConfigurationManager has the info about the configuration (e.g., parameter
settings).  It seems to me that all of the instances in the AE pool (plus the
main MultiprocessingAE) might want to share this,

There is a test case, currently, which

a) sets a parameter in the MultiprocessingAE instance, using
mae.setConfigParameterValue("StringParam", "Test2"); and then
b) does a reconfigure call on the MultiprocessingAE, and expects that to
reconfigure the params in the individual AEs. 

The reconfigure method does indeed forward the reconfigure call to all the AE's
in the pool, but they don't see the changed parameters, since they have their
own ConfigurationManager data, I think.

So it seems we either set things up to share this (recognizing the need for
thread safety), or we change the reconfigure impl of MultiprocessingAE to also
propagate the changes in the ConfigurationManager to the other
ConfigurationManagers.

Does anyone see a downside to sharing the ConfigurationManager in this case?

-Marshall


On 3/25/2011 12:12 PM, Marshall Schor (JIRA) wrote:
>     [ https://issues.apache.org/jira/browse/UIMA-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011255#comment-13011255 ] 
>
> Marshall Schor commented on UIMA-2078:
> --------------------------------------
>
> After some discussion on the mailing list, the right fix seems to be to have non-shared UIMA-Contexts (to make Cas Multipliers work) but to have a shared Resource Manager (to preserve the capability to share large external resources among the multiple instances).  With this design, each instance ought to have its own cas pool (for cas Multipliers).
>
>> Fix intermittent problem with CAS release while running scaled up AE in UIMA AS 
>> --------------------------------------------------------------------------------
>>
>>                 Key: UIMA-2078
>>                 URL: https://issues.apache.org/jira/browse/UIMA-2078
>>             Project: UIMA
>>          Issue Type: Bug
>>          Components: Async Scaleout, Core Java Framework
>>            Reporter: Jerry Cwiklik
>>            Assignee: Marshall Schor
>>             Fix For: 2.3.2SDK
>>
>>
>> While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:
>> org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
>> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
>> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
>> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
>> 	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
>> 	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
>> 	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
>> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
>> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
>> 	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
>> 	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
>> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
>> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
>> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
>> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
>> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
>> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
>> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
>> 	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
>> 	at java.lang.Thread.run(Thread.java:680)\
>> Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
>> 	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
>> 	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
>> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
>> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
>> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\
>> The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 
>> Further diagnosis points toward an issue with the UIMA Java SDK, for use cases where multiple instances of an aggregate (containing a Cas Multiplier) are created and run using a shared common UIMA Context / Resource Manager.  Several parts of the Resource Manager are not thread safe.  
>> Figure out where things like getEmptyCas need to be synchronized.
>> Come up with a better test case (that isn't so intermittent).
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>

[jira] [Commented] (UIMA-2078) Fix intermittent problem with CAS release while running scaled up AE in UIMA AS

Posted by "Marshall Schor (JIRA)" <de...@uima.apache.org>.
    [ https://issues.apache.org/jira/browse/UIMA-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011255#comment-13011255 ] 

Marshall Schor commented on UIMA-2078:
--------------------------------------

After some discussion on the mailing list, the right fix seems to be to have non-shared UIMA-Contexts (to make Cas Multipliers work) but to have a shared Resource Manager (to preserve the capability to share large external resources among the multiple instances).  With this design, each instance ought to have its own cas pool (for cas Multipliers).

> Fix intermittent problem with CAS release while running scaled up AE in UIMA AS 
> --------------------------------------------------------------------------------
>
>                 Key: UIMA-2078
>                 URL: https://issues.apache.org/jira/browse/UIMA-2078
>             Project: UIMA
>          Issue Type: Bug
>          Components: Async Scaleout, Core Java Framework
>            Reporter: Jerry Cwiklik
>            Assignee: Marshall Schor
>             Fix For: 2.3.2SDK
>
>
> While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:
> org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
> 	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
> 	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
> 	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
> 	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
> 	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
> 	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
> 	at java.lang.Thread.run(Thread.java:680)\
> Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
> 	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
> 	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\
> The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 
> Further diagnosis points toward an issue with the UIMA Java SDK, for use cases where multiple instances of an aggregate (containing a Cas Multiplier) are created and run using a shared common UIMA Context / Resource Manager.  Several parts of the Resource Manager are not thread safe.  
> Figure out where things like getEmptyCas need to be synchronized.
> Come up with a better test case (that isn't so intermittent).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (UIMA-2078) Fix intermittent problem with CAS release while running scaled up AE in UIMA AS

Posted by "Marshall Schor (JIRA)" <de...@uima.apache.org>.
     [ https://issues.apache.org/jira/browse/UIMA-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marshall Schor resolved UIMA-2078.
----------------------------------

    Resolution: Fixed

The fix had many parts.

1) The upper level (root) UIMA context for the multiple instantiations is no longer shared (because it contains a ref to the CasManager impl instance, which is used for Cas Multipliers, and cannot be shared among instances).

2) Although the UIMA context is not shared, two objects it contains are: the ResourceManager and the ConfigurationManager.  This allows, for instance, parameter settings to be changed and reconfigure called, and it will change the param settings for all instances.

3) Parts of the CasManager and ResourceManger, called on multiple threads, were "synchronized".  

4) Several HashMaps used in these facilities were replaced with synchronized versions.  Concurrent Hash Maps were not used - the expected contention among these maps is likely very low.

5) Several process(...) method variants were missing (and defaulting to the base impl) in MultiprocessingAE class - this was causing a loss of integrity when running in multiple threads, when Cas Multipliers were used.

6) Several tests were added to the test suite, and were used to (a) verify that with the fixes, the tests (frequently) fail, and (b) verify that with the fixes, things work OK. (Tested on Windows and Mac)

5) 

> Fix intermittent problem with CAS release while running scaled up AE in UIMA AS 
> --------------------------------------------------------------------------------
>
>                 Key: UIMA-2078
>                 URL: https://issues.apache.org/jira/browse/UIMA-2078
>             Project: UIMA
>          Issue Type: Bug
>          Components: Async Scaleout, Core Java Framework
>            Reporter: Jerry Cwiklik
>            Assignee: Marshall Schor
>             Fix For: 2.3.2SDK
>
>
> While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:
> org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
> 	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
> 	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
> 	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
> 	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
> 	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
> 	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
> 	at java.lang.Thread.run(Thread.java:680)\
> Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
> 	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
> 	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\
> The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 
> Further diagnosis points toward an issue with the UIMA Java SDK, for use cases where multiple instances of an aggregate (containing a Cas Multiplier) are created and run using a shared common UIMA Context / Resource Manager.  Several parts of the Resource Manager are not thread safe.  
> Figure out where things like getEmptyCas need to be synchronized.
> Come up with a better test case (that isn't so intermittent).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Closed] (UIMA-2078) Fix intermittent problem with CAS release while running scaled up AE in UIMA AS

Posted by "Marshall Schor (Closed) (JIRA)" <de...@uima.apache.org>.
     [ https://issues.apache.org/jira/browse/UIMA-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marshall Schor closed UIMA-2078.
--------------------------------

    
> Fix intermittent problem with CAS release while running scaled up AE in UIMA AS 
> --------------------------------------------------------------------------------
>
>                 Key: UIMA-2078
>                 URL: https://issues.apache.org/jira/browse/UIMA-2078
>             Project: UIMA
>          Issue Type: Bug
>          Components: Async Scaleout, Core Java Framework
>            Reporter: Jerry Cwiklik
>            Assignee: Marshall Schor
>             Fix For: 2.4.0SDK
>
>
> While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:
> org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
> 	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
> 	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
> 	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
> 	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
> 	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
> 	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
> 	at java.lang.Thread.run(Thread.java:680)\
> Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
> 	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
> 	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\
> The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 
> Further diagnosis points toward an issue with the UIMA Java SDK, for use cases where multiple instances of an aggregate (containing a Cas Multiplier) are created and run using a shared common UIMA Context / Resource Manager.  Several parts of the Resource Manager are not thread safe.  
> Figure out where things like getEmptyCas need to be synchronized.
> Come up with a better test case (that isn't so intermittent).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Updated: (UIMA-2078) Fix intermittent problem with CAS release while running scaled up AE in UIMA AS

Posted by "Marshall Schor (JIRA)" <de...@uima.apache.org>.
     [ https://issues.apache.org/jira/browse/UIMA-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marshall Schor updated UIMA-2078:
---------------------------------

      Description: 
While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:

org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
	at java.lang.Thread.run(Thread.java:680)\
Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\


The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 

Further diagnosis points toward an issue with the UIMA Java SDK, for use cases where multiple instances of an aggregate (containing a Cas Multiplier) are created and run using a shared common UIMA Context / Resource Manager.  Several parts of the Resource Manager are not thread safe.  

Figure out where things like getEmptyCas need to be synchronized.
Come up with a better test case (that isn't so intermittent).

  was:
While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:

org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
	at java.lang.Thread.run(Thread.java:680)\
Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\


The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 

Postponing fix beyond 2.3.1 release

    Fix Version/s:     (was: 2.3.1AS)
                   2.3.2SDK
         Assignee: Marshall Schor  (was: Jerry Cwiklik)

> Fix intermittent problem with CAS release while running scaled up AE in UIMA AS 
> --------------------------------------------------------------------------------
>
>                 Key: UIMA-2078
>                 URL: https://issues.apache.org/jira/browse/UIMA-2078
>             Project: UIMA
>          Issue Type: Bug
>          Components: Async Scaleout, Core Java Framework
>            Reporter: Jerry Cwiklik
>            Assignee: Marshall Schor
>             Fix For: 2.3.2SDK
>
>
> While running extended tests on Mac, the testScaledSyncAggregateProcess() test failed with:
> org.apache.uima.analysis_engine.AnalysisEngineProcessException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:701)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:409)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:342)\
> 	at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:267)\
> 	at org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.process(PrimitiveAnalysisEngineController_impl.java:514)\
> 	at org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:121)\
> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handleProcessRequestFromRemoteClient(ProcessRequestHandler_impl.java:474)\
> 	at org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:952)\
> 	at org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:78)\
> 	at org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:702)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)\
> 	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)\
> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:323)\
> 	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:261)\
> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1056)\
> 	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)\
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\
> 	at org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFactory.java:106)\
> 	at java.lang.Thread.run(Thread.java:680)\
> Caused by: org.apache.uima.UIMARuntimeException: CasManager.release(CAS) was called with a CAS that does not belong to this CasManager.\
> 	at org.apache.uima.resource.impl.CasManager_impl.releaseCas(CasManager_impl.java:131)\
> 	at org.apache.uima.cas.AbstractCas_ImplBase.release(AbstractCas_ImplBase.java:35)\
> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3604)\
> 	at org.apache.uima.cas.impl.CASImpl.release(CASImpl.java:3602)\
> 	at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:689)\
> The above test deploys 5 instances of synchronous aggregate containing a CM and a NoOp Annotator. The CM generates 1 CAS for each input CAS. Total of 5 CASes are sent to the service for processing. Some of the CASes are processed and returned back to the client. I am not totally sure what the problem is at this time. It seems like the ASB is trying to release a child CAS when the above exception occurs. All instances of sync aggregates share Resource Manager containing common CasManager. Unable to reproduce this problem on Linux, Windows or Mac. 
> Further diagnosis points toward an issue with the UIMA Java SDK, for use cases where multiple instances of an aggregate (containing a Cas Multiplier) are created and run using a shared common UIMA Context / Resource Manager.  Several parts of the Resource Manager are not thread safe.  
> Figure out where things like getEmptyCas need to be synchronized.
> Come up with a better test case (that isn't so intermittent).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira