You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/08/17 22:15:52 UTC
Jenkins build became unstable: UIMA-Ruta » Apache UIMA Ruta: ruta-core #788
See <https://builds.apache.org/job/UIMA-Ruta/org.apache.uima$ruta-core/788/>
Jenkins build is back to stable : UIMA-Ruta » Apache UIMA Ruta: ruta-core #792
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/UIMA-Ruta/org.apache.uima$ruta-core/792/changes>
Jenkins build is still unstable: UIMA-Ruta » Apache UIMA Ruta: ruta-core #791
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/UIMA-Ruta/org.apache.uima$ruta-core/changes>
Jenkins build is still unstable: UIMA-Ruta » Apache UIMA Ruta: ruta-core #790
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/UIMA-Ruta/org.apache.uima$ruta-core/changes>
Re: Deep classloader switching fails Ruta tests - some thoughts
Posted by Peter Klügl <pe...@averbis.com>.
ruta-core was built successfully on jenkins :-)
Am 19.08.2016 um 11:09 schrieb Peter Klgl:
> Thanks Richard, I forgot that there are no problems at all if I use
> uimaFIT "correctly".
>
>
> +1 I will adapt ruta to use descriptions
>
>
> I do already some stuff you mention below, but the part with the inlined
> analysis engines was a bit neglected.
>
>
> Best,
>
>
> Peter
>
>
> Am 19.08.2016 um 10:08 schrieb Richard Eckart de Castilho:
>> I have been thinking a bit about this.
>>
>>> On 19.08.2016, at 09:57, Peter Klgl <pe...@averbis.com> wrote:
>>>
>>> What are our options concerning the three projects? The functionality in
>>> ruta calling analysis engines is needed (preferable without switching in
>>> future).
>> With the present release versions of UIMA/uimaFIT, the following should
>> be done when creating analysis engines inside other analysis engines:
>>
>> // use createEngineDescription instead of createEngine
>> AnalysisEngineDescription aed = createEngineDescription(MyNestedAE.class);
>>
>> // Obtain resource manager from enclosing AE
>> ResourceManager mgr = ((UimaContextAdmin) getContext()).getResourceManager();
>>
>> // Use that resource manager when creating the actual engine
>> AnalysisEngine engine = UIMAFramework.produceAnalysisEngine(aed, mgr, null);
>>
>>> Yes, uimaFIT should not create a resource manager, or at least there
>>> should be an option to reuse a given one...
>> There is no uimaFIT method for this because it can easily be
>> accomplished using UIMAFramework.
>>
>>> or at least a given classloader. The switching is somewhat problematic
>>> for me and Ruta, but did not cause too serious problems yet.
>> You could also do that by creating your own resource manager and using
>> it when creating the engine:
>>
>> ResourceManager resMgr = UIMAFramework.newDefaultResourceManager();
>> resMgr.setExtensionClassPath(myClassLoader, "", true);
>>
>>> There is of course ResourceManagerFactory.setResourceManagerCreator().
>>> With this, I could delegate the existing/ruta resource manager to the
>>> uimaFIT factories, but I do not know if I should really do this in a
>>> static manner.
>> I don't think that makes sense in this case. I added that function in
>> case somebody wants to use uimaFIT with the original UIMA behavior
>> (i.e. without using the Thread context classloader) or to experiment
>> with other classloading scenarios.
>>
>> What do you think?
>>
>> Cheers,
>>
>> -- Richard
>>
>>
Re: Deep classloader switching fails Ruta tests - some thoughts
Posted by Peter Klügl <pe...@averbis.com>.
Thanks Richard, I forgot that there are no problems at all if I use
uimaFIT "correctly".
+1 I will adapt ruta to use descriptions
I do already some stuff you mention below, but the part with the inlined
analysis engines was a bit neglected.
Best,
Peter
Am 19.08.2016 um 10:08 schrieb Richard Eckart de Castilho:
> I have been thinking a bit about this.
>
>> On 19.08.2016, at 09:57, Peter Klgl <pe...@averbis.com> wrote:
>>
>> What are our options concerning the three projects? The functionality in
>> ruta calling analysis engines is needed (preferable without switching in
>> future).
>
> With the present release versions of UIMA/uimaFIT, the following should
> be done when creating analysis engines inside other analysis engines:
>
> // use createEngineDescription instead of createEngine
> AnalysisEngineDescription aed = createEngineDescription(MyNestedAE.class);
>
> // Obtain resource manager from enclosing AE
> ResourceManager mgr = ((UimaContextAdmin) getContext()).getResourceManager();
>
> // Use that resource manager when creating the actual engine
> AnalysisEngine engine = UIMAFramework.produceAnalysisEngine(aed, mgr, null);
>
>> Yes, uimaFIT should not create a resource manager, or at least there
>> should be an option to reuse a given one...
>
> There is no uimaFIT method for this because it can easily be
> accomplished using UIMAFramework.
>
>> or at least a given classloader. The switching is somewhat problematic
>> for me and Ruta, but did not cause too serious problems yet.
>
> You could also do that by creating your own resource manager and using
> it when creating the engine:
>
> ResourceManager resMgr = UIMAFramework.newDefaultResourceManager();
> resMgr.setExtensionClassPath(myClassLoader, "", true);
>
>> There is of course ResourceManagerFactory.setResourceManagerCreator().
>> With this, I could delegate the existing/ruta resource manager to the
>> uimaFIT factories, but I do not know if I should really do this in a
>> static manner.
> I don't think that makes sense in this case. I added that function in
> case somebody wants to use uimaFIT with the original UIMA behavior
> (i.e. without using the Thread context classloader) or to experiment
> with other classloading scenarios.
>
> What do you think?
>
> Cheers,
>
> -- Richard
>
>
Re: Deep classloader switching fails Ruta tests - some thoughts
Posted by Richard Eckart de Castilho <re...@apache.org>.
I have been thinking a bit about this.
> On 19.08.2016, at 09:57, Peter Klügl <pe...@averbis.com> wrote:
>
> What are our options concerning the three projects? The functionality in
> ruta calling analysis engines is needed (preferable without switching in
> future).
With the present release versions of UIMA/uimaFIT, the following should
be done when creating analysis engines inside other analysis engines:
// use createEngineDescription instead of createEngine
AnalysisEngineDescription aed = createEngineDescription(MyNestedAE.class);
// Obtain resource manager from enclosing AE
ResourceManager mgr = ((UimaContextAdmin) getContext()).getResourceManager();
// Use that resource manager when creating the actual engine
AnalysisEngine engine = UIMAFramework.produceAnalysisEngine(aed, mgr, null);
> Yes, uimaFIT should not create a resource manager, or at least there
> should be an option to reuse a given one...
There is no uimaFIT method for this because it can easily be
accomplished using UIMAFramework.
> or at least a given classloader. The switching is somewhat problematic
> for me and Ruta, but did not cause too serious problems yet.
You could also do that by creating your own resource manager and using
it when creating the engine:
ResourceManager resMgr = UIMAFramework.newDefaultResourceManager();
resMgr.setExtensionClassPath(myClassLoader, "", true);
> There is of course ResourceManagerFactory.setResourceManagerCreator().
> With this, I could delegate the existing/ruta resource manager to the
> uimaFIT factories, but I do not know if I should really do this in a
> static manner.
I don't think that makes sense in this case. I added that function in
case somebody wants to use uimaFIT with the original UIMA behavior
(i.e. without using the Thread context classloader) or to experiment
with other classloading scenarios.
What do you think?
Cheers,
-- Richard
Re: Deep classloader switching fails Ruta tests - some thoughts
Posted by Peter Klügl <pe...@averbis.com>.
Yes, uimaFIT should not create a resource manager, or at least there
should be an option to reuse a given one... or at least a given
classloader. The switching is somewhat problematic for me and Ruta, but
did not cause too serious problems yet.
There is of course ResourceManagerFactory.setResourceManagerCreator().
With this, I could delegate the existing/ruta resource manager to the
uimaFIT factories, but I do not know if I should really do this in a
static manner.
I do not know if we should/need support multiple levels of nesting.
What are our options concerning the three projects? The functionality in
ruta calling analysis engines is needed (preferable without switching in
future). The next uimaFIT release is not planned yet, right? If we keep
the exception in uimaj before the switching is addressed in uimaFIT,
then I have a serious problem in Ruta.
Best,
Peter
Am 19.08.2016 um 07:53 schrieb Richard Eckart de Castilho:
> Hm, I think this could also be addressed via the UimaContextHolder that was
> discussed elsewhere. uimaFIT could check if there is an active context
> and re-use its resource manager instead of creating a new one.
>
> What do you think?
>
> Cheers,
>
> -- Richard
>
>> On 18.08.2016, at 23:16, Marshall Schor <ms...@schor.com> wrote:
>>
>> Some other things to consider:
>>
>> The way Ruta is using uimaFIT in the failing case results in multiple analysis
>> engines created with different resource managers, but the CAS from one is (I
>> think - didn't check this...) sent through the AE of another.
>>
>> When multiple AEs are used this way with a common CAS, then the type system,
>> type priorities and indexes definition associated with this common CAS are used.
>> So it makes sense to share these parts of the Resource Manager.
>>
>> And, if the intent is to have JCas cover classes produced by one of the AEs
>> usable by the other AEs, then there should not be extra UIMA Class loaders, and
>> class loader switching. This is because one of the things that class loader
>> switching does is to swap in/ out different JCasHashMaps. These are maps that
>> associated already created JCas instances with specific Cas Feature Structures.
>> This could be important, if the JCas instance was carrying along extra
>> information (not in the CAS) that you wanted shared with other
>> class-loading-contexts (e.g., an "outer" AE, or an "application").
>>
>> -Marshall
Re: Deep classloader switching fails Ruta tests - some thoughts
Posted by Richard Eckart de Castilho <re...@apache.org>.
Hm, I think this could also be addressed via the UimaContextHolder that was
discussed elsewhere. uimaFIT could check if there is an active context
and re-use its resource manager instead of creating a new one.
What do you think?
Cheers,
-- Richard
> On 18.08.2016, at 23:16, Marshall Schor <ms...@schor.com> wrote:
>
> Some other things to consider:
>
> The way Ruta is using uimaFIT in the failing case results in multiple analysis
> engines created with different resource managers, but the CAS from one is (I
> think - didn't check this...) sent through the AE of another.
>
> When multiple AEs are used this way with a common CAS, then the type system,
> type priorities and indexes definition associated with this common CAS are used.
> So it makes sense to share these parts of the Resource Manager.
>
> And, if the intent is to have JCas cover classes produced by one of the AEs
> usable by the other AEs, then there should not be extra UIMA Class loaders, and
> class loader switching. This is because one of the things that class loader
> switching does is to swap in/ out different JCasHashMaps. These are maps that
> associated already created JCas instances with specific Cas Feature Structures.
> This could be important, if the JCas instance was carrying along extra
> information (not in the CAS) that you wanted shared with other
> class-loading-contexts (e.g., an "outer" AE, or an "application").
>
> -Marshall
Re: Deep classloader switching fails Ruta tests - some thoughts
Posted by Marshall Schor <ms...@schor.com>.
Some other things to consider:
The way Ruta is using uimaFIT in the failing case results in multiple analysis
engines created with different resource managers, but the CAS from one is (I
think - didn't check this...) sent through the AE of another.
When multiple AEs are used this way with a common CAS, then the type system,
type priorities and indexes definition associated with this common CAS are used.
So it makes sense to share these parts of the Resource Manager.
And, if the intent is to have JCas cover classes produced by one of the AEs
usable by the other AEs, then there should not be extra UIMA Class loaders, and
class loader switching. This is because one of the things that class loader
switching does is to swap in/ out different JCasHashMaps. These are maps that
associated already created JCas instances with specific Cas Feature Structures.
This could be important, if the JCas instance was carrying along extra
information (not in the CAS) that you wanted shared with other
class-loading-contexts (e.g., an "outer" AE, or an "application").
-Marshall
Re: Deep classloader switching fails Ruta tests - here's the analysis
Posted by Marshall Schor <ms...@schor.com>.
It didn't fail for me on trunk when running the whole test suite. but it did
fail running just the one test UimaFitAnalysisEngineWithManditoryParameterTest.
I put in a catcher to see where all the class loaders are being created, and
they're all (all 8 of them!) being created as a result of the test case line 66:
AnalysisEngine ae = AnalysisEngineFactory.createEngine(RutaEngine.class,
RutaEngine.PARAM_RULES, script);
The uimaFIT code for createEngineDescription has:
TypeSystemDescription typeSystem = createTypeSystemDescription();
TypePriorities typePriorities = createTypePriorities();
...
createEngineDescription(...)
Each of these calls ends up creating a newResourceManager, which in turn is
setting up a new UIMA Extension Class Path.
Following the creation of the description, createEngine(desc) is called. This
creates another UIMA Class loader because of line 205 in AnalysisEngineFactory
(the call to newResourceManager()).
Then the produceAnalysisEngine eventually calls the "initialize" method, which
then runs RutaEngine.initialize, which calls
addUimafitAnalsysEngine, which then creates another Engine Description (doing
another 2 classloaders, one for the type system description, the other for the
type priorities (see above) ), and then creates the engine description from
those (another classloader),
and then creates an Engine from that description.
All of the extra class loader instances are mostly thrown away I think, but
there are two nested Engines, which is causing this issue.
So, one fix is to not have uimaFIT stick in the placeholder UIMA extension
loader every time it makes an Engine.
Another fix is to extend core UIMA to support multiple levels of nesting.
Any Opinions?
-Marshall
Re: Deep classloader switching fails Ruta tests
Posted by Marshall Schor <ms...@schor.com>.
ok, I'll take a look.
On 8/18/2016 1:20 PM, Richard Eckart de Castilho wrote:
> On 18.08.2016, at 08:39, Peter Kl�gl <pe...@averbis.com> wrote:
>> No idea yet why the tests fail...
>>
>> Am 18.08.2016 um 02:29 schrieb Apache Jenkins Server:
>>> See <https://builds.apache.org/job/UIMA-Ruta/org.apache.uima$ruta-core/789/>
>
>> On 17.08.2016, at 19:18, Marshall Schor (JIRA) <de...@uima.apache.org> wrote:
>> Marshall Schor resolved UIMA-5056.
>>
>> following Richard's suggestion, added exception if more than 1 nesting switch for switching class loaders.
>
> @Marshall/Peter: Looks like we found a case where classloader switching *does* go across more than 2 levels (no idea if this is intentional or not):
>
> Caused by: org.apache.uima.cas.CASRuntimeException: Multiply nested classloaders not supported. Original base loader: sun.misc.Launcher$AppClassLoader@712801c5, current nested loader: org.apache.uima.internal.util.UIMAClassLoader@3f89fc46, trying to switch to loader: org.apache.uima.internal.util.UIMAClassLoader@3e872ac6.
> at org.apache.uima.cas.impl.CASImpl.switchClassLoader(CASImpl.java:4257)
> at org.apache.uima.cas.impl.CASImpl.switchClassLoaderLockCasCL(CASImpl.java:4246)
> at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentProcess(PrimitiveAnalysisEngine_impl.java:382)
> ... 46 more
>
> Cheers,
>
> -- Richard
Deep classloader switching fails Ruta tests
Posted by Richard Eckart de Castilho <re...@apache.org>.
On 18.08.2016, at 08:39, Peter Klügl <pe...@averbis.com> wrote:
>
> No idea yet why the tests fail...
>
> Am 18.08.2016 um 02:29 schrieb Apache Jenkins Server:
>> See <https://builds.apache.org/job/UIMA-Ruta/org.apache.uima$ruta-core/789/>
> On 17.08.2016, at 19:18, Marshall Schor (JIRA) <de...@uima.apache.org> wrote:
> Marshall Schor resolved UIMA-5056.
>
> following Richard's suggestion, added exception if more than 1 nesting switch for switching class loaders.
@Marshall/Peter: Looks like we found a case where classloader switching *does* go across more than 2 levels (no idea if this is intentional or not):
Caused by: org.apache.uima.cas.CASRuntimeException: Multiply nested classloaders not supported. Original base loader: sun.misc.Launcher$AppClassLoader@712801c5, current nested loader: org.apache.uima.internal.util.UIMAClassLoader@3f89fc46, trying to switch to loader: org.apache.uima.internal.util.UIMAClassLoader@3e872ac6.
at org.apache.uima.cas.impl.CASImpl.switchClassLoader(CASImpl.java:4257)
at org.apache.uima.cas.impl.CASImpl.switchClassLoaderLockCasCL(CASImpl.java:4246)
at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentProcess(PrimitiveAnalysisEngine_impl.java:382)
... 46 more
Cheers,
-- Richard
Re: Jenkins build is still unstable: UIMA-Ruta » Apache UIMA Ruta: ruta-core #789
Posted by Peter Klügl <pe...@averbis.com>.
No idea yet why the tests fail...
Am 18.08.2016 um 02:29 schrieb Apache Jenkins Server:
> See <https://builds.apache.org/job/UIMA-Ruta/org.apache.uima$ruta-core/789/>
>
Jenkins build is still unstable: UIMA-Ruta » Apache UIMA Ruta: ruta-core #789
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/UIMA-Ruta/org.apache.uima$ruta-core/789/>