clearing the JCas Cache in some circumstances

[Marshall Schor <ms...@schor.com>]

The JCas Cache is a specialized hash table that enables iterators and 
reference following from CAS Feature Structures to return the exact same 
JCas cover Java instance that was previously returned.  Besides avoiding 
some GC overhead for the alternative (which is to create a new Java 
object each time rather than reuse the previously created one), there is 
a logical consequence for this in the case where a JCas cover class had 
additional features added manually which want to be remembered / 
retained between references; for this case, the cache is needed.

In addition to the capability to completely disable the JCas Cache 
(already implemented), here are some ideas around resetting the JCas 
cache to consider:

In applications where some annotators are put together into a pipeline, 
some of the annotators may be using JCas, and other may not.  For common 
flows (linear fixed flow, for instance), if, at some point in the 
pipeline, no further downstream annotators were using JCas, it would be 
possible to reset the JCas cache, because there would be no further use 
of it.

This could be done programmatically, by an annotator, for instance, or 
by the framework. A trivial annotator could be written which does 
nothing except reset the cache, and then inserted by a deployer in the 
flow at the right spot.  Or, the reset could be done as part of another 
annotator's work. If it was done by the framework, the framework would 
have to have some indication of when it could/should do this reset. 

This could be in the form of some additional specification, or the 
framework could "deduce" when to do this by performing a flow analysis 
(for simple linear flows) and looking at the kind of interface the 
annotator implemented (JCas or plain CAS). This inference would be 
approximate, though, because there are "corner cases" which may not 
occur often in practice but are still possible.  For instance, an 
annotator could implement the CAS interface, and still use the JCas 
inside itself for some things.

Another interesting case to consider is when a Pear specifier is used 
inside an aggregate.  This usage switches the class loader for the scope 
of the Pear.  Because the new class loader could load alternative JCas 
cover classes, a separate JCas cache is kept for each Pear.  It would 
seem reasonable to reset that cache when the scope of the Pear specifier 
is exited.  A corner case here might be if the flow routed a CAS back to 
the same Pear specifier multiple times.

Another kind of optimization that could be considered would be to allow 
cache reset by CAS Type (including subtypes, I think) - so a flow 
analysis coupled with the component capability specification could be 
used (at initialize time) to figure out when each particular CAS Type 
was no longer used in a pipeline.  This would require some redesign of 
the cache - which today isn't split along CAS Types - which could have 
some space cost tradeoffs to consider.

Of the various ideas above, I think the best ones are the programmatic 
reset (letting the application / annotator designer or deployer decide 
when to reset it), and possibly the reset for the Pear-specific cache 
when exiting the Pear specifier scope.  I'm not sure there's enough of a 
use case to justify adding a capability to partially reset the cache by 
CAS Type/subtype.  Other opinions?

-Marshall

Re: clearing the JCas Cache in some circumstances

[Thilo Goetz <tw...@gmx.de>]

I don't know.  Aren't we designing something that 99%
of our users are never going to use?  Is this really
worth the added complexity of the API and documentation?
I'm -0 to a user-facing API to clear the JCas cache.

--Thilo

Marshall Schor wrote:
> The JCas Cache is a specialized hash table that enables iterators and 
> reference following from CAS Feature Structures to return the exact same 
> JCas cover Java instance that was previously returned.  Besides avoiding 
> some GC overhead for the alternative (which is to create a new Java 
> object each time rather than reuse the previously created one), there is 
> a logical consequence for this in the case where a JCas cover class had 
> additional features added manually which want to be remembered / 
> retained between references; for this case, the cache is needed.
> 
> In addition to the capability to completely disable the JCas Cache 
> (already implemented), here are some ideas around resetting the JCas 
> cache to consider:
> 
> In applications where some annotators are put together into a pipeline, 
> some of the annotators may be using JCas, and other may not.  For common 
> flows (linear fixed flow, for instance), if, at some point in the 
> pipeline, no further downstream annotators were using JCas, it would be 
> possible to reset the JCas cache, because there would be no further use 
> of it.
> 
> This could be done programmatically, by an annotator, for instance, or 
> by the framework. A trivial annotator could be written which does 
> nothing except reset the cache, and then inserted by a deployer in the 
> flow at the right spot.  Or, the reset could be done as part of another 
> annotator's work. If it was done by the framework, the framework would 
> have to have some indication of when it could/should do this reset.
> This could be in the form of some additional specification, or the 
> framework could "deduce" when to do this by performing a flow analysis 
> (for simple linear flows) and looking at the kind of interface the 
> annotator implemented (JCas or plain CAS). This inference would be 
> approximate, though, because there are "corner cases" which may not 
> occur often in practice but are still possible.  For instance, an 
> annotator could implement the CAS interface, and still use the JCas 
> inside itself for some things.
> 
> Another interesting case to consider is when a Pear specifier is used 
> inside an aggregate.  This usage switches the class loader for the scope 
> of the Pear.  Because the new class loader could load alternative JCas 
> cover classes, a separate JCas cache is kept for each Pear.  It would 
> seem reasonable to reset that cache when the scope of the Pear specifier 
> is exited.  A corner case here might be if the flow routed a CAS back to 
> the same Pear specifier multiple times.
> 
> Another kind of optimization that could be considered would be to allow 
> cache reset by CAS Type (including subtypes, I think) - so a flow 
> analysis coupled with the component capability specification could be 
> used (at initialize time) to figure out when each particular CAS Type 
> was no longer used in a pipeline.  This would require some redesign of 
> the cache - which today isn't split along CAS Types - which could have 
> some space cost tradeoffs to consider.
> 
> Of the various ideas above, I think the best ones are the programmatic 
> reset (letting the application / annotator designer or deployer decide 
> when to reset it), and possibly the reset for the Pear-specific cache 
> when exiting the Pear specifier scope.  I'm not sure there's enough of a 
> use case to justify adding a capability to partially reset the cache by 
> CAS Type/subtype.  Other opinions?
> 
> -Marshall
>