You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Adam Lally <al...@alum.rpi.edu> on 2006/12/14 18:12:18 UTC
DocumentAnnotation and type-merging
Yes, this topic again...
While working on a utility to migrate code from IBM UIMA to Apache
UIMA, I encountered the case where the user's project has a definition
of com.ibm.uima.jcas.tcas.DocumentAnnotation. That's because JCasGen
creates this, to account for cases where the user has defined their
own features to add to DocumentAnnotation.
As far as the migration goes, this would need to become
org.apache.uima.jcas.tcas.DocumentAnnotation. But I can't do that
with search-and-replace, I'd actually have to move the file to a
different directory. Instead of doing that, this may become a manual
migration step. (If the user did not add custom code, just deleting
this file and rerunning JCasGen is sufficient.)
It also occurred to me that we lost uima_jcas_builtin_types.jar as
part of our Maven restructuring. This jar contained the
DocumentAnnotation class, which was not in uima_core.jar. This was
done so that applications could exclude this jar from their classpath
to allow a user-generated DocumentAnnotation class to be used instead
of the one that ships with UIMA. But this is pretty ugly.
Questions:
1) Do we continue to support adding features to DocumentAnnotation?
2) If so, should we delete the JCAS DocumentAnnotation class from the
framework code entirely, to avoid the problem that required the
uima_jcas_builtin_types.jar workaround?
3) What should the migration story be for existing user code?
-Adam
Re: DocumentAnnotation and type-merging
Posted by Adam Lally <al...@alum.rpi.edu>.
On 1/9/07, Michael Baessler <mb...@michael-baessler.de> wrote:
> Adam Lally wrote:
> > For (c) I think I will have to create a new component
> > uimaj-document-annotation that has just the one Java class
> > (org.apache.uima.jcas.tcas.DocumentAnnotation) in it, and which builds
> > into uimaj-document-annotation.jar. It's a little ugly but I haven't
> > heard a better suggestion. Agreed?
> >
> Right now, it seems to be the only way. Fine with me.
>
There's a further complication. Moving DocumentAnnotation out of
uimaj-core means that there can be no references to it from anywhere
in uimaj-core (otherwise we have a circular dependency problem).
Fortunately, there are only two references.
One reference is in JCAS.getDocumentAnnotation(). This is already
deprecated, because it's unsafe when the DocumentAnnotation is loaded
via the UIMA extension ClassLoader. We would need to delete it. That
might break some code, but maybe that's OK since there have been
deprecation warnings there for a while now. OK?
The other is from FileSystemCollectionReader. Why is this in
uimaj-core, you ask? This I can just move to uimaj-tools, where it
probably belongs anyway.
> What about the documentation. I think we should add a paragraph about
> the uimaj-document-annotation.jar and how it should be used/replaced.
> There is already a chapter in the documentation writing about issues
> with the document annotation, but this mainly talks about the extension
> class loader issues.
>
Yes, I agree we need a documentation update. I would like this to
read like a warning label on a pack of cigarettes (at least the ones
in the U.S.). If you extend DocumentAnnotation while using JCAS, your
application it may be OK for now but after a few years when your
application "grows up" it may suffer a painful death. :-)
-Adam
Re: DocumentAnnotation and type-merging
Posted by Michael Baessler <mb...@michael-baessler.de>.
Adam Lally wrote:
> On 12/22/06, Adam Lally <al...@alum.rpi.edu> wrote:
>> On 12/22/06, Marshall Schor <ms...@schor.com> wrote:
>> > Re: What to do about Document Annotation for 2.1.
>> >
>> > a) Do the work to make it easy to get singletons (or whatever we're
>> > calling this feature) out of the CAS
>> >
>> > b) Change JCasGen to not generate DocumentAnnotation if the merged
>> > version = the base
>> >
>> > c) Add the DocumentAnnotation jar back to the build, with instructions
>> > to remove it from the
>> > deployment and substitute a JCasGenerated one if the application
>> needs a
>> > special one.
>> >
>> > I've may have forgotten some point(s) - does this address the issues?
>>
>>
>> My only complaints are that I think it is ugly to have a
>> uima-jcas-builtin-types.jar (or uima-document-annotation.jar or
>> whatever its called) that has just this one class in it, and I think
>> this this is a dangerous feature that users can hang themselves with.
>>
>> But, I suppose I can live with this solution... keeping support for
>> the feature so we don't break compatibility, but putting sufficient
>> warnings in to do our best to keep users from hanging themselves with
>> it.
>>
>> I do think that it seems like an improvement to not generate a new
>> version of DocumentAnnotation if the merged version = the base. Why
>> didn't we think of that before?
>>
>
>
> I created a JIRA issue for this.
>
> I'm pretty sure (a) is still controversial, and I suppose not strictly
> needed for v2.1. I'll post on a separate thread to at least get the
> ball rolling.
>
> For (c) I think I will have to create a new component
> uimaj-document-annotation that has just the one Java class
> (org.apache.uima.jcas.tcas.DocumentAnnotation) in it, and which builds
> into uimaj-document-annotation.jar. It's a little ugly but I haven't
> heard a better suggestion. Agreed?
>
> -Adam
Right now, it seems to be the only way. Fine with me.
What about the documentation. I think we should add a paragraph about
the uimaj-document-annotation.jar and how it should be used/replaced.
There is already a chapter in the documentation writing about issues
with the document annotation, but this mainly talks about the extension
class loader issues.
-- Michael
Re: DocumentAnnotation and type-merging
Posted by Adam Lally <al...@alum.rpi.edu>.
On 12/22/06, Adam Lally <al...@alum.rpi.edu> wrote:
> On 12/22/06, Marshall Schor <ms...@schor.com> wrote:
> > Re: What to do about Document Annotation for 2.1.
> >
> > a) Do the work to make it easy to get singletons (or whatever we're
> > calling this feature) out of the CAS
> >
> > b) Change JCasGen to not generate DocumentAnnotation if the merged
> > version = the base
> >
> > c) Add the DocumentAnnotation jar back to the build, with instructions
> > to remove it from the
> > deployment and substitute a JCasGenerated one if the application needs a
> > special one.
> >
> > I've may have forgotten some point(s) - does this address the issues?
>
>
> My only complaints are that I think it is ugly to have a
> uima-jcas-builtin-types.jar (or uima-document-annotation.jar or
> whatever its called) that has just this one class in it, and I think
> this this is a dangerous feature that users can hang themselves with.
>
> But, I suppose I can live with this solution... keeping support for
> the feature so we don't break compatibility, but putting sufficient
> warnings in to do our best to keep users from hanging themselves with
> it.
>
> I do think that it seems like an improvement to not generate a new
> version of DocumentAnnotation if the merged version = the base. Why
> didn't we think of that before?
>
I created a JIRA issue for this.
I'm pretty sure (a) is still controversial, and I suppose not strictly
needed for v2.1. I'll post on a separate thread to at least get the
ball rolling.
For (c) I think I will have to create a new component
uimaj-document-annotation that has just the one Java class
(org.apache.uima.jcas.tcas.DocumentAnnotation) in it, and which builds
into uimaj-document-annotation.jar. It's a little ugly but I haven't
heard a better suggestion. Agreed?
-Adam
Re: DocumentAnnotation and type-merging
Posted by Adam Lally <al...@alum.rpi.edu>.
On 12/22/06, Marshall Schor <ms...@schor.com> wrote:
> Re: What to do about Document Annotation for 2.1.
>
> a) Do the work to make it easy to get singletons (or whatever we're
> calling this feature) out of the CAS
>
> b) Change JCasGen to not generate DocumentAnnotation if the merged
> version = the base
>
> c) Add the DocumentAnnotation jar back to the build, with instructions
> to remove it from the
> deployment and substitute a JCasGenerated one if the application needs a
> special one.
>
> I've may have forgotten some point(s) - does this address the issues?
My only complaints are that I think it is ugly to have a
uima-jcas-builtin-types.jar (or uima-document-annotation.jar or
whatever its called) that has just this one class in it, and I think
this this is a dangerous feature that users can hang themselves with.
But, I suppose I can live with this solution... keeping support for
the feature so we don't break compatibility, but putting sufficient
warnings in to do our best to keep users from hanging themselves with
it.
I do think that it seems like an improvement to not generate a new
version of DocumentAnnotation if the merged version = the base. Why
didn't we think of that before?
-Adam
Re: DocumentAnnotation and type-merging
Posted by Marshall Schor <ms...@schor.com>.
Re: What to say about type merging and JCas -
I like your point: "Be sure that all the
users of your component are willing and have the know-how to run
JCasGen when they combine your component with others."
And I would use your following point: "that would be a dangerous assumption
for any annotator developer to make"
and recommend to the Annotator Developer that they do not
depend on other users using feature extension, if they want their annotator
to be easily usable by others. If they insist on this, then we should
ask them
to include the steps in my note below as part of their documentation on
how users make use of their component. :-)
Re: What to do about Document Annotation for 2.1.
a) Do the work to make it easy to get singletons (or whatever we're
calling this feature) out of the CAS
b) Change JCasGen to not generate DocumentAnnotation if the merged
version = the base
c) Add the DocumentAnnotation jar back to the build, with instructions
to remove it from the
deployment and substitute a JCasGenerated one if the application needs a
special one.
I've may have forgotten some point(s) - does this address the issues?
-Marshall
Adam Lally wrote:
> On 12/19/06, Marshall Schor <ms...@schor.com> wrote:
>> How about: We explicitly document that if "feature extension" is used
>> with
>> JCas, you need to
>>
>> (a) package type systems and the JCas generated classes
>> separately from other packagings, and
>>
>> (b) be willing to re-run JCasGen when
>> your type package is combined with others, and
>>
>> (c) if hand-modifications are
>> done to the generated JCas classes, then that version of the source
>> must be
>> available when running JCasGen and the "merge" option must be used.
>>
>> Limitation: if multiple versions of the JCasGen generated sources were
>> independently
>> hand-modified, then the assembler has to hand-merge the resulting source
>> for those
>> types to pick up all the hand-modifications into the result.
>>
>
>
> Can we put all that in a warning dialog that pops up from the CDE and
> JCasGen? That ought to be enough to discourage users from doing this.
> :)
>
> But I would say that (b) should really say: "Be sure that all the
> users of your component are willing and have the know-how to run
> JCasGen when they combine your component with others."
>
> But the trouble is that I think that would be a dangerous assumption
> for any annotator developer to make.
>
>
>
>>> > This seems particularly important for applications that host
>>> arbitrary
>>> > UIMA analytics. End users want to grab the latest, greatest
>>> annotator
>>> > and drop it in. This should work smoothly, or UIMA isn't meeting one
>>> > of its most important goals.
>>> I agree. Perhaps we should figure out what (if anything) is inhibiting
>>> this,
>>> and see if it can be be addressed. One concept might be to require
>>> JCas
>>> source/class files to be packaged in a particular way, and to improve
>>> the
>>> "merge" logic to cover more cases (and report on the cases where it
>>> fails
>>> and a "manual" merge step might be needed). I think in most pragmatic
>>> cases it will work fine "automatically".
>
> I'm wary of the complexity that's introduced when we have to muck with
> source code at assembly time. In principle I could see how we can
> make this work if annotators always included their JCAS cover classes
> in source form (if they had any extensions) in a certain specified wa
> of packaging, and if had an enahcned merge utility (like what's in
> SVN) that could merge source files as long as they didn't have
> overlapping edits, and we required that this be done as part of
> aggregate assembly (so assembly now requires using a tool or invoking
> an API, and can no longer be safely done by working with XML). But
> I'm not sure the added complexity is worth the gain, versus the
> alternative of asking people to use typical programming practices to
> manage their JCAS classes as they would other shared Java code (i.e.,
> people have to negotiate to create the one standard version that they
> can all share).
>
> Anyway, I doubt we're solving this problem for v2.1, so in the short
> term what can we do about it? And specifically, what can we do about
> DocumentAnnotation?
>
> -Adam
>
>
Re: DocumentAnnotation and type-merging
Posted by Adam Lally <al...@alum.rpi.edu>.
On 12/19/06, Marshall Schor <ms...@schor.com> wrote:
> How about: We explicitly document that if "feature extension" is used with
> JCas, you need to
>
> (a) package type systems and the JCas generated classes
> separately from other packagings, and
>
> (b) be willing to re-run JCasGen when
> your type package is combined with others, and
>
> (c) if hand-modifications are
> done to the generated JCas classes, then that version of the source must be
> available when running JCasGen and the "merge" option must be used.
>
> Limitation: if multiple versions of the JCasGen generated sources were
> independently
> hand-modified, then the assembler has to hand-merge the resulting source
> for those
> types to pick up all the hand-modifications into the result.
>
Can we put all that in a warning dialog that pops up from the CDE and
JCasGen? That ought to be enough to discourage users from doing this.
:)
But I would say that (b) should really say: "Be sure that all the
users of your component are willing and have the know-how to run
JCasGen when they combine your component with others."
But the trouble is that I think that would be a dangerous assumption
for any annotator developer to make.
>> > This seems particularly important for applications that host arbitrary
>> > UIMA analytics. End users want to grab the latest, greatest annotator
>> > and drop it in. This should work smoothly, or UIMA isn't meeting one
>> > of its most important goals.
>> I agree. Perhaps we should figure out what (if anything) is inhibiting
>> this,
>> and see if it can be be addressed. One concept might be to require JCas
>> source/class files to be packaged in a particular way, and to improve
>> the
>> "merge" logic to cover more cases (and report on the cases where it
>> fails
>> and a "manual" merge step might be needed). I think in most pragmatic
>> cases it will work fine "automatically".
I'm wary of the complexity that's introduced when we have to muck with
source code at assembly time. In principle I could see how we can
make this work if annotators always included their JCAS cover classes
in source form (if they had any extensions) in a certain specified wa
of packaging, and if had an enahcned merge utility (like what's in
SVN) that could merge source files as long as they didn't have
overlapping edits, and we required that this be done as part of
aggregate assembly (so assembly now requires using a tool or invoking
an API, and can no longer be safely done by working with XML). But
I'm not sure the added complexity is worth the gain, versus the
alternative of asking people to use typical programming practices to
manage their JCAS classes as they would other shared Java code (i.e.,
people have to negotiate to create the one standard version that they
can all share).
Anyway, I doubt we're solving this problem for v2.1, so in the short
term what can we do about it? And specifically, what can we do about
DocumentAnnotation?
-Adam
Re: DocumentAnnotation and type-merging
Posted by Marshall Schor <ms...@schor.com>.
Adam Lally wrote:
> On 12/18/06, Marshall Schor <ms...@schor.com> wrote:
>> I think what we have is annotator A might have a version of types for it
>> (T/A) and
>> annotator B might have a version of types for it (T/B). The "assembly"
>> of A and B
>> has a process whereby T/A and T/B are "merged", and a new T/A&B is
>> created. This seems different form the Xerces example. It's not
>> ncesessarily a "newer"
>> versus "older" thing, for UIMA assemblies.
>
> To me it's very similar, the difference being that in most cases it
> would be easier to merge T/A and T/B than it would be the merge Xerces
> versions X and Y. But in both cases there are additional options
> other than merging:
> 1) If one version is newer than another and backwards compatible then
> you can just use the newer one. I think this can be common in UIMA,
> where someone has extended an existing type.
True, but I think many times the extension is not
backward compatible (e.g. multiple
components extending the DocumentAnnotation differently).
> 2) You can use separate ClassLoaders for the two components so that
> they can each use their own version of the class. This would also
> mean that they can't share the JCAS cover objects directly, but would
> each need their own instance. But that seems doable, and perhaps
> simpler than the automatic merge and more "normal" in the Java world.
Interesting... But this would reduce the effect of an optimization we
have now when
running multiple components in the same JVM (the JCas class instances are
reused). Each having their own instance would imply more Java objects.
>> I agree this is a good goal. It seems achievable with some "automation"
>> introduced
>> into the assembly step.
>> > This seems particularly important for applications that host arbitrary
>> > UIMA analytics. End users want to grab the latest, greatest annotator
>> > and drop it in. This should work smoothly, or UIMA isn't meeting one
>> > of its most important goals.
>> I agree. Perhaps we should figure out what (if anything) is inhibiting
>> this,
>> and see if it can be be addressed. One concept might be to require JCas
>> source/class files to be packaged in a particular way, and to improve
>> the
>> "merge" logic to cover more cases (and report on the cases where it
>> fails
>> and a "manual" merge step might be needed). I think in most pragmatic
>> cases it will work fine "automatically".
>
> What about the following adjustment to my original proposal:
>
> We explicitly document that JCAS and "feature extension" are not
> compatible in the *current* version of UIMA, but that we may figure
> out some way to make them compatible in the future.
I think this might be too strong a statement. If you're willing to run
JCasGen
and compile the results as part of an assembly process, it seems there
is a way to
make JCas compatible up to a point with feature extension, using the
fact that JCasGen will merge sources to handle keeping hand-done
customizations (in most cases).
How about: We explicitly document that if "feature extension" is used with
JCas, you need to
(a) package type systems and the JCas generated classes
separately from other packagings, and
(b) be willing to re-run JCasGen when
your type package is combined with others, and
(c) if hand-modifications are
done to the generated JCas classes, then that version of the source must be
available when running JCasGen and the "merge" option must be used.
Limitation: if multiple versions of the JCasGen generated sources were
independently
hand-modified, then the assembler has to hand-merge the resulting source
for those
types to pick up all the hand-modifications into the result.
-Marshall
Re: DocumentAnnotation and type-merging
Posted by Adam Lally <al...@alum.rpi.edu>.
On 12/18/06, Marshall Schor <ms...@schor.com> wrote:
> I think what we have is annotator A might have a version of types for it
> (T/A) and
> annotator B might have a version of types for it (T/B). The "assembly"
> of A and B
> has a process whereby T/A and T/B are "merged", and a new T/A&B is
> created. This seems different form the Xerces example. It's not
> ncesessarily a "newer"
> versus "older" thing, for UIMA assemblies.
To me it's very similar, the difference being that in most cases it
would be easier to merge T/A and T/B than it would be the merge Xerces
versions X and Y. But in both cases there are additional options
other than merging:
1) If one version is newer than another and backwards compatible then
you can just use the newer one. I think this can be common in UIMA,
where someone has extended an existing type.
2) You can use separate ClassLoaders for the two components so that
they can each use their own version of the class. This would also
mean that they can't share the JCAS cover objects directly, but would
each need their own instance. But that seems doable, and perhaps
simpler than the automatic merge and more "normal" in the Java world.
> I agree this is a good goal. It seems achievable with some "automation"
> introduced
> into the assembly step.
> > This seems particularly important for applications that host arbitrary
> > UIMA analytics. End users want to grab the latest, greatest annotator
> > and drop it in. This should work smoothly, or UIMA isn't meeting one
> > of its most important goals.
> I agree. Perhaps we should figure out what (if anything) is inhibiting
> this,
> and see if it can be be addressed. One concept might be to require JCas
> source/class files to be packaged in a particular way, and to improve the
> "merge" logic to cover more cases (and report on the cases where it fails
> and a "manual" merge step might be needed). I think in most pragmatic
> cases it will work fine "automatically".
What about the following adjustment to my original proposal:
We explicitly document that JCAS and "feature extension" are not
compatible in the *current* version of UIMA, but that we may figure
out some way to make them compatible in the future.
-Adam
Re: DocumentAnnotation and type-merging
Posted by Marshall Schor <ms...@schor.com>.
Adam Lally wrote:
> On 12/18/06, Marshall Schor <ms...@schor.com> wrote:
>> It seems to me that developers choose to put data into a CAS because
>> they envision "sharing"
>> that data with other (independently-developed) components. If they're
>> planning
>> to do that, then the "definitions" of the types they're sharing in some
>> sense
>> naturally belong to multiple components (the components that are
>> "sharing" that
>> type definition).
>>
>> Thus, it seems a little bit illogical to envision the standard packaging
>> of types
>> go with a particular "component" - a better practice might be thinking
>> of types
>> more as 1st class parts in themselves.
>
> Hmmm.. good point, alghough I would say that the type defintions don't
> belong to the components at all. Type systems should be 1st-class
> objects that are maintained as separate entities and have an owner who
> decides what the official definition of the type is.
I agree. The best practice would be to keep the generated sources and
class files in
a Jar.
> It seems to me that if we have a jar containing JCAS cover classes and
> this jar is needed by multiple annotators, then we have similar issues
> that come up for any shared library. For example, if annotator A
> bundles one version of Xerces and annotator B bundles a nother
> version, there may be problems if you try to deploy both of these into
> the same application. Usually, the library will be backwards
> compatible so the only trick is to make sure the newer version is in
> use. If it's not backwards compatible then it gets uglier.
I think what we have is annotator A might have a version of types for it
(T/A) and
annotator B might have a version of types for it (T/B). The "assembly"
of A and B
has a process whereby T/A and T/B are "merged", and a new T/A&B is
created. This seems different form the Xerces example. It's not
ncesessarily a "newer"
versus "older" thing, for UIMA assemblies.
>> together, I imagine
>> I would regenerate the JCas classes for this aggregate, and package
>> these as
>> 2 separate things. This would allow future users of my part (the
>> aggregate) to
>> combine it with other parts, and re-run JCasGen on that new amalgam,
>> etc.
>>
>> This concept seems a fundamental principle of how components are
>> hooked up
>> together in UIMA. Our approach differs from WSDL, in that it strives to
>> avoid
>> translating data formats / representations, by instead having a
>> pre-figured-out
>> "merged" design for shared data at the start of a "run".
>>
> If I assemble several parts
>
> I think what is a fundamental principle of UIMA (or should be, anyway)
> is that components interoperate without manual intervention. So if I
> want to run a pipeline of any 5 UIMA components, I can just grab them
> off the shelf and run.
I agree this is a good goal. It seems achievable with some "automation"
introduced
into the assembly step.
> This seems particularly important for applications that host arbitrary
> UIMA analytics. End users want to grab the latest, greatest annotator
> and drop it in. This should work smoothly, or UIMA isn't meeting one
> of its most important goals.
I agree. Perhaps we should figure out what (if anything) is inhibiting
this,
and see if it can be be addressed. One concept might be to require JCas
source/class files to be packaged in a particular way, and to improve the
"merge" logic to cover more cases (and report on the cases where it fails
and a "manual" merge step might be needed). I think in most pragmatic
cases it will work fine "automatically".
-Marshall
Re: DocumentAnnotation and type-merging
Posted by Adam Lally <al...@alum.rpi.edu>.
On 12/18/06, Marshall Schor <ms...@schor.com> wrote:
> It seems to me that developers choose to put data into a CAS because
> they envision "sharing"
> that data with other (independently-developed) components. If they're
> planning
> to do that, then the "definitions" of the types they're sharing in some
> sense
> naturally belong to multiple components (the components that are
> "sharing" that
> type definition).
>
> Thus, it seems a little bit illogical to envision the standard packaging
> of types
> go with a particular "component" - a better practice might be thinking
> of types
> more as 1st class parts in themselves.
Hmmm.. good point, alghough I would say that the type defintions don't
belong to the components at all. Type systems should be 1st-class
objects that are maintained as separate entities and have an owner who
decides what the official definition of the type is.
It seems to me that if we have a jar containing JCAS cover classes and
this jar is needed by multiple annotators, then we have similar issues
that come up for any shared library. For example, if annotator A
bundles one version of Xerces and annotator B bundles a nother
version, there may be problems if you try to deploy both of these into
the same application. Usually, the library will be backwards
compatible so the only trick is to make sure the newer version is in
use. If it's not backwards compatible then it gets uglier.
> If I assemble several parts
> together, I imagine
> I would regenerate the JCas classes for this aggregate, and package
> these as
> 2 separate things. This would allow future users of my part (the
> aggregate) to
> combine it with other parts, and re-run JCasGen on that new amalgam, etc.
>
> This concept seems a fundamental principle of how components are hooked up
> together in UIMA. Our approach differs from WSDL, in that it strives to
> avoid
> translating data formats / representations, by instead having a
> pre-figured-out
> "merged" design for shared data at the start of a "run".
>
I think what is a fundamental principle of UIMA (or should be, anyway)
is that components interoperate without manual intervention. So if I
want to run a pipeline of any 5 UIMA components, I can just grab them
off the shelf and run.
This seems particularly important for applications that host arbitrary
UIMA analytics. End users want to grab the latest, greatest annotator
and drop it in. This should work smoothly, or UIMA isn't meeting one
of its most important goals.
-Adam
Re: DocumentAnnotation and type-merging
Posted by Marshall Schor <ms...@schor.com>.
I think I'm leaning toward keeping this capability in JCasGen. Here's
what I'm thinking:
The concerns Adam raises arise when the classes that JCasGen generates
are packaged in the same JAR file as the component they go with. But I
think
they are mitigated if we keep type systems packaged separately from
components. (I'm imagining there's a way to do this, for now).
It seems to me that developers choose to put data into a CAS because
they envision "sharing"
that data with other (independently-developed) components. If they're
planning
to do that, then the "definitions" of the types they're sharing in some
sense
naturally belong to multiple components (the components that are
"sharing" that
type definition).
Thus, it seems a little bit illogical to envision the standard packaging
of types
go with a particular "component" - a better practice might be thinking
of types
more as 1st class parts in themselves. If I assemble several parts
together, I imagine
I would regenerate the JCas classes for this aggregate, and package
these as
2 separate things. This would allow future users of my part (the
aggregate) to
combine it with other parts, and re-run JCasGen on that new amalgam, etc.
This concept seems a fundamental principle of how components are hooked up
together in UIMA. Our approach differs from WSDL, in that it strives to
avoid
translating data formats / representations, by instead having a
pre-figured-out
"merged" design for shared data at the start of a "run".
The JCas approach does have a deficiency in that if the user has
hand-customized
the generated code, in more than one component, the merging of the
customized
code has to be done by hand. (It does work if only one of the parts
sharing a type
has the hand-customized code, or if more than one of the parts have
this, that they
are identical).
Adam Lally wrote:
> After thinking about this some more here's my a proposed plan of action:
>
> 1) We explicilty document that "feature extension" (meaning the
> practice of defining a type in two different places, with different
> features, causing those feature sets to be merged) is incompatible
> with JCAS. The "owner" of a type (whoever defined it first) should
> get to choose the features and generate the one and only JCas class.
> If a user of this type wants to add their own features, UIMA supports
> this only through the "plain" CAS API, not the JCAS. JCasGen and CDE
> should give warnings when they encounter multiple non-identical
> definitions of the same type.
I wonder if this might be too constraining for part - combining.
> 2) The UIMA Framework "owns" the definition of
> uima.tcas.DocumentAnnotation and its corresponding JCAS class. So
> users MAY NOT have their own JCAS cover classes for
> DocumentAnnotation. JCasGen should stop generating DocumentAnnotation
> classes.
>
> 3) We will provide a convient way for users to define their own global
> metadata types and easily access them in the CAS.
+1 to this
> 4) Migration path: Users must delete any DocumentAnnotation JCas
> cover class that they have. If they have added custom features, then
> they either have to change their code to not use JCAS (for accessing
> these particular features), or they have to define their own global
> metadata type and change their code to use that rather than the
> DocumentAnnotation.
>
> I don't think there's a good way to make JCAS and feature extension
> coexist happily, so let's acknowledge that they don't. It's never
> ideal to break users' code but I think this is worth it. I'm guessing
> most users haven't used feature extension anyway, so the effect is
> minor.
The way described at the top might be a reasonably good way to make JCas
and feature extension co-exist happily: keep JCas generated
sources/classes in
separate Jars, "live-with" the limit of only one "hand-done customization".
>
> Thoughts?
>
> -Adam
>
>
Re: DocumentAnnotation and type-merging
Posted by Adam Lally <al...@alum.rpi.edu>.
After thinking about this some more here's my a proposed plan of action:
1) We explicilty document that "feature extension" (meaning the
practice of defining a type in two different places, with different
features, causing those feature sets to be merged) is incompatible
with JCAS. The "owner" of a type (whoever defined it first) should
get to choose the features and generate the one and only JCas class.
If a user of this type wants to add their own features, UIMA supports
this only through the "plain" CAS API, not the JCAS. JCasGen and CDE
should give warnings when they encounter multiple non-identical
definitions of the same type.
2) The UIMA Framework "owns" the definition of
uima.tcas.DocumentAnnotation and its corresponding JCAS class. So
users MAY NOT have their own JCAS cover classes for
DocumentAnnotation. JCasGen should stop generating DocumentAnnotation
classes.
3) We will provide a convient way for users to define their own global
metadata types and easily access them in the CAS.
4) Migration path: Users must delete any DocumentAnnotation JCas
cover class that they have. If they have added custom features, then
they either have to change their code to not use JCAS (for accessing
these particular features), or they have to define their own global
metadata type and change their code to use that rather than the
DocumentAnnotation.
I don't think there's a good way to make JCAS and feature extension
coexist happily, so let's acknowledge that they don't. It's never
ideal to break users' code but I think this is worth it. I'm guessing
most users haven't used feature extension anyway, so the effect is
minor.
Thoughts?
-Adam
Re: DocumentAnnotation and type-merging
Posted by Adam Lally <al...@alum.rpi.edu>.
On 12/15/06, Marshall Schor <ms...@schor.com> wrote:
> > Questions:
> > 1) Do we continue to support adding features to DocumentAnnotation?
> It breaks a lot of user code if we stop supporting this. We might
> consider doing this as part of the change-over to Apache, but if we did, I think we
> would need to implement the ability for users to more easily store and find data in
> the CAS that they were using the Document Annotation for, before.
I think we should provide that feature anyway. IMO we need a better
way to index and retrieve arbitrary FS without having to declare a
custom index in your descriptor.
I do share your concern that this breaks user code in a way for which
there's no quick fix. Maybe, we can hope that very few users have
actually added their own features to DocumentAnnotation, though.
> > 2) If so, should we delete the JCAS DocumentAnnotation class from the
> > framework code entirely, to avoid the problem that required the
> > uima_jcas_builtin_types.jar workaround?
> If it is deleted, it might break old code generated before JCasGen was
> modified to create this class. In that case, there would be no JCas cover class
> for it at all. That's unlikely to be a problem, unless the user was looking for the
> only default built-in feature (language). And there's an easy work-around - run JCasGen.
>
Asking the user to run JCasGen in this case seems OK to me. JCasGen
has generated DocumentAnnotation since UIMA v1.1, I believe, so only
code older than that would be affected.
> I don't recall other reasons for having it - does anyone else?
There are a couple of uses of it in the framework itself.
a) The JCas has a deprecated method: DocumentAnnotation
getDocumentAnnotation(). This was deprecated because it is not safe
to use if the annotator is loaded under a different classloader than
the framework, and the annotator has its own definition of
DocumentAnnotation. If we remove the DocumentAnnotation class from
the framework we'd have to remove this method, and users would have to
change their code to use the alternative method
getDocumentAnnoationFs(), which returns type TOP and requires the user
to cast to DocumentAnnotation.
b) The example FileSystemCollectionReader uses the DocumentAnnotation
class to set the language feature (to a value specified in a config
parameter). Either we remove this feature, change it to not use JCAS
(kind of an ugly workaround to have to expose in example code), or we
could move the DocumentAnnotation class from uima-core.jar to
uima-examples.jar (which is a slight improvement, but in practice
unlikely to help much since I think most users just include all the
uima jars in their classpath anyway).
However, there's still a big problem with having JCasGen generate the
class for DocumentAnnotation: it is unsafe with respect to
composability. If two annotators bundle differing versions of
DocumentAnnotation they cannot be used together without rerunning
JCasGen and then dealing with all the issues that arise because you
now have three versions of DocumentAnnotation hanging around (so you
either need to get your classpath ordering right or actually go into
the annotators' jars and delete the versions of DocumentAnnotation
that they bundled).
No solution seems perfect, but if there were ever a time to remove
support for adding features to DocumentAnnotation, I think now is that
time.
-Adam
Re: DocumentAnnotation and type-merging
Posted by Marshall Schor <ms...@schor.com>.
Adam Lally wrote:
> Yes, this topic again...
>
> While working on a utility to migrate code from IBM UIMA to Apache
> UIMA, I encountered the case where the user's project has a definition
> of com.ibm.uima.jcas.tcas.DocumentAnnotation. That's because JCasGen
> creates this, to account for cases where the user has defined their
> own features to add to DocumentAnnotation.
>
> As far as the migration goes, this would need to become
> org.apache.uima.jcas.tcas.DocumentAnnotation. But I can't do that
> with search-and-replace, I'd actually have to move the file to a
> different directory. Instead of doing that, this may become a manual
> migration step. (If the user did not add custom code, just deleting
> this file and rerunning JCasGen is sufficient.)
I would recommend that the migration tool move the file (to the new
'package').
> It also occurred to me that we lost uima_jcas_builtin_types.jar as
> part of our Maven restructuring. This jar contained the
> DocumentAnnotation class, which was not in uima_core.jar. This was
> done so that applications could exclude this jar from their classpath
> to allow a user-generated DocumentAnnotation class to be used instead
> of the one that ships with UIMA. But this is pretty ugly.
>
> Questions:
> 1) Do we continue to support adding features to DocumentAnnotation?
It breaks a lot of user code if we stop supporting this. We might
consider doing
this as part of the change-over to Apache, but if we did, I think we
would need
to implement the ability for users to more easily store and find data in
the CAS that
they were using the Document Annotation for, before.
> 2) If so, should we delete the JCAS DocumentAnnotation class from the
> framework code entirely, to avoid the problem that required the
> uima_jcas_builtin_types.jar workaround?
If it is deleted, it might break old code generated before JCasGen was
modified
to create this class. In that case, there would be no JCas cover class
for it at all.
That's unlikely to be a problem, unless the user was looking for the
only default
built-in feature (language). And there's an easy work-around - run JCasGen.
I don't recall other reasons for having it - does anyone else?
> 3) What should the migration story be for existing user code?
Depends if we keep the ability to expand the features of this class or
not, and
if we implement an easier way to get specific data items from the CAS.
-Marshall