You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@uima.apache.org by "Coarr, Matt" <mc...@mitre.org> on 2012/01/17 23:25:22 UTC
question about annotation not being loaded
Hello,
We normally use the FileSystemCollectionReader to load in plain text files. This collection reader populates an annotation in the example namespace called SourceDocumentInformation (that has a uri that includes the full path to the original file).
For some testing, we have a group of generated test xmi files that will be used as input. For this test pipeline we have a new CPE config that uses the XmiCollectionReader. In general this is working great! We get most of our saved annotations loaded into the CAS and everything works great!
The problem we're having is that we've added the examples:SourceDocumentInformation annotation to the input test documents (xmi docs) along with the required xml namespace declaration (xmlns:examples="http:///org/apache/uima/examples.ecore"). Here's a sample (I've replace the real filename and input size with dummy data):
<examples:SourceDocumentInformation xmi:id="10" sofa="1" begin="0" end="0" uri="file:/tmp/myfiles/0123.xmi" offsetInSource="0" documentSize="1234" lastSegment="false"/>
However, this annotation isn't getting loaded. We can't see the value in our analysis engine, and we're writing these files out at the end with the example xmi writer and the SourceDocumentInformation annotation doesn't show up in the output xmi files. So it appears that this annotation isn't being loaded or it's being filtered out.
No errors show up either.
Any ideas? Has anyone run into this before?
Thanks!
Matt
Re: question about annotation not being loaded
Posted by Burn Lewis <bu...@gmail.com>.
I believe the default for that config parameter is true, and the inverse of
it is passed as the lenient parameter to the XMI deserializer. So if you
set your config parameter to true you will get an exception if there is a
type in the CAS that is not in your declared typesystem ... which would
explain why an annotation appears to be deleted.
~Burn
Re: question about annotation not being loaded
Posted by Joel Dubbels <du...@us.ibm.com>.
I was referring to the "aLenient" parameter on the XmiCasDeserializer
which defaults to false..which implies that an exception WOULD be thrown
if there was a type system mismatch. This default I would equate to the
config parm FailOnUnknownType being set to TRUE. However, you said
FailOnUnknownType was set to false which implies to me that it would
silently ignore mismatches. Having said that, I'm not sure how these two
values affect each other, if at all.
Here's the description of the XmiCasDeserializer method I'm talking about.
static void deserialize(InputStream aStream, CAS aCAS, boolean aLenient)
Deserializes a CAS from XMI.
aStream - input stream from which to read the XCMI document
aCAS - CAS into which to deserialize. This CAS must be set up with a type
system that is compatible with that in the XMI
aLenient - if true, unknown Types will be ignored. If false, unknown Types
will cause an exception. The default is false.
Joel Dubbels
Senior Software Engineer | Software Group
email: dubbels@us.ibm.com | Tel: 507-253-1165
From:
"Coarr, Matt" <mc...@mitre.org>
To:
"user@uima.apache.org" <us...@uima.apache.org>
Cc:
"Wellner, Benjamin R." <we...@mitre.org>
Date:
01/18/2012 07:44 AM
Subject:
Re: question about annotation not being loaded
Thanks for the idea. Are you talking about the FailOnUnknownType config
param? It is set to false (as you said was default).
Matt
From: Joel Dubbels <du...@us.ibm.com>>
Reply-To: <us...@uima.apache.org>>
Date: Tue, 17 Jan 2012 17:11:06 -0600
To: <us...@uima.apache.org>>
Cc: "user@uima.apache.org<ma...@uima.apache.org>"
<us...@uima.apache.org>>, "Wellner, Benjamin
R." <we...@mitre.org>>
Subject: Re: question about annotation not being loaded
I don't know if this is it, but I'll offer it up just in case......could
it be a type system mismatch? I've had problems in the past where I've
deserialized CASes and the deserializer was set to "lenient" (which would
only pull in the types the deserializer recognizes and ignores all
others). The default for the XmiCasDeserializer is lenient=false, so, by
default it should throw an exception if something mismatches.
________________________________
Joel Dubbels
Senior Software Engineer | Software Group
email: dubbels@us.ibm.com<ma...@region.ibm.com> | Tel: 507-253-1165
[cid:_1_0C3FE8300C3FE45C007F5C7C86257988]
[cid:_1_0C3FD6DC0C3FD308007F5C7C86257988]
________________________________
From: "Coarr, Matt" <mc...@mitre.org>>
To: "user@uima.apache.org<ma...@uima.apache.org>"
<us...@uima.apache.org>>
Cc: "Wellner, Benjamin R." <wellner@mitre.org<mailto:wellner@mitre.org
>>
Date: 01/17/2012 04:27 PM
Subject: question about annotation not being loaded
________________________________
Hello,
We normally use the FileSystemCollectionReader to load in plain text
files. This collection reader populates an annotation in the example
namespace called SourceDocumentInformation (that has a uri that includes
the full path to the original file).
For some testing, we have a group of generated test xmi files that will be
used as input. For this test pipeline we have a new CPE config that uses
the XmiCollectionReader. In general this is working great! We get most
of our saved annotations loaded into the CAS and everything works great!
The problem we're having is that we've added the
examples:SourceDocumentInformation annotation to the input test documents
(xmi docs) along with the required xml namespace declaration
(xmlns:examples="http:///org/apache/uima/examples.ecore"). Here's a
sample (I've replace the real filename and input size with dummy data):
<examples:SourceDocumentInformation xmi:id="10" sofa="1" begin="0" end="0"
uri="file:/tmp/myfiles/0123.xmi" offsetInSource="0" documentSize="1234"
lastSegment="false"/>
However, this annotation isn't getting loaded. We can't see the valuein
our analysis engine, and we're writing these files out at the end with the
example xmi writer and the SourceDocumentInformation annotation
doesn'tshow up in the output xmi files. So it appears that this
annotation isn't being loaded or it's being filtered out.
No errors show up either.
Any ideas? Has anyone run into this before?
Thanks!
Matt
Re: question about annotation not being loaded
Posted by "Coarr, Matt" <mc...@mitre.org>.
Thanks for the idea. Are you talking about the FailOnUnknownType config param? It is set to false (as you said was default).
Matt
From: Joel Dubbels <du...@us.ibm.com>>
Reply-To: <us...@uima.apache.org>>
Date: Tue, 17 Jan 2012 17:11:06 -0600
To: <us...@uima.apache.org>>
Cc: "user@uima.apache.org<ma...@uima.apache.org>" <us...@uima.apache.org>>, "Wellner, Benjamin R." <we...@mitre.org>>
Subject: Re: question about annotation not being loaded
I don't know if this is it, but I'll offer it up just in case......could it be a type system mismatch? I've had problems in the past where I've deserialized CASes and the deserializer was set to "lenient" (which would only pull in the types the deserializer recognizes and ignores all others). The default for the XmiCasDeserializer is lenient=false, so, by default it should throw an exception if something mismatches.
________________________________
Joel Dubbels
Senior Software Engineer | Software Group
email: dubbels@us.ibm.com<ma...@region.ibm.com> | Tel: 507-253-1165
[cid:_1_0C3FE8300C3FE45C007F5C7C86257988]
[cid:_1_0C3FD6DC0C3FD308007F5C7C86257988]
________________________________
From: "Coarr, Matt" <mc...@mitre.org>>
To: "user@uima.apache.org<ma...@uima.apache.org>" <us...@uima.apache.org>>
Cc: "Wellner, Benjamin R." <we...@mitre.org>>
Date: 01/17/2012 04:27 PM
Subject: question about annotation not being loaded
________________________________
Hello,
We normally use the FileSystemCollectionReader to load in plain text files. This collection reader populates an annotation in the example namespace called SourceDocumentInformation (that has a uri that includes the full path to the original file).
For some testing, we have a group of generated test xmi files that will be used as input. For this test pipeline we have a new CPE config that uses the XmiCollectionReader. In general this is working great! We get most of our saved annotations loaded into the CAS and everything works great!
The problem we're having is that we've added the examples:SourceDocumentInformation annotation to the input test documents (xmi docs) along with the required xml namespace declaration (xmlns:examples="http:///org/apache/uima/examples.ecore"). Here's a sample (I've replace the real filename and input size with dummy data):
<examples:SourceDocumentInformation xmi:id="10" sofa="1" begin="0" end="0" uri="file:/tmp/myfiles/0123.xmi" offsetInSource="0" documentSize="1234" lastSegment="false"/>
However, this annotation isn't getting loaded. We can't see the valuein our analysis engine, and we're writing these files out at the end with the example xmi writer and the SourceDocumentInformation annotation doesn'tshow up in the output xmi files. So it appears that this annotation isn't being loaded or it's being filtered out.
No errors show up either.
Any ideas? Has anyone run into this before?
Thanks!
Matt
Re: question about annotation not being loaded
Posted by Joel Dubbels <du...@us.ibm.com>.
I don't know if this is it, but I'll offer it up just in case......could
it be a type system mismatch? I've had problems in the past where I've
deserialized CASes and the deserializer was set to "lenient" (which would
only pull in the types the deserializer recognizes and ignores all
others). The default for the XmiCasDeserializer is lenient=false, so, by
default it should throw an exception if something mismatches.
Joel Dubbels
Senior Software Engineer | Software Group
email: dubbels@us.ibm.com | Tel: 507-253-1165
From:
"Coarr, Matt" <mc...@mitre.org>
To:
"user@uima.apache.org" <us...@uima.apache.org>
Cc:
"Wellner, Benjamin R." <we...@mitre.org>
Date:
01/17/2012 04:27 PM
Subject:
question about annotation not being loaded
Hello,
We normally use the FileSystemCollectionReader to load in plain text
files. This collection reader populates an annotation in the example
namespace called SourceDocumentInformation (that has a uri that includes
the full path to the original file).
For some testing, we have a group of generated test xmi files that will be
used as input. For this test pipeline we have a new CPE config that uses
the XmiCollectionReader. In general this is working great! We get most
of our saved annotations loaded into the CAS and everything works great!
The problem we're having is that we've added the
examples:SourceDocumentInformation annotation to the input test documents
(xmi docs) along with the required xml namespace declaration
(xmlns:examples="http:///org/apache/uima/examples.ecore"). Here's a
sample (I've replace the real filename and input size with dummy data):
<examples:SourceDocumentInformation xmi:id="10" sofa="1" begin="0" end="0"
uri="file:/tmp/myfiles/0123.xmi" offsetInSource="0" documentSize="1234"
lastSegment="false"/>
However, this annotation isn't getting loaded. We can't see the value in
our analysis engine, and we're writing these files out at the end with the
example xmi writer and the SourceDocumentInformation annotation doesn't
show up in the output xmi files. So it appears that this annotation isn't
being loaded or it's being filtered out.
No errors show up either.
Any ideas? Has anyone run into this before?
Thanks!
Matt