You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Marshall Schor <ms...@schor.com> on 2014/11/21 17:03:13 UTC
Help test UIMA SDK 2.7.0
Hi,
If you can, please help test the new 2.7.0-SNAPSHOT, by building from trunk.
When you do test runs, please add the JVM param
-Duima.check_invalid_fs_updates
to activate the new check to see if code is potentially corrupting any UIMA index.
This activates a new kind of check that checks all modifications to features to
see if that feature is
being used as a key in a Sort or Set index, and the Feature Structure being
modified is currently in
one or more indices (Jira issue UIMA-4059). Doing such a modification can
cause the index to become corrupt.
(The correct way to do an update in this case is to
first remove the Feature Structure from the indices,
do the modification,
and then add it back to the indices.)
It is somewhat likely that old code may start failing, due to this test, and due
to the stricter enforcement of correct Sofa references for adding subtypes of
AnnotationBase to the right view.
Currently, the JUnit tests pass, but several of these needed fixing due to this
increased checking.
I plan to write a few more unit tests to test the new deserialization of delta
CASes updating existing FSs (to insure indexed item updates don't corrupt indices).
-M
Re: Help test UIMA SDK 2.7.0
Posted by Peter Klügl <pk...@uni-wuerzburg.de>.
Hi,
no problems observed for ruta.
Best,
Peter
Am 21.11.2014 17:03, schrieb Marshall Schor:
> Hi,
>
> If you can, please help test the new 2.7.0-SNAPSHOT, by building from trunk.
> When you do test runs, please add the JVM param
>
> -Duima.check_invalid_fs_updates
>
> to activate the new check to see if code is potentially corrupting any UIMA index.
> This activates a new kind of check that checks all modifications to features to
> see if that feature is
> being used as a key in a Sort or Set index, and the Feature Structure being
> modified is currently in
> one or more indices (Jira issue UIMA-4059). Doing such a modification can
> cause the index to become corrupt.
> (The correct way to do an update in this case is to
> first remove the Feature Structure from the indices,
> do the modification,
> and then add it back to the indices.)
>
> It is somewhat likely that old code may start failing, due to this test, and due
> to the stricter enforcement of correct Sofa references for adding subtypes of
> AnnotationBase to the right view.
>
> Currently, the JUnit tests pass, but several of these needed fixing due to this
> increased checking.
>
> I plan to write a few more unit tests to test the new deserialization of delta
> CASes updating existing FSs (to insure indexed item updates don't corrupt indices).
>
> -M
>
Re: Help test UIMA SDK 2.7.0
Posted by Marshall Schor <ms...@schor.com>.
On 11/22/2014 4:13 PM, Richard Eckart de Castilho wrote:
> Hi Marshall,
>
> I have a unit test that fails on uimaj-core 2.7.0-SNAPSHOT but not on uimaj-core 2.6.0.
> Apparently, some features structures no longer appear multiple times in the index
> if they are added multiple times.
Yes, this is changed with Jira https://issues.apache.org/jira/browse/UIMA-3399
>From the upcoming Release Notes:
The meaning of "bag" and "sorted" index has been made consistent with how these
are handled
when sending CASes to remote services. This means that adding the same
identical FS to the
indexes, multiple times, will not add duplicate index entries. And, removing a
FS from an index
is guaranteed that that particular FS will no longer be in any index. (Before,
if you had
added a particular FS to a sorted or bag index, multiple times, the remove
behavior would
remove just one of the instances). Deserialization of CASes sent to remote
services has never
added Feature Structures to the index multiple times, so this change makes that
behavior
consistent. For more details, see https://issues.apache.org/jira/browse/UIMA-3399.
Because some users may prefer the previous behavior that permitted duplicates of
identical Feature Structures in
the sorted and bag indexes. this change can be disabled, by running the JVM with
the defined property
"uima.allow_duplicate_add_to_indices".</p>
Thanks for testing!
-M
>
> I'm not sure if this was an intentional change or not, so I didn't open an issue yet.
> However, here is a minimalized and purified version of my test. With 2.7.0-SNAPSHOT,
> the second assert fails because count() returns 1.
>
> @Test
> public void tooManyAnnotationsTest()
> throws Exception
> {
> CAS cas = CasCreationUtils.createCas((TypeSystemDescription) null, null, null);
> AnnotationFS documentAnnotation = cas.getDocumentAnnotation();
> assertEquals(1, count(cas, documentAnnotation.getType()));
>
> // Add a second time to the indexes
> cas.addFsToIndexes(documentAnnotation);
> assertEquals(2, count(cas, documentAnnotation.getType()));
> }
>
> private int count(CAS aCas, Type aType) {
> FSIterator<FeatureStructure> i = aCas.getIndexRepository().getAllIndexedFS(aType);
> int n = 0;
> while (i.hasNext()) {
> n++;
> i.next();
> }
> return n;
> }
>
> Cheers,
>
> -- Richard
>
>
>
Re: Help test UIMA SDK 2.7.0
Posted by Richard Eckart de Castilho <re...@apache.org>.
Hi Marshall,
I have a unit test that fails on uimaj-core 2.7.0-SNAPSHOT but not on uimaj-core 2.6.0.
Apparently, some features structures no longer appear multiple times in the index
if they are added multiple times.
I'm not sure if this was an intentional change or not, so I didn't open an issue yet.
However, here is a minimalized and purified version of my test. With 2.7.0-SNAPSHOT,
the second assert fails because count() returns 1.
@Test
public void tooManyAnnotationsTest()
throws Exception
{
CAS cas = CasCreationUtils.createCas((TypeSystemDescription) null, null, null);
AnnotationFS documentAnnotation = cas.getDocumentAnnotation();
assertEquals(1, count(cas, documentAnnotation.getType()));
// Add a second time to the indexes
cas.addFsToIndexes(documentAnnotation);
assertEquals(2, count(cas, documentAnnotation.getType()));
}
private int count(CAS aCas, Type aType) {
FSIterator<FeatureStructure> i = aCas.getIndexRepository().getAllIndexedFS(aType);
int n = 0;
while (i.hasNext()) {
n++;
i.next();
}
return n;
}
Cheers,
-- Richard