You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Jens Grivolla <j+...@grivolla.net> on 2014/08/13 22:46:45 UTC
Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs and
UIMA Descriptors
Hi, I am very interested in this.
In particular, we have so far stored CASs as compressed XMI in MySQL but
are now moving to MongoDB. Having a lossless generic JSON serialization
(equivalent to XMI) would be a much better fit as MongoDB could then store
it pretty much natively and it would even enable some simple queries on the
annotations directly in MongoDB.
I'm not sure if there are any special considerations to make the JSON
serialization fully compatible with MongoDBs BSON format.
-- Jens
On Wed, Jul 30, 2014 at 10:44 PM, Marshall Schor (JIRA) <dev@uima.apache.org
> wrote:
> Marshall Schor created UIMA-3969:
> ------------------------------------
>
> Summary: Add JSON Serialization for CASs and UIMA Descriptors
> Key: UIMA-3969
> URL: https://issues.apache.org/jira/browse/UIMA-3969
> Project: UIMA
> Issue Type: New Feature
> Components: Core Java Framework
> Affects Versions: 2.6.0SDK
> Reporter: Marshall Schor
> Assignee: Marshall Schor
> Priority: Minor
> Fix For: 2.6.1SDK
>
>
> Recent trends toward moving things into the cloud motivated me to consider
> what a JSON serialization of the CAS and descriptor metadata (more
> particularly, type systems) might look like.
>
> I've put up a Wiki page with some of the thoughts so far in this
> exploration, here:
> https://cwiki.apache.org/confluence/display/UIMA/JSON+serialization+for+UIMA
>
> I'm also fooling around with a proof-of-concept implementation, based on
> our current XMI serialization for the CAS, as well as our
> MetaDataObject_impl serialization for UIMA descriptors, in order to work
> out the details. There are additional nits (like how to configure things)
> not yet worked out.
>
> Comments and discussion appreciated; I've put this up as a Jira to record
> them together - but feel free to use email also for any comments you feel
> might be better being more ephemeral.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.2#6252)
>
Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs and
UIMA Descriptors
Posted by John Green <jo...@gmail.com>.
Very good
On Wed, Aug 27, 2014 at 2:39 AM, AndyMC@apache.org (Andy McMurry) <
mcmurry.andy@gmail.com> wrote:
> Interesting thread in UIMA core about JSON Serialization CAS and
> Descriptors.
>
>
> Begin forwarded message:
>
> > From: Marshall Schor <ms...@schor.com>
> > Subject: Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for
> CASs and UIMA Descriptors
> > Date: August 25, 2014 at 8:33:54 PM PDT
> > To: dev@uima.apache.org
> > Reply-To: dev@uima.apache.org
> >
> >
> > On 8/25/2014 6:54 PM, Jens Grivolla wrote:
> >> Is the JSON serialization documented somewhere?
> > Yes, there's a chapter in the reference book. You can build that
> > (uima-docbook-references), until it's released.
> >
> > There are also lots of Javadocs in the main implementing class:
> > XmiCasSerializer. (It's in this class because it shares a lot of the
> machinery
> > with Xmi serialization).
> >
> >>
> >> I saw that there appear to be quite a few alternative serializations. It
> >> seems to include something like a typesystem definition, but only with a
> >> list of feature names, not their types, if I understood the format
> >> correctly (@featureRefs has a list of the features that are not of
> >> primitive types, it seems).
> > The @featureRefs is only those features which are "references" to other
> feature
> > structures.
> >
> > You're correct, in noticing that the feature "range" types are not
> present.
> > This is because the serialization is to JSON, which supports a native
> > representation of things that are collections (JSON arrays) which could
> be uima
> > Arrays or Lists, and ranges that are boolean are representable by JSON
> true and
> > false values. There is no distinction that a number is a
> byte/short/int/long,
> > because those are all represented as a JSON "number". And so forth...
> >
> > The Json serialization for a CAS can optionally include parts of the type
> > system: It can include what the supertypes are for serialized types (to
> enable
> > iterating over a type and all of its subtypes, like Cas iterators
> normally do);
> > it can also identify which slots which appear to have number values are
> actually
> > to be interpreted as references to other feature structures. Otherwise,
> the
> > serialized form might have a slot "foo" : 111 which is a number value,
> and a
> > slot "bar" : 112 which is a reference to another feature structure whose
> ID is
> > 112. This extra information (in @featureRefs) permits the user of the
> JSON
> > serialized form a way to distinguish these two case.
> >
> >>
> >> It would be very useful if the serialization allowed one to easily pull
> out
> >> a partial CAS with just a subset of the views (by only including some
> >> subtrees of the JSON structure), and merge views into it.
> > Another optional part of the serialization is a list of views, together
> with an
> > array of numbers each one of which represents a serialized Feature
> Structure
> > that is indexed in that view.
> >> This might be
> >> complicated, as I understand that the views define annotation indices,
> but
> >> the same annotation can be indexed in several views, right?
> >
> > Feature Structures can be classified into "Annotations" and other types
> (not a
> > subtype of Annotation).
> >
> > Annotations are special - they have an implied reference to a particular
> subject
> > of analysis. So they are restricted to being indexed in the view that is
> > associated with that subject-of-analysis.
> >
> > Other types (not subtypes of Annotation (or more precisely,
> AnnotationBase)) do
> > not have this restriction, and can be indexed in multiple views.
> >
> > See
> >
> http://uima.apache.org/d/uimaj-2.6.0/tutorials_and_users_guides.html#ugr.tug.aas.annotations_associated_sofa
> .
> >
> > Let me know where the documentation might be improved :-)
> >
> > -Marshall
> >>
> >> -- Jens
> >>
> >>
> >>
> >
>
>
Fwd: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs and UIMA Descriptors
Posted by "AndyMC@apache.org (Andy McMurry)" <mc...@gmail.com>.
Interesting thread in UIMA core about JSON Serialization CAS and Descriptors.
Begin forwarded message:
> From: Marshall Schor <ms...@schor.com>
> Subject: Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs and UIMA Descriptors
> Date: August 25, 2014 at 8:33:54 PM PDT
> To: dev@uima.apache.org
> Reply-To: dev@uima.apache.org
>
>
> On 8/25/2014 6:54 PM, Jens Grivolla wrote:
>> Is the JSON serialization documented somewhere?
> Yes, there's a chapter in the reference book. You can build that
> (uima-docbook-references), until it's released.
>
> There are also lots of Javadocs in the main implementing class:
> XmiCasSerializer. (It's in this class because it shares a lot of the machinery
> with Xmi serialization).
>
>>
>> I saw that there appear to be quite a few alternative serializations. It
>> seems to include something like a typesystem definition, but only with a
>> list of feature names, not their types, if I understood the format
>> correctly (@featureRefs has a list of the features that are not of
>> primitive types, it seems).
> The @featureRefs is only those features which are "references" to other feature
> structures.
>
> You're correct, in noticing that the feature "range" types are not present.
> This is because the serialization is to JSON, which supports a native
> representation of things that are collections (JSON arrays) which could be uima
> Arrays or Lists, and ranges that are boolean are representable by JSON true and
> false values. There is no distinction that a number is a byte/short/int/long,
> because those are all represented as a JSON "number". And so forth...
>
> The Json serialization for a CAS can optionally include parts of the type
> system: It can include what the supertypes are for serialized types (to enable
> iterating over a type and all of its subtypes, like Cas iterators normally do);
> it can also identify which slots which appear to have number values are actually
> to be interpreted as references to other feature structures. Otherwise, the
> serialized form might have a slot "foo" : 111 which is a number value, and a
> slot "bar" : 112 which is a reference to another feature structure whose ID is
> 112. This extra information (in @featureRefs) permits the user of the JSON
> serialized form a way to distinguish these two case.
>
>>
>> It would be very useful if the serialization allowed one to easily pull out
>> a partial CAS with just a subset of the views (by only including some
>> subtrees of the JSON structure), and merge views into it.
> Another optional part of the serialization is a list of views, together with an
> array of numbers each one of which represents a serialized Feature Structure
> that is indexed in that view.
>> This might be
>> complicated, as I understand that the views define annotation indices, but
>> the same annotation can be indexed in several views, right?
>
> Feature Structures can be classified into "Annotations" and other types (not a
> subtype of Annotation).
>
> Annotations are special - they have an implied reference to a particular subject
> of analysis. So they are restricted to being indexed in the view that is
> associated with that subject-of-analysis.
>
> Other types (not subtypes of Annotation (or more precisely, AnnotationBase)) do
> not have this restriction, and can be indexed in multiple views.
>
> See
> http://uima.apache.org/d/uimaj-2.6.0/tutorials_and_users_guides.html#ugr.tug.aas.annotations_associated_sofa.
>
> Let me know where the documentation might be improved :-)
>
> -Marshall
>>
>> -- Jens
>>
>>
>>
>
Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs
and UIMA Descriptors
Posted by Marshall Schor <ms...@schor.com>.
On 8/26/2014 11:41 AM, Jens Grivolla wrote:
> On Tue, Aug 26, 2014 at 4:33 AM, Marshall Schor <ms...@schor.com> wrote:
>
>> On 8/25/2014 6:54 PM, Jens Grivolla wrote:
>>> Is the JSON serialization documented somewhere?
>> Yes, there's a chapter in the reference book. You can build that
>> (uima-docbook-references), until it's released.
>>
>> There are also lots of Javadocs in the main implementing class:
>> XmiCasSerializer. (It's in this class because it shares a lot of the
>> machinery
>> with Xmi serialization).
>
> Ok, I'll take another deeper look at this.
>
>
>>> It would be very useful if the serialization allowed one to easily pull
>> out
>>> a partial CAS with just a subset of the views (by only including some
>>> subtrees of the JSON structure), and merge views into it.
>> Another optional part of the serialization is a list of views, together
>> with an
>> array of numbers each one of which represents a serialized Feature
>> Structure
>> that is indexed in that view.
>>
> Yes, I saw that. But the FSs themselves are all together.
>
> Feature Structures can be classified into "Annotations" and other types
>> (not a
>> subtype of Annotation).
>>
>> Annotations are special - they have an implied reference to a particular
>> subject
>> of analysis. So they are restricted to being indexed in the view that is
>> associated with that subject-of-analysis.
>>
>> Other types (not subtypes of Annotation (or more precisely,
>> AnnotationBase)) do
>> not have this restriction, and can be indexed in multiple views.
>>
> Thanks for the refresher. That means that the view-specific parts (Sofa and
> Annotations) could be grouped with the view they belong to and there could
> just be an additional section for FSs that are not bound to a view.
>
> That would make it much easier to just pull out a subset of views (e.g.
> when storing the CAS in MongoDB), as well as write individual views back,
> without retrieving all the FSs one doesn't need. The main problem I see is
> with the FS ids, which probably need to be unique across the entire CAS.
> Thus creating new annotations in a partial CAS that doesn't know about the
> other views could create conflicts.
>
> My usecase is e.g. passing a document through video analysis (a batch
> process not directly integrated as a UIMA AE), which would generate a lot
> of output that gets integrated in the CAS. I then need to pass just the
> view that includes the OCR output to a text analysis pipeline, and would
> like to avoid retrieving all the other views from the database and passing
> a huge CAS through my pipeline when I really only need a small part of the
> information.
Hmmm, It seems your use case is a requirement on how things are stored in your
Database.
All FSs include a "sofaId" field, identifying their sofa. So it seems to me that
if all the FSs are stored in the Database, you need a query which retrieves just
those that match
a particular sofaId, and also perhaps match just a particular set of types. Is
this correct?
If, on the other hand, you have a use case where you want UIMA to serialize out
a CAS in JSON
format while filtering so only some types associated with some Views are output,
that would
take some work :-) . There is already a "filter" (also available in Xmi
Serialization) that lets you
filter the things that are serialized by both Type and Features within a Type.
But, currently,
there is no filter that specifies a View. If this was added, there are some
edge cases to figure out.
Assume you ask for only View 1.
If a FS was indexed in View 1, but had references to FSs that were indexed in
another view, those
could be included, or not. (The basic algorithm serializes FSs that are
indexed, and other FSs that are reachable from those.)
And, if there are FSs that are indexed in View 2, that have a reference to FSs
that are associated (let's say they're subtypes of Annotation", which causes an
association) with View 1, should those be serialized, even though we're not
serializing the FSs that perhaps was the only way it could be found? [ I'm
thinking that the referenced FS might not be indexed in View 1...].
And, all this has to work also with "Delta CAS" serialization, where another
filter is used to only send "new" FSs created after a mark was set.
Anyway, I appreciate any input / thoughts on how view filtering ought to work.
-Marshall
>
> -- Jens
>
Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs and
UIMA Descriptors
Posted by Jens Grivolla <j+...@grivolla.net>.
On Tue, Aug 26, 2014 at 4:33 AM, Marshall Schor <ms...@schor.com> wrote:
>
> On 8/25/2014 6:54 PM, Jens Grivolla wrote:
> > Is the JSON serialization documented somewhere?
> Yes, there's a chapter in the reference book. You can build that
> (uima-docbook-references), until it's released.
>
> There are also lots of Javadocs in the main implementing class:
> XmiCasSerializer. (It's in this class because it shares a lot of the
> machinery
> with Xmi serialization).
Ok, I'll take another deeper look at this.
> > It would be very useful if the serialization allowed one to easily pull
> out
> > a partial CAS with just a subset of the views (by only including some
> > subtrees of the JSON structure), and merge views into it.
> Another optional part of the serialization is a list of views, together
> with an
> array of numbers each one of which represents a serialized Feature
> Structure
> that is indexed in that view.
>
Yes, I saw that. But the FSs themselves are all together.
Feature Structures can be classified into "Annotations" and other types
> (not a
> subtype of Annotation).
>
> Annotations are special - they have an implied reference to a particular
> subject
> of analysis. So they are restricted to being indexed in the view that is
> associated with that subject-of-analysis.
>
> Other types (not subtypes of Annotation (or more precisely,
> AnnotationBase)) do
> not have this restriction, and can be indexed in multiple views.
>
Thanks for the refresher. That means that the view-specific parts (Sofa and
Annotations) could be grouped with the view they belong to and there could
just be an additional section for FSs that are not bound to a view.
That would make it much easier to just pull out a subset of views (e.g.
when storing the CAS in MongoDB), as well as write individual views back,
without retrieving all the FSs one doesn't need. The main problem I see is
with the FS ids, which probably need to be unique across the entire CAS.
Thus creating new annotations in a partial CAS that doesn't know about the
other views could create conflicts.
My usecase is e.g. passing a document through video analysis (a batch
process not directly integrated as a UIMA AE), which would generate a lot
of output that gets integrated in the CAS. I then need to pass just the
view that includes the OCR output to a text analysis pipeline, and would
like to avoid retrieving all the other views from the database and passing
a huge CAS through my pipeline when I really only need a small part of the
information.
-- Jens
Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs
and UIMA Descriptors
Posted by Marshall Schor <ms...@schor.com>.
On 8/25/2014 6:54 PM, Jens Grivolla wrote:
> Is the JSON serialization documented somewhere?
Yes, there's a chapter in the reference book. You can build that
(uima-docbook-references), until it's released.
There are also lots of Javadocs in the main implementing class:
XmiCasSerializer. (It's in this class because it shares a lot of the machinery
with Xmi serialization).
>
> I saw that there appear to be quite a few alternative serializations. It
> seems to include something like a typesystem definition, but only with a
> list of feature names, not their types, if I understood the format
> correctly (@featureRefs has a list of the features that are not of
> primitive types, it seems).
The @featureRefs is only those features which are "references" to other feature
structures.
You're correct, in noticing that the feature "range" types are not present.
This is because the serialization is to JSON, which supports a native
representation of things that are collections (JSON arrays) which could be uima
Arrays or Lists, and ranges that are boolean are representable by JSON true and
false values. There is no distinction that a number is a byte/short/int/long,
because those are all represented as a JSON "number". And so forth...
The Json serialization for a CAS can optionally include parts of the type
system: It can include what the supertypes are for serialized types (to enable
iterating over a type and all of its subtypes, like Cas iterators normally do);
it can also identify which slots which appear to have number values are actually
to be interpreted as references to other feature structures. Otherwise, the
serialized form might have a slot "foo" : 111 which is a number value, and a
slot "bar" : 112 which is a reference to another feature structure whose ID is
112. This extra information (in @featureRefs) permits the user of the JSON
serialized form a way to distinguish these two case.
>
> It would be very useful if the serialization allowed one to easily pull out
> a partial CAS with just a subset of the views (by only including some
> subtrees of the JSON structure), and merge views into it.
Another optional part of the serialization is a list of views, together with an
array of numbers each one of which represents a serialized Feature Structure
that is indexed in that view.
> This might be
> complicated, as I understand that the views define annotation indices, but
> the same annotation can be indexed in several views, right?
Feature Structures can be classified into "Annotations" and other types (not a
subtype of Annotation).
Annotations are special - they have an implied reference to a particular subject
of analysis. So they are restricted to being indexed in the view that is
associated with that subject-of-analysis.
Other types (not subtypes of Annotation (or more precisely, AnnotationBase)) do
not have this restriction, and can be indexed in multiple views.
See
http://uima.apache.org/d/uimaj-2.6.0/tutorials_and_users_guides.html#ugr.tug.aas.annotations_associated_sofa.
Let me know where the documentation might be improved :-)
-Marshall
>
> -- Jens
>
>
>
Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs and
UIMA Descriptors
Posted by Jens Grivolla <j+...@grivolla.net>.
Is the JSON serialization documented somewhere?
I saw that there appear to be quite a few alternative serializations. It
seems to include something like a typesystem definition, but only with a
list of feature names, not their types, if I understood the format
correctly (@featureRefs has a list of the features that are not of
primitive types, it seems).
It would be very useful if the serialization allowed one to easily pull out
a partial CAS with just a subset of the views (by only including some
subtrees of the JSON structure), and merge views into it. This might be
complicated, as I understand that the views define annotation indices, but
the same annotation can be indexed in several views, right?
-- Jens
On Fri, Aug 15, 2014 at 5:00 PM, Marshall Schor <ms...@schor.com> wrote:
> In the current design, both UIMA arrays and lists are serialized using JSON
> arrays, if the feature value is marked as MultipleReferencesAllowed -
> false .
> So the "list" versus "array" nature in UIMA would be lost in the
> serialization,
> unless the type system information is available.
>
> I suspect that in most cases, this won't be important. But if it is, it
> can be
> avoided by specifying MultipleReferencesAllowed - true in the UIMA type
> system
> description.
>
> -Marshall
>
> On 8/15/2014 8:55 AM, Marshall Schor wrote:
> > Hi,
> >
> > The trunk is beginning to have a mostly working version of this
> serialization.
> > I'm checking out the edge cases with test cases (and finding the usual
> bugs that
> > are being fixed).
> >
> > I'm not very familiar with BSON, so if someone knows how a conversion to
> that
> > format from JSON might inform the JSON design, please post :-)
> >
> > -Marshall
> >
> >
> > On 8/13/2014 4:46 PM, Jens Grivolla wrote:
> >> Hi, I am very interested in this.
> >>
> >> In particular, we have so far stored CASs as compressed XMI in MySQL but
> >> are now moving to MongoDB. Having a lossless generic JSON serialization
> >> (equivalent to XMI) would be a much better fit as MongoDB could then
> store
> >> it pretty much natively and it would even enable some simple queries on
> the
> >> annotations directly in MongoDB.
> >>
> >> I'm not sure if there are any special considerations to make the JSON
> >> serialization fully compatible with MongoDBs BSON format.
> >>
> >> -- Jens
> >>
> >>
> >> On Wed, Jul 30, 2014 at 10:44 PM, Marshall Schor (JIRA) <
> dev@uima.apache.org
> >>> wrote:
> >>> Marshall Schor created UIMA-3969:
> >>> ------------------------------------
> >>>
> >>> Summary: Add JSON Serialization for CASs and UIMA
> Descriptors
> >>> Key: UIMA-3969
> >>> URL: https://issues.apache.org/jira/browse/UIMA-3969
> >>> Project: UIMA
> >>> Issue Type: New Feature
> >>> Components: Core Java Framework
> >>> Affects Versions: 2.6.0SDK
> >>> Reporter: Marshall Schor
> >>> Assignee: Marshall Schor
> >>> Priority: Minor
> >>> Fix For: 2.6.1SDK
> >>>
> >>>
> >>> Recent trends toward moving things into the cloud motivated me to
> consider
> >>> what a JSON serialization of the CAS and descriptor metadata (more
> >>> particularly, type systems) might look like.
> >>>
> >>> I've put up a Wiki page with some of the thoughts so far in this
> >>> exploration, here:
> >>>
> https://cwiki.apache.org/confluence/display/UIMA/JSON+serialization+for+UIMA
> >>>
> >>> I'm also fooling around with a proof-of-concept implementation, based
> on
> >>> our current XMI serialization for the CAS, as well as our
> >>> MetaDataObject_impl serialization for UIMA descriptors, in order to
> work
> >>> out the details. There are additional nits (like how to configure
> things)
> >>> not yet worked out.
> >>>
> >>> Comments and discussion appreciated; I've put this up as a Jira to
> record
> >>> them together - but feel free to use email also for any comments you
> feel
> >>> might be better being more ephemeral.
> >>>
> >>>
> >>>
> >>> --
> >>> This message was sent by Atlassian JIRA
> >>> (v6.2#6252)
> >>>
> >
> >
>
>
Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs
and UIMA Descriptors
Posted by Marshall Schor <ms...@schor.com>.
In the current design, both UIMA arrays and lists are serialized using JSON
arrays, if the feature value is marked as MultipleReferencesAllowed - false .
So the "list" versus "array" nature in UIMA would be lost in the serialization,
unless the type system information is available.
I suspect that in most cases, this won't be important. But if it is, it can be
avoided by specifying MultipleReferencesAllowed - true in the UIMA type system
description.
-Marshall
On 8/15/2014 8:55 AM, Marshall Schor wrote:
> Hi,
>
> The trunk is beginning to have a mostly working version of this serialization.
> I'm checking out the edge cases with test cases (and finding the usual bugs that
> are being fixed).
>
> I'm not very familiar with BSON, so if someone knows how a conversion to that
> format from JSON might inform the JSON design, please post :-)
>
> -Marshall
>
>
> On 8/13/2014 4:46 PM, Jens Grivolla wrote:
>> Hi, I am very interested in this.
>>
>> In particular, we have so far stored CASs as compressed XMI in MySQL but
>> are now moving to MongoDB. Having a lossless generic JSON serialization
>> (equivalent to XMI) would be a much better fit as MongoDB could then store
>> it pretty much natively and it would even enable some simple queries on the
>> annotations directly in MongoDB.
>>
>> I'm not sure if there are any special considerations to make the JSON
>> serialization fully compatible with MongoDBs BSON format.
>>
>> -- Jens
>>
>>
>> On Wed, Jul 30, 2014 at 10:44 PM, Marshall Schor (JIRA) <dev@uima.apache.org
>>> wrote:
>>> Marshall Schor created UIMA-3969:
>>> ------------------------------------
>>>
>>> Summary: Add JSON Serialization for CASs and UIMA Descriptors
>>> Key: UIMA-3969
>>> URL: https://issues.apache.org/jira/browse/UIMA-3969
>>> Project: UIMA
>>> Issue Type: New Feature
>>> Components: Core Java Framework
>>> Affects Versions: 2.6.0SDK
>>> Reporter: Marshall Schor
>>> Assignee: Marshall Schor
>>> Priority: Minor
>>> Fix For: 2.6.1SDK
>>>
>>>
>>> Recent trends toward moving things into the cloud motivated me to consider
>>> what a JSON serialization of the CAS and descriptor metadata (more
>>> particularly, type systems) might look like.
>>>
>>> I've put up a Wiki page with some of the thoughts so far in this
>>> exploration, here:
>>> https://cwiki.apache.org/confluence/display/UIMA/JSON+serialization+for+UIMA
>>>
>>> I'm also fooling around with a proof-of-concept implementation, based on
>>> our current XMI serialization for the CAS, as well as our
>>> MetaDataObject_impl serialization for UIMA descriptors, in order to work
>>> out the details. There are additional nits (like how to configure things)
>>> not yet worked out.
>>>
>>> Comments and discussion appreciated; I've put this up as a Jira to record
>>> them together - but feel free to use email also for any comments you feel
>>> might be better being more ephemeral.
>>>
>>>
>>>
>>> --
>>> This message was sent by Atlassian JIRA
>>> (v6.2#6252)
>>>
>
>
Re: [jira] [Created] (UIMA-3969) Add JSON Serialization for CASs
and UIMA Descriptors
Posted by Marshall Schor <ms...@schor.com>.
Hi,
The trunk is beginning to have a mostly working version of this serialization.
I'm checking out the edge cases with test cases (and finding the usual bugs that
are being fixed).
I'm not very familiar with BSON, so if someone knows how a conversion to that
format from JSON might inform the JSON design, please post :-)
-Marshall
On 8/13/2014 4:46 PM, Jens Grivolla wrote:
> Hi, I am very interested in this.
>
> In particular, we have so far stored CASs as compressed XMI in MySQL but
> are now moving to MongoDB. Having a lossless generic JSON serialization
> (equivalent to XMI) would be a much better fit as MongoDB could then store
> it pretty much natively and it would even enable some simple queries on the
> annotations directly in MongoDB.
>
> I'm not sure if there are any special considerations to make the JSON
> serialization fully compatible with MongoDBs BSON format.
>
> -- Jens
>
>
> On Wed, Jul 30, 2014 at 10:44 PM, Marshall Schor (JIRA) <dev@uima.apache.org
>> wrote:
>> Marshall Schor created UIMA-3969:
>> ------------------------------------
>>
>> Summary: Add JSON Serialization for CASs and UIMA Descriptors
>> Key: UIMA-3969
>> URL: https://issues.apache.org/jira/browse/UIMA-3969
>> Project: UIMA
>> Issue Type: New Feature
>> Components: Core Java Framework
>> Affects Versions: 2.6.0SDK
>> Reporter: Marshall Schor
>> Assignee: Marshall Schor
>> Priority: Minor
>> Fix For: 2.6.1SDK
>>
>>
>> Recent trends toward moving things into the cloud motivated me to consider
>> what a JSON serialization of the CAS and descriptor metadata (more
>> particularly, type systems) might look like.
>>
>> I've put up a Wiki page with some of the thoughts so far in this
>> exploration, here:
>> https://cwiki.apache.org/confluence/display/UIMA/JSON+serialization+for+UIMA
>>
>> I'm also fooling around with a proof-of-concept implementation, based on
>> our current XMI serialization for the CAS, as well as our
>> MetaDataObject_impl serialization for UIMA descriptors, in order to work
>> out the details. There are additional nits (like how to configure things)
>> not yet worked out.
>>
>> Comments and discussion appreciated; I've put this up as a Jira to record
>> them together - but feel free to use email also for any comments you feel
>> might be better being more ephemeral.
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v6.2#6252)
>>