You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by "A. Soroka" <aj...@virginia.edu> on 2015/10/29 20:02:44 UTC
dataset assembler for JENA-624
Hey, Jena-folks!
One of Andy’s criticisms of my JENA-624 PR was that it lacked Assembler machinery to let people put together datasets. I’ve written a simple one at:
https://github.com/ajs6f/jena/blob/jena-624-dexx/jena-arq/src/main/java/org/apache/jena/sparql/core/assembler/InMemDatasetAssembler.java
https://github.com/ajs6f/jena/blob/jena-624-dexx/jena-arq/src/test/java/org/apache/jena/sparql/core/assembler/TestInMemDatasetAssembler.java
for use with definitions like that shown below my sig.
Does this seem like a reasonable approach? It should allow users to build up their data by whatever means they like, including using inferencing models to generate assertions, then add them to the in-memory container.
---
A. Soroka
The University of Virginia Library
<test:simpleExample> a <http://jena.hpl.hp.com/2005/11/Assembler#MemoryDataset> ;
<http://jena.hpl.hp.com/2005/11/Assembler#defaultGraph> <test:defaultGraphDef> ;
<http://jena.hpl.hp.com/2005/11/Assembler#namedGraph> <test:namedGraphDef> .
<test:defaultGraphDef> a <http://jena.hpl.hp.com/2005/11/Assembler#MemoryModel> ;
<http://jena.hpl.hp.com/2005/11/Assembler#content>
[ <http://jena.hpl.hp.com/2005/11/Assembler#externalContent> <file:/some/triples.nt> . ] .
<test:namedGraphDef> a <http://jena.hpl.hp.com/2005/11/Assembler#MemoryModel> ;
<http://jena.hpl.hp.com/2005/11/Assembler#content>
[ <http://jena.hpl.hp.com/2005/11/Assembler#externalContent> <file:/some/other/triples.nt> . ];
<http://jena.hpl.hp.com/2005/11/Assembler#graphName> <test:namedGraphExample> .
Re: dataset assembler for JENA-624
Posted by "A. Soroka" <aj...@virginia.edu>.
Ha! Actually, that’s how I originally had it (direct link to datasources, files of quads or triples, etc.). But I wanted to make it "look more like" what already exists in the Assembler vocabulary! {grin}
I can definitely work back to that approach, but also leave the option of the “build a model by some means and copy it in” for flexibility. That feature will, as you say, require some careful docs. We don’t want people thinking that they are building some kind of inferring Dataset.
I will add ja:data as you show and set it up to act usefully.
Incidentally, I've added ja:MemoryDataset and now I’ll be adding ja:data. Is there some way I can add these to the assembler vocabulary at:
https://jena.apache.org/documentation/assembler/assembler.ttl
or is that generated from something?
P.S. Sorry about the missing prefixes!
---
A. Soroka
The University of Virginia Library
> On Oct 30, 2015, at 3:05 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> Does this seem like a reasonable approach? It should allow users to
>> build up their data by whatever means they like, including using
>> inferencing models to generate assertions, then add them to the
>> in-memory container.
>
> That's an interesting possibility that hadn't occurred to me. It's a copy-in and the implications of that will need to be clear.
>
> Just relying on ja:MemoryModel and ja:externalContent for the cases of loading files risks a load-copy though. It could be "optimzied"
>
> What about allowing files to be directly pointed to from the assembler for the ja:MemoryDataset?
>
> Example:
>
> -----------------------------------
> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>
> <test:simpleExample> a ja:MemoryDataset ;
> ja:data <file:///some/data.trig> ;
> ja:data <file:///some/data.ttl> ;
> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
> ja:graph
> [ ja:graphName <test:namedGraphExample> ;
> ja:data <file:///some/data3.ttl> ] .
> -----------------------------------
>
> The ability to load trig, NQuads in the dataset:
>
> ja:data <file:///some/data.trig> ;
>
> If a triples form is given, it loads the default graph. This is what Jena does already. These are both RDFDataMgr.read(dataset, file) ;
>
> The ja:graph for specific graphs:
> Default graph (uniformity):
>
> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>
> and named graphs
> ja:graph
> [ ja:graphName <test:namedGraphExample> ;
> ja:data <file:///some/data3.ttl> ] .
>
> I like the build-copy idea - these also add the core tasks of loading a file into a dataset in a direct way.
>
> Thoughts?
>
> Andy
>
>>
>> --- A. Soroka The University of Virginia Library
>
> Prefixes !!!!!!!!!!!!!1
>
> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>
> <test:simpleExample> a ja:MemoryDataset ;
> ja:defaultGraph <test:defaultGraphDef> ;
> ja:namedGraph <test:namedGraphDef> .
>
> <test:defaultGraphDef>
> a ja:MemoryModel ;
> ja:content [ ja:externalContent <file:///some/triples.nt> ] .
>
> <test:namedGraphDef> a ja:MemoryModel ;
> ja:content
> [ ja:externalContent <file:///some/other/triples.nt> ] ;
> ja:graphName <test:namedGraphExample> .
Re: dataset assembler for JENA-624
Posted by "A. Soroka" <aj...@virginia.edu>.
Okay, as per comments on the PR itself, I have removed all use of ja:defaultGraph and ja:graph and simplified to just plain ja:data.
There is no more functionality to create a complex model (e.g. an inferring model) and copy it into the in-memory dataset, which both simplifies the code and will simplify the documentation. There are only the basic use cases as Andy outlines them below.
---
A. Soroka
The University of Virginia Library
> On Nov 5, 2015, at 11:49 AM, Andy Seaborne <an...@apache.org> wrote:
>
> On 05/11/15 16:11, A. Soroka wrote:
>> On Nov 5, 2015, at 11:02 AM, Andy Seaborne <an...@apache.org> wrote:
>>> On 05/11/15 15:44, A. Soroka wrote:
>>>> Yeah, I basically copied the “parallel semantics” of ja:graph and ja:defaultGraph from DatasetAssembler. Perhaps I misunderstood them?
>>> If you think it's a problem, do you have a better name?
>>
>> I’m now a bit confused about what they mean and what ja:data is supposed to mean, and I don’t want to spread my confusion any further! {grin} To (try to) get some clarity: are they supposed to have the same meaning (they do now in my PR, and I really think they do in DatasetAssembler)? If they are supposed to have a different meaning, and one of those meanings is “load from this URI”, what is the difference in that and the newly-introduced ja:data predicate?
>>
>
> They are different.
>
> If you call assembler.open for the object of ja:defaultGraph it will create a model. It does not know about datasets at that point. You get a regular model in-memory.
>
> Adding to the in-mem dataset with addNamedModel or setDefaultModel will be a copy into the datastructures of the in-mem dataset.
>
> The use cases for direct file loading are:
>
>
> 1/ Load file into dataset.
> RDFDataMgr.read(dataset, file)
> Case 1a: quads
> Case 1b: triples
>
>
> 2/ Load file to graph
> Case 2a: default graph
> RDFDataMgr.read(dataset.getdefaultModel(), file)
> Case 2b: named graph
> RDFDataMgr.read(dataset.getNamedModel(), file)
>
> Note: When asked to read triples and it's a quads file, the RDFDataMgr outputs just the triples from the default graph of the input (it does not know it's reading into a named graph - it's just a destination of a stream of triples).
>
> Andy
>
>
>
>
>>>>
>>>> I will definitely gin up some file-based examples. Is there a standard way to do that in Jena tests? I.e. should I put documents in the test classpath, or something else?
>>>
>>> See all the testing/ directories.
>>
>> Thanks! I’ll get myself up to speed on this style of testing.
>>
>> ---
>> A. Soroka
>> The University of Virginia Library
>>
>>>
>>> They could go in src/test/resources but as they are loaded a plain files from testing/ it's more realistic.
>>>
>>> For lots of files and manifests its easier.
>>>
>>> And the base URI issues of class loaded resources are horrible.
>>>
>>> Andy
>>>
>>>>
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>>
>>>>> On Nov 5, 2015, at 10:35 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>>
>>>>> Hi there,
>>>>>
>>>>> I got ja:data working but ja:graph alluded me. After some code digging, it seems it is a synonym for ja:defaultGraph but ja:defaultGraph goes to a full-blown model description.
>>>>>
>>>>> Tests from files would be good and they can be used in documentation examples as well.
>>>>>
>>>>> -----------------------
>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>>
>>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>>> ja:data <file:data.trig> ;
>>>>> ja:data <file:data1.ttl> ;
>>>>> ja:graph [ ja:data <file:data2.ttl> ] ;
>>>>> ja:graph
>>>>> [ ja:graphName <http://example/g3> ;
>>>>> ja:data <file:data3.ttl> ] ;
>>>>> .
>>>>> -----------------------
>>>>>
>>>>> and I tried:
>>>>>
>>>>> DatasetFactory.assemble("assembler.ttl") ;
>>>>>
>>>>> OK - wiring not in place yet
>>>>> then
>>>>>
>>>>> Model m = RDFDataMgr.loadModel("assembler.ttl") ;
>>>>> Resource r = m.createResource("test:simpleExample") ;
>>>>> Dataset ds = (Dataset)new InMemDatasetAssembler().open(r) ;
>>>>>
>>>>> I was expecting ja:graph to be a direct description - not needing a ja:MemoryModel description
>>>>>
>>>>> ja:graph is needed to associate a name with a triples file.
>>>>>
>>>>> I took a go at an implementation:
>>>>> ---------------------
>>>>>
>>>>> open( ...) {
>>>>> ...
>>>>> // Instead of: final Resource defaultGraphDef =
>>>>>
>>>>> multiValueResource(root, pGraph)
>>>>> .forEach(r->readGraphDesc(dataset, r));
>>>>> ...
>>>>>
>>>>>
>>>>>
>>>>> }
>>>>>
>>>>> // May need better checking for expected Resource/String things.
>>>>> private void readGraphDesc(Dataset dataset, Resource r) {
>>>>> String gn = null ;
>>>>> if ( r.hasProperty(pGraphName)) {
>>>>> Resource rgn = r.getProperty(pGraphName).getResource() ;
>>>>> gn = rgn.getURI() ;
>>>>> }
>>>>> String dataFn = getAsStringValue(r, data) ;
>>>>> if ( gn == null )
>>>>> RDFDataMgr.read(dataset.getDefaultModel(), dataFn);
>>>>> else
>>>>> RDFDataMgr.read(dataset.getNamedModel(gn), dataFn);
>>>>> }
>>>>> ---------------------
>>>>>
>>>>> Andy
>>>>>
>>>>> On 02/11/15 16:44, A. Soroka wrote:
>>>>>> I’ve added “direct data links” in the style Andy outlines below to this JENA-624 PR, with tests.
>>>>>>
>>>>>> Feedback eagerly desired!
>>>>>>
>>>>>> ---
>>>>>> A. Soroka
>>>>>> The University of Virginia Library
>>>>>>
>>>>>>> On Oct 30, 2015, at 3:05 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>>>>
>>>>>>>> Does this seem like a reasonable approach? It should allow users to
>>>>>>>> build up their data by whatever means they like, including using
>>>>>>>> inferencing models to generate assertions, then add them to the
>>>>>>>> in-memory container.
>>>>>>>
>>>>>>> That's an interesting possibility that hadn't occurred to me. It's a copy-in and the implications of that will need to be clear.
>>>>>>>
>>>>>>> Just relying on ja:MemoryModel and ja:externalContent for the cases of loading files risks a load-copy though. It could be "optimzied"
>>>>>>>
>>>>>>> What about allowing files to be directly pointed to from the assembler for the ja:MemoryDataset?
>>>>>>>
>>>>>>> Example:
>>>>>>>
>>>>>>> -----------------------------------
>>>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>>>>
>>>>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>>>>> ja:data <file:///some/data.trig> ;
>>>>>>> ja:data <file:///some/data.ttl> ;
>>>>>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>>>>> ja:graph
>>>>>>> [ ja:graphName <test:namedGraphExample> ;
>>>>>>> ja:data <file:///some/data3.ttl> ] .
>>>>>>> -----------------------------------
>>>>>>>
>>>>>>> The ability to load trig, NQuads in the dataset:
>>>>>>>
>>>>>>> ja:data <file:///some/data.trig> ;
>>>>>>>
>>>>>>> If a triples form is given, it loads the default graph. This is what Jena does already. These are both RDFDataMgr.read(dataset, file) ;
>>>>>>>
>>>>>>> The ja:graph for specific graphs:
>>>>>>> Default graph (uniformity):
>>>>>>>
>>>>>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>>>>>
>>>>>>> and named graphs
>>>>>>> ja:graph
>>>>>>> [ ja:graphName <test:namedGraphExample> ;
>>>>>>> ja:data <file:///some/data3.ttl> ] .
>>>>>>>
>>>>>>> I like the build-copy idea - these also add the core tasks of loading a file into a dataset in a direct way.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> Andy
>>>>>>>
>>>>>>>>
>>>>>>>> --- A. Soroka The University of Virginia Library
>>>>>>>
>>>>>>> Prefixes !!!!!!!!!!!!!1
>>>>>>>
>>>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>>>>
>>>>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>>>>> ja:defaultGraph <test:defaultGraphDef> ;
>>>>>>> ja:namedGraph <test:namedGraphDef> .
>>>>>>>
>>>>>>> <test:defaultGraphDef>
>>>>>>> a ja:MemoryModel ;
>>>>>>> ja:content [ ja:externalContent <file:///some/triples.nt> ] .
>>>>>>>
>>>>>>> <test:namedGraphDef> a ja:MemoryModel ;
>>>>>>> ja:content
>>>>>>> [ ja:externalContent <file:///some/other/triples.nt> ] ;
>>>>>>> ja:graphName <test:namedGraphExample> .
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: dataset assembler for JENA-624
Posted by "A. Soroka" <aj...@virginia.edu>.
Okay, I will roll on with the current design, which uses ja:namedGraph to call out named graphs, ja:graphName to name them, and ja:data to load content into them, and ja:data directly on the root to load either triples into the default graph or quads into wherever they need to go, based on their graph node.
I will make a ticket to bring assembler.ttl into sync with the various assembler vocabularies, which we’ll eventually do with all of the copious free time that Jena devs have. {grin}
I will add some file-based tests (as discussed previously) and get another revision in for review.
---
A. Soroka
The University of Virginia Library
> On Nov 7, 2015, at 11:49 AM, Andy Seaborne <an...@apache.org> wrote:
>
> On 05/11/15 18:14, A. Soroka wrote:
>> Ah, you use that code because using ja:graph in the way that you are using it requires distinguishing between named graphs and the default graph, whereas because I am requiring the use of ja:namedGraph to construct a named graph, that part of my code is simpler. This gives me to ask yet another question (sorry!). Should I be allowing the use of ja:graph to construct a named graph as you show? I had restricted it to the use of ja:namedGraph.
>
> Personally, I expect ja:data to be the important case. That's creating the file or files and just loading them.
>
> ja:namedGraph is already in use but the purpose is roughly the same so it's a reasonable use.
>
> ja:defaultGraph is a related case - loading into the default graph and ignoring any named graphs.
>
> That's why I had ja:graph - one predciate, and the further information (name or no name) determined the place to put the triples.
>
> ja:namedGraph + ja:defaultGraph without ja:graph is fine. ja:data is the form to encourage.
>
>> Unfortunately, these predicates don’t appear in assembler.ttl and the versions in DatasetAssemblerVocab have no comments to clarify their use.
>
> History. jena-core is graphs, jena-arq is datasets.
>
> All mixed up with the jena-core/jena-arq split. I felt it was too big a step to do a major code reorg for Jena3 to sort that out overall - and people-bandwidth limitation with seemingly random impact on users/applications.
>
>> Speaking of the fact that there is a separate DatasetAssemblerVocab that contains ja:graph, ja:defaultGraph, and so forth: I added ja:MemoryDataset and ja:data directly to JA. Should they go in another class, specific to InMemoryDatasetAssembler?
>
> Where does not matter too much. ja:MemoryDataset in JA can't be used in jena-core for anything meaningful. ja:data is a braod term so JA makes sense.
>
> We can fine tune when the code is in.
>
> Andy
Re: dataset assembler for JENA-624
Posted by Andy Seaborne <an...@apache.org>.
On 05/11/15 18:14, A. Soroka wrote:
> Ah, you use that code because using ja:graph in the way that you are
> using it requires distinguishing between named graphs and the default
> graph, whereas because I am requiring the use of ja:namedGraph to
> construct a named graph, that part of my code is simpler. This gives
> me to ask yet another question (sorry!). Should I be allowing the use
> of ja:graph to construct a named graph as you show? I had restricted
> it to the use of ja:namedGraph.
Personally, I expect ja:data to be the important case. That's creating
the file or files and just loading them.
ja:namedGraph is already in use but the purpose is roughly the same so
it's a reasonable use.
ja:defaultGraph is a related case - loading into the default graph and
ignoring any named graphs.
That's why I had ja:graph - one predciate, and the further information
(name or no name) determined the place to put the triples.
ja:namedGraph + ja:defaultGraph without ja:graph is fine. ja:data is
the form to encourage.
> Unfortunately, these predicates don’t
> appear in assembler.ttl and the versions in DatasetAssemblerVocab
> have no comments to clarify their use.
History. jena-core is graphs, jena-arq is datasets.
All mixed up with the jena-core/jena-arq split. I felt it was too big a
step to do a major code reorg for Jena3 to sort that out overall - and
people-bandwidth limitation with seemingly random impact on
users/applications.
> Speaking of the fact that there is a separate DatasetAssemblerVocab
> that contains ja:graph, ja:defaultGraph, and so forth: I added
> ja:MemoryDataset and ja:data directly to JA. Should they go in
> another class, specific to InMemoryDatasetAssembler?
>
Where does not matter too much.
ja:MemoryDataset in JA can't be used in jena-core for anything
meaningful. ja:data is a braod term so JA makes sense.
We can fine tune when the code is in.
Andy
Re: dataset assembler for JENA-624
Posted by "A. Soroka" <aj...@virginia.edu>.
Ah, you use that code because using ja:graph in the way that you are using it requires distinguishing between named graphs and the default graph, whereas because I am requiring the use of ja:namedGraph to construct a named graph, that part of my code is simpler. This gives me to ask yet another question (sorry!). Should I be allowing the use of ja:graph to construct a named graph as you show? I had restricted it to the use of ja:namedGraph. Unfortunately, these predicates don’t appear in assembler.ttl and the versions in DatasetAssemblerVocab have no comments to clarify their use.
Speaking of the fact that there is a separate DatasetAssemblerVocab that contains ja:graph, ja:defaultGraph, and so forth: I added ja:MemoryDataset and ja:data directly to JA. Should they go in another class, specific to InMemoryDatasetAssembler?
---
A. Soroka
The University of Virginia Library
> On Nov 5, 2015, at 12:57 PM, Andy Seaborne <an...@apache.org> wrote:
>
> On 05/11/15 17:22, A. Soroka wrote:
>> I just pushed an update to simplify this PR as discussed. It looks pretty similar to what you are showing below, but I used ja:data in both places (i.e. into the dataset and into a given graph, default or named) to allow for directly reading files into either. If ja:data is on the root, it reads either quads into the dataset or triples into the default graph, depending on what is on the other end of the ja:data triple. If ja:data is on the object of a ja:namedGraph triple, it reads into the appropriate named graph as specified by a ja:graphName triple, so:
>>
>> root a ja:MemoryDataset ;
>> ja:data <file://myQuads.nq> ;
>> ja:data <file://myTriplesforDefaultGraph.nt> ;
>> ja:namedGraph [ ja:graphName <info:mygraph> ; ja:data <file://someTriplesForMyGraph> ] .
>>
>> I don’t understand what you mean by the method readGraphDesc? Is that supposed to read an assembler description of some kind? I thought we were excising that?
>
> See earlier: it handles the [] case which takes a touch more code so I put it a name to that.
>
>>> // May need better checking for expected Resource/String things.
>>> private void readGraphDesc(Dataset dataset, Resource r) {
>>> String gn = null ;
>>> if ( r.hasProperty(pGraphName)) {
>>> Resource rgn = r.getProperty(pGraphName).getResource() ;
>>> gn = rgn.getURI() ;
>>> }
>>> String dataFn = getAsStringValue(r, data) ;
>>> if ( gn == null )
>>> RDFDataMgr.read(dataset.getDefaultModel(), dataFn);
>>> else
>>> RDFDataMgr.read(dataset.getNamedModel(gn), dataFn);
>>> }
Re: dataset assembler for JENA-624
Posted by Andy Seaborne <an...@apache.org>.
On 05/11/15 17:22, A. Soroka wrote:
> I just pushed an update to simplify this PR as discussed. It looks pretty similar to what you are showing below, but I used ja:data in both places (i.e. into the dataset and into a given graph, default or named) to allow for directly reading files into either. If ja:data is on the root, it reads either quads into the dataset or triples into the default graph, depending on what is on the other end of the ja:data triple. If ja:data is on the object of a ja:namedGraph triple, it reads into the appropriate named graph as specified by a ja:graphName triple, so:
>
> root a ja:MemoryDataset ;
> ja:data <file://myQuads.nq> ;
> ja:data <file://myTriplesforDefaultGraph.nt> ;
> ja:namedGraph [ ja:graphName <info:mygraph> ; ja:data <file://someTriplesForMyGraph> ] .
>
> I don’t understand what you mean by the method readGraphDesc? Is that supposed to read an assembler description of some kind? I thought we were excising that?
See earlier: it handles the [] case which takes a touch more code so I
put it a name to that.
>> // May need better checking for expected Resource/String things.
>> private void readGraphDesc(Dataset dataset, Resource r) {
>> String gn = null ;
>> if ( r.hasProperty(pGraphName)) {
>> Resource rgn = r.getProperty(pGraphName).getResource() ;
>> gn = rgn.getURI() ;
>> }
>> String dataFn = getAsStringValue(r, data) ;
>> if ( gn == null )
>> RDFDataMgr.read(dataset.getDefaultModel(), dataFn);
>> else
>> RDFDataMgr.read(dataset.getNamedModel(gn), dataFn);
>> }
Re: dataset assembler for JENA-624
Posted by "A. Soroka" <aj...@virginia.edu>.
I just pushed an update to simplify this PR as discussed. It looks pretty similar to what you are showing below, but I used ja:data in both places (i.e. into the dataset and into a given graph, default or named) to allow for directly reading files into either. If ja:data is on the root, it reads either quads into the dataset or triples into the default graph, depending on what is on the other end of the ja:data triple. If ja:data is on the object of a ja:namedGraph triple, it reads into the appropriate named graph as specified by a ja:graphName triple, so:
root a ja:MemoryDataset ;
ja:data <file://myQuads.nq> ;
ja:data <file://myTriplesforDefaultGraph.nt> ;
ja:namedGraph [ ja:graphName <info:mygraph> ; ja:data <file://someTriplesForMyGraph> ] .
I don’t understand what you mean by the method readGraphDesc? Is that supposed to read an assembler description of some kind? I thought we were excising that?
---
A. Soroka
The University of Virginia Library
> On Nov 5, 2015, at 12:05 PM, Andy Seaborne <an...@apache.org> wrote:
>
> On 05/11/15 16:49, Andy Seaborne wrote:
>> On 05/11/15 16:11, A. Soroka wrote:
>>> On Nov 5, 2015, at 11:02 AM, Andy Seaborne <an...@apache.org> wrote:
>>>> On 05/11/15 15:44, A. Soroka wrote:
>>>>> Yeah, I basically copied the “parallel semantics” of ja:graph and
>>>>> ja:defaultGraph from DatasetAssembler. Perhaps I misunderstood them?
>>>> If you think it's a problem, do you have a better name?
>>>
>>> I’m now a bit confused about what they mean and what ja:data is
>>> supposed to mean, and I don’t want to spread my confusion any further!
>>> {grin} To (try to) get some clarity: are they supposed to have the
>>> same meaning (they do now in my PR, and I really think they do in
>>> DatasetAssembler)? If they are supposed to have a different meaning,
>>> and one of those meanings is “load from this URI”, what is the
>>> difference in that and the newly-introduced ja:data predicate?
>>>
>>
>> They are different.
>>
>> If you call assembler.open for the object of ja:defaultGraph it will
>> create a model. It does not know about datasets at that point. You get
>> a regular model in-memory.
>>
>> Adding to the in-mem dataset with addNamedModel or setDefaultModel will
>> be a copy into the datastructures of the in-mem dataset.
>>
>> The use cases for direct file loading are:
>>
>>
>> 1/ Load file into dataset.
>> RDFDataMgr.read(dataset, file)
>> Case 1a: quads
>> Case 1b: triples
>>
>>
>> 2/ Load file to graph
>> Case 2a: default graph
>> RDFDataMgr.read(dataset.getdefaultModel(), file)
>> Case 2b: named graph
>> RDFDataMgr.read(dataset.getNamedModel(), file)
>>
>> Note: When asked to read triples and it's a quads file, the RDFDataMgr
>> outputs just the triples from the default graph of the input (it does
>> not know it's reading into a named graph - it's just a destination of a
>> stream of triples).
>>
>> Andy
>>
>>
>>
>
> In code, that is something like this: (error checking could be enhanced):
>
> @Override
> public Dataset open(final Assembler assembler, final Resource root, final Mode mode) {
> checkType(root, MemoryDataset);
> Dataset dataset = createTxnMem();
> setContext(root, dataset.getContext());
>
> // Case 1
> multiValue(root, pGraph).stream().map(RDFNode::asResource)
> .forEach(r->readGraphDesc(dataset, r));
>
> // Case 2
> multiValue(root, data).stream().map(RDFNode::asResource)
> .forEach(x -> RDFDataMgr.read(dataset, x.getURI()));
>
> // If desired:
> // ... ja:defaultGraph
> // ... ja:namedGraph
>
> return dataset;
> }
>
> (and I hope my plague of tabs got sorted out).
>
> Andy
>
Re: dataset assembler for JENA-624
Posted by Andy Seaborne <an...@apache.org>.
On 05/11/15 16:49, Andy Seaborne wrote:
> On 05/11/15 16:11, A. Soroka wrote:
>> On Nov 5, 2015, at 11:02 AM, Andy Seaborne <an...@apache.org> wrote:
>>> On 05/11/15 15:44, A. Soroka wrote:
>>>> Yeah, I basically copied the “parallel semantics” of ja:graph and
>>>> ja:defaultGraph from DatasetAssembler. Perhaps I misunderstood them?
>>> If you think it's a problem, do you have a better name?
>>
>> I’m now a bit confused about what they mean and what ja:data is
>> supposed to mean, and I don’t want to spread my confusion any further!
>> {grin} To (try to) get some clarity: are they supposed to have the
>> same meaning (they do now in my PR, and I really think they do in
>> DatasetAssembler)? If they are supposed to have a different meaning,
>> and one of those meanings is “load from this URI”, what is the
>> difference in that and the newly-introduced ja:data predicate?
>>
>
> They are different.
>
> If you call assembler.open for the object of ja:defaultGraph it will
> create a model. It does not know about datasets at that point. You get
> a regular model in-memory.
>
> Adding to the in-mem dataset with addNamedModel or setDefaultModel will
> be a copy into the datastructures of the in-mem dataset.
>
> The use cases for direct file loading are:
>
>
> 1/ Load file into dataset.
> RDFDataMgr.read(dataset, file)
> Case 1a: quads
> Case 1b: triples
>
>
> 2/ Load file to graph
> Case 2a: default graph
> RDFDataMgr.read(dataset.getdefaultModel(), file)
> Case 2b: named graph
> RDFDataMgr.read(dataset.getNamedModel(), file)
>
> Note: When asked to read triples and it's a quads file, the RDFDataMgr
> outputs just the triples from the default graph of the input (it does
> not know it's reading into a named graph - it's just a destination of a
> stream of triples).
>
> Andy
>
>
>
In code, that is something like this: (error checking could be enhanced):
@Override
public Dataset open(final Assembler assembler, final Resource root,
final Mode mode) {
checkType(root, MemoryDataset);
Dataset dataset = createTxnMem();
setContext(root, dataset.getContext());
// Case 1
multiValue(root, pGraph).stream().map(RDFNode::asResource)
.forEach(r->readGraphDesc(dataset, r));
// Case 2
multiValue(root, data).stream().map(RDFNode::asResource)
.forEach(x -> RDFDataMgr.read(dataset, x.getURI()));
// If desired:
// ... ja:defaultGraph
// ... ja:namedGraph
return dataset;
}
(and I hope my plague of tabs got sorted out).
Andy
Re: dataset assembler for JENA-624
Posted by Andy Seaborne <an...@apache.org>.
On 05/11/15 16:11, A. Soroka wrote:
> On Nov 5, 2015, at 11:02 AM, Andy Seaborne <an...@apache.org> wrote:
>> On 05/11/15 15:44, A. Soroka wrote:
>>> Yeah, I basically copied the “parallel semantics” of ja:graph and ja:defaultGraph from DatasetAssembler. Perhaps I misunderstood them?
>> If you think it's a problem, do you have a better name?
>
> I’m now a bit confused about what they mean and what ja:data is supposed to mean, and I don’t want to spread my confusion any further! {grin} To (try to) get some clarity: are they supposed to have the same meaning (they do now in my PR, and I really think they do in DatasetAssembler)? If they are supposed to have a different meaning, and one of those meanings is “load from this URI”, what is the difference in that and the newly-introduced ja:data predicate?
>
They are different.
If you call assembler.open for the object of ja:defaultGraph it will
create a model. It does not know about datasets at that point. You get
a regular model in-memory.
Adding to the in-mem dataset with addNamedModel or setDefaultModel will
be a copy into the datastructures of the in-mem dataset.
The use cases for direct file loading are:
1/ Load file into dataset.
RDFDataMgr.read(dataset, file)
Case 1a: quads
Case 1b: triples
2/ Load file to graph
Case 2a: default graph
RDFDataMgr.read(dataset.getdefaultModel(), file)
Case 2b: named graph
RDFDataMgr.read(dataset.getNamedModel(), file)
Note: When asked to read triples and it's a quads file, the RDFDataMgr
outputs just the triples from the default graph of the input (it does
not know it's reading into a named graph - it's just a destination of a
stream of triples).
Andy
>>>
>>> I will definitely gin up some file-based examples. Is there a standard way to do that in Jena tests? I.e. should I put documents in the test classpath, or something else?
>>
>> See all the testing/ directories.
>
> Thanks! I’ll get myself up to speed on this style of testing.
>
> ---
> A. Soroka
> The University of Virginia Library
>
>>
>> They could go in src/test/resources but as they are loaded a plain files from testing/ it's more realistic.
>>
>> For lots of files and manifests its easier.
>>
>> And the base URI issues of class loaded resources are horrible.
>>
>> Andy
>>
>>>
>>> ---
>>> A. Soroka
>>> The University of Virginia Library
>>>
>>>> On Nov 5, 2015, at 10:35 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>> Hi there,
>>>>
>>>> I got ja:data working but ja:graph alluded me. After some code digging, it seems it is a synonym for ja:defaultGraph but ja:defaultGraph goes to a full-blown model description.
>>>>
>>>> Tests from files would be good and they can be used in documentation examples as well.
>>>>
>>>> -----------------------
>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>
>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>> ja:data <file:data.trig> ;
>>>> ja:data <file:data1.ttl> ;
>>>> ja:graph [ ja:data <file:data2.ttl> ] ;
>>>> ja:graph
>>>> [ ja:graphName <http://example/g3> ;
>>>> ja:data <file:data3.ttl> ] ;
>>>> .
>>>> -----------------------
>>>>
>>>> and I tried:
>>>>
>>>> DatasetFactory.assemble("assembler.ttl") ;
>>>>
>>>> OK - wiring not in place yet
>>>> then
>>>>
>>>> Model m = RDFDataMgr.loadModel("assembler.ttl") ;
>>>> Resource r = m.createResource("test:simpleExample") ;
>>>> Dataset ds = (Dataset)new InMemDatasetAssembler().open(r) ;
>>>>
>>>> I was expecting ja:graph to be a direct description - not needing a ja:MemoryModel description
>>>>
>>>> ja:graph is needed to associate a name with a triples file.
>>>>
>>>> I took a go at an implementation:
>>>> ---------------------
>>>>
>>>> open( ...) {
>>>> ...
>>>> // Instead of: final Resource defaultGraphDef =
>>>>
>>>> multiValueResource(root, pGraph)
>>>> .forEach(r->readGraphDesc(dataset, r));
>>>> ...
>>>>
>>>>
>>>>
>>>> }
>>>>
>>>> // May need better checking for expected Resource/String things.
>>>> private void readGraphDesc(Dataset dataset, Resource r) {
>>>> String gn = null ;
>>>> if ( r.hasProperty(pGraphName)) {
>>>> Resource rgn = r.getProperty(pGraphName).getResource() ;
>>>> gn = rgn.getURI() ;
>>>> }
>>>> String dataFn = getAsStringValue(r, data) ;
>>>> if ( gn == null )
>>>> RDFDataMgr.read(dataset.getDefaultModel(), dataFn);
>>>> else
>>>> RDFDataMgr.read(dataset.getNamedModel(gn), dataFn);
>>>> }
>>>> ---------------------
>>>>
>>>> Andy
>>>>
>>>> On 02/11/15 16:44, A. Soroka wrote:
>>>>> I’ve added “direct data links” in the style Andy outlines below to this JENA-624 PR, with tests.
>>>>>
>>>>> Feedback eagerly desired!
>>>>>
>>>>> ---
>>>>> A. Soroka
>>>>> The University of Virginia Library
>>>>>
>>>>>> On Oct 30, 2015, at 3:05 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>>>
>>>>>>> Does this seem like a reasonable approach? It should allow users to
>>>>>>> build up their data by whatever means they like, including using
>>>>>>> inferencing models to generate assertions, then add them to the
>>>>>>> in-memory container.
>>>>>>
>>>>>> That's an interesting possibility that hadn't occurred to me. It's a copy-in and the implications of that will need to be clear.
>>>>>>
>>>>>> Just relying on ja:MemoryModel and ja:externalContent for the cases of loading files risks a load-copy though. It could be "optimzied"
>>>>>>
>>>>>> What about allowing files to be directly pointed to from the assembler for the ja:MemoryDataset?
>>>>>>
>>>>>> Example:
>>>>>>
>>>>>> -----------------------------------
>>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>>>
>>>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>>>> ja:data <file:///some/data.trig> ;
>>>>>> ja:data <file:///some/data.ttl> ;
>>>>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>>>> ja:graph
>>>>>> [ ja:graphName <test:namedGraphExample> ;
>>>>>> ja:data <file:///some/data3.ttl> ] .
>>>>>> -----------------------------------
>>>>>>
>>>>>> The ability to load trig, NQuads in the dataset:
>>>>>>
>>>>>> ja:data <file:///some/data.trig> ;
>>>>>>
>>>>>> If a triples form is given, it loads the default graph. This is what Jena does already. These are both RDFDataMgr.read(dataset, file) ;
>>>>>>
>>>>>> The ja:graph for specific graphs:
>>>>>> Default graph (uniformity):
>>>>>>
>>>>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>>>>
>>>>>> and named graphs
>>>>>> ja:graph
>>>>>> [ ja:graphName <test:namedGraphExample> ;
>>>>>> ja:data <file:///some/data3.ttl> ] .
>>>>>>
>>>>>> I like the build-copy idea - these also add the core tasks of loading a file into a dataset in a direct way.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>>>
>>>>>>> --- A. Soroka The University of Virginia Library
>>>>>>
>>>>>> Prefixes !!!!!!!!!!!!!1
>>>>>>
>>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>>>
>>>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>>>> ja:defaultGraph <test:defaultGraphDef> ;
>>>>>> ja:namedGraph <test:namedGraphDef> .
>>>>>>
>>>>>> <test:defaultGraphDef>
>>>>>> a ja:MemoryModel ;
>>>>>> ja:content [ ja:externalContent <file:///some/triples.nt> ] .
>>>>>>
>>>>>> <test:namedGraphDef> a ja:MemoryModel ;
>>>>>> ja:content
>>>>>> [ ja:externalContent <file:///some/other/triples.nt> ] ;
>>>>>> ja:graphName <test:namedGraphExample> .
>>>>>
>>>>
>>>
>>
>
Re: dataset assembler for JENA-624
Posted by "A. Soroka" <aj...@virginia.edu>.
On Nov 5, 2015, at 11:02 AM, Andy Seaborne <an...@apache.org> wrote:
> On 05/11/15 15:44, A. Soroka wrote:
>> Yeah, I basically copied the “parallel semantics” of ja:graph and ja:defaultGraph from DatasetAssembler. Perhaps I misunderstood them?
> If you think it's a problem, do you have a better name?
I’m now a bit confused about what they mean and what ja:data is supposed to mean, and I don’t want to spread my confusion any further! {grin} To (try to) get some clarity: are they supposed to have the same meaning (they do now in my PR, and I really think they do in DatasetAssembler)? If they are supposed to have a different meaning, and one of those meanings is “load from this URI”, what is the difference in that and the newly-introduced ja:data predicate?
>>
>> I will definitely gin up some file-based examples. Is there a standard way to do that in Jena tests? I.e. should I put documents in the test classpath, or something else?
>
> See all the testing/ directories.
Thanks! I’ll get myself up to speed on this style of testing.
---
A. Soroka
The University of Virginia Library
>
> They could go in src/test/resources but as they are loaded a plain files from testing/ it's more realistic.
>
> For lots of files and manifests its easier.
>
> And the base URI issues of class loaded resources are horrible.
>
> Andy
>
>>
>> ---
>> A. Soroka
>> The University of Virginia Library
>>
>>> On Nov 5, 2015, at 10:35 AM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> Hi there,
>>>
>>> I got ja:data working but ja:graph alluded me. After some code digging, it seems it is a synonym for ja:defaultGraph but ja:defaultGraph goes to a full-blown model description.
>>>
>>> Tests from files would be good and they can be used in documentation examples as well.
>>>
>>> -----------------------
>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>
>>> <test:simpleExample> a ja:MemoryDataset ;
>>> ja:data <file:data.trig> ;
>>> ja:data <file:data1.ttl> ;
>>> ja:graph [ ja:data <file:data2.ttl> ] ;
>>> ja:graph
>>> [ ja:graphName <http://example/g3> ;
>>> ja:data <file:data3.ttl> ] ;
>>> .
>>> -----------------------
>>>
>>> and I tried:
>>>
>>> DatasetFactory.assemble("assembler.ttl") ;
>>>
>>> OK - wiring not in place yet
>>> then
>>>
>>> Model m = RDFDataMgr.loadModel("assembler.ttl") ;
>>> Resource r = m.createResource("test:simpleExample") ;
>>> Dataset ds = (Dataset)new InMemDatasetAssembler().open(r) ;
>>>
>>> I was expecting ja:graph to be a direct description - not needing a ja:MemoryModel description
>>>
>>> ja:graph is needed to associate a name with a triples file.
>>>
>>> I took a go at an implementation:
>>> ---------------------
>>>
>>> open( ...) {
>>> ...
>>> // Instead of: final Resource defaultGraphDef =
>>>
>>> multiValueResource(root, pGraph)
>>> .forEach(r->readGraphDesc(dataset, r));
>>> ...
>>>
>>>
>>>
>>> }
>>>
>>> // May need better checking for expected Resource/String things.
>>> private void readGraphDesc(Dataset dataset, Resource r) {
>>> String gn = null ;
>>> if ( r.hasProperty(pGraphName)) {
>>> Resource rgn = r.getProperty(pGraphName).getResource() ;
>>> gn = rgn.getURI() ;
>>> }
>>> String dataFn = getAsStringValue(r, data) ;
>>> if ( gn == null )
>>> RDFDataMgr.read(dataset.getDefaultModel(), dataFn);
>>> else
>>> RDFDataMgr.read(dataset.getNamedModel(gn), dataFn);
>>> }
>>> ---------------------
>>>
>>> Andy
>>>
>>> On 02/11/15 16:44, A. Soroka wrote:
>>>> I’ve added “direct data links” in the style Andy outlines below to this JENA-624 PR, with tests.
>>>>
>>>> Feedback eagerly desired!
>>>>
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>>
>>>>> On Oct 30, 2015, at 3:05 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>>
>>>>>> Does this seem like a reasonable approach? It should allow users to
>>>>>> build up their data by whatever means they like, including using
>>>>>> inferencing models to generate assertions, then add them to the
>>>>>> in-memory container.
>>>>>
>>>>> That's an interesting possibility that hadn't occurred to me. It's a copy-in and the implications of that will need to be clear.
>>>>>
>>>>> Just relying on ja:MemoryModel and ja:externalContent for the cases of loading files risks a load-copy though. It could be "optimzied"
>>>>>
>>>>> What about allowing files to be directly pointed to from the assembler for the ja:MemoryDataset?
>>>>>
>>>>> Example:
>>>>>
>>>>> -----------------------------------
>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>>
>>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>>> ja:data <file:///some/data.trig> ;
>>>>> ja:data <file:///some/data.ttl> ;
>>>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>>> ja:graph
>>>>> [ ja:graphName <test:namedGraphExample> ;
>>>>> ja:data <file:///some/data3.ttl> ] .
>>>>> -----------------------------------
>>>>>
>>>>> The ability to load trig, NQuads in the dataset:
>>>>>
>>>>> ja:data <file:///some/data.trig> ;
>>>>>
>>>>> If a triples form is given, it loads the default graph. This is what Jena does already. These are both RDFDataMgr.read(dataset, file) ;
>>>>>
>>>>> The ja:graph for specific graphs:
>>>>> Default graph (uniformity):
>>>>>
>>>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>>>
>>>>> and named graphs
>>>>> ja:graph
>>>>> [ ja:graphName <test:namedGraphExample> ;
>>>>> ja:data <file:///some/data3.ttl> ] .
>>>>>
>>>>> I like the build-copy idea - these also add the core tasks of loading a file into a dataset in a direct way.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Andy
>>>>>
>>>>>>
>>>>>> --- A. Soroka The University of Virginia Library
>>>>>
>>>>> Prefixes !!!!!!!!!!!!!1
>>>>>
>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>>
>>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>>> ja:defaultGraph <test:defaultGraphDef> ;
>>>>> ja:namedGraph <test:namedGraphDef> .
>>>>>
>>>>> <test:defaultGraphDef>
>>>>> a ja:MemoryModel ;
>>>>> ja:content [ ja:externalContent <file:///some/triples.nt> ] .
>>>>>
>>>>> <test:namedGraphDef> a ja:MemoryModel ;
>>>>> ja:content
>>>>> [ ja:externalContent <file:///some/other/triples.nt> ] ;
>>>>> ja:graphName <test:namedGraphExample> .
>>>>
>>>
>>
>
Re: dataset assembler for JENA-624
Posted by Andy Seaborne <an...@apache.org>.
On 05/11/15 15:44, A. Soroka wrote:
> Yeah, I basically copied the “parallel semantics” of ja:graph and ja:defaultGraph from DatasetAssembler. Perhaps I misunderstood them?
If you think it's a problem, do you have a better name?
>
> I will definitely gin up some file-based examples. Is there a standard way to do that in Jena tests? I.e. should I put documents in the test classpath, or something else?
See all the testing/ directories.
They could go in src/test/resources but as they are loaded a plain files
from testing/ it's more realistic.
For lots of files and manifests its easier.
And the base URI issues of class loaded resources are horrible.
Andy
>
> ---
> A. Soroka
> The University of Virginia Library
>
>> On Nov 5, 2015, at 10:35 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>> Hi there,
>>
>> I got ja:data working but ja:graph alluded me. After some code digging, it seems it is a synonym for ja:defaultGraph but ja:defaultGraph goes to a full-blown model description.
>>
>> Tests from files would be good and they can be used in documentation examples as well.
>>
>> -----------------------
>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>
>> <test:simpleExample> a ja:MemoryDataset ;
>> ja:data <file:data.trig> ;
>> ja:data <file:data1.ttl> ;
>> ja:graph [ ja:data <file:data2.ttl> ] ;
>> ja:graph
>> [ ja:graphName <http://example/g3> ;
>> ja:data <file:data3.ttl> ] ;
>> .
>> -----------------------
>>
>> and I tried:
>>
>> DatasetFactory.assemble("assembler.ttl") ;
>>
>> OK - wiring not in place yet
>> then
>>
>> Model m = RDFDataMgr.loadModel("assembler.ttl") ;
>> Resource r = m.createResource("test:simpleExample") ;
>> Dataset ds = (Dataset)new InMemDatasetAssembler().open(r) ;
>>
>> I was expecting ja:graph to be a direct description - not needing a ja:MemoryModel description
>>
>> ja:graph is needed to associate a name with a triples file.
>>
>> I took a go at an implementation:
>> ---------------------
>>
>> open( ...) {
>> ...
>> // Instead of: final Resource defaultGraphDef =
>>
>> multiValueResource(root, pGraph)
>> .forEach(r->readGraphDesc(dataset, r));
>> ...
>>
>>
>>
>> }
>>
>> // May need better checking for expected Resource/String things.
>> private void readGraphDesc(Dataset dataset, Resource r) {
>> String gn = null ;
>> if ( r.hasProperty(pGraphName)) {
>> Resource rgn = r.getProperty(pGraphName).getResource() ;
>> gn = rgn.getURI() ;
>> }
>> String dataFn = getAsStringValue(r, data) ;
>> if ( gn == null )
>> RDFDataMgr.read(dataset.getDefaultModel(), dataFn);
>> else
>> RDFDataMgr.read(dataset.getNamedModel(gn), dataFn);
>> }
>> ---------------------
>>
>> Andy
>>
>> On 02/11/15 16:44, A. Soroka wrote:
>>> I’ve added “direct data links” in the style Andy outlines below to this JENA-624 PR, with tests.
>>>
>>> Feedback eagerly desired!
>>>
>>> ---
>>> A. Soroka
>>> The University of Virginia Library
>>>
>>>> On Oct 30, 2015, at 3:05 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>>> Does this seem like a reasonable approach? It should allow users to
>>>>> build up their data by whatever means they like, including using
>>>>> inferencing models to generate assertions, then add them to the
>>>>> in-memory container.
>>>>
>>>> That's an interesting possibility that hadn't occurred to me. It's a copy-in and the implications of that will need to be clear.
>>>>
>>>> Just relying on ja:MemoryModel and ja:externalContent for the cases of loading files risks a load-copy though. It could be "optimzied"
>>>>
>>>> What about allowing files to be directly pointed to from the assembler for the ja:MemoryDataset?
>>>>
>>>> Example:
>>>>
>>>> -----------------------------------
>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>
>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>> ja:data <file:///some/data.trig> ;
>>>> ja:data <file:///some/data.ttl> ;
>>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>> ja:graph
>>>> [ ja:graphName <test:namedGraphExample> ;
>>>> ja:data <file:///some/data3.ttl> ] .
>>>> -----------------------------------
>>>>
>>>> The ability to load trig, NQuads in the dataset:
>>>>
>>>> ja:data <file:///some/data.trig> ;
>>>>
>>>> If a triples form is given, it loads the default graph. This is what Jena does already. These are both RDFDataMgr.read(dataset, file) ;
>>>>
>>>> The ja:graph for specific graphs:
>>>> Default graph (uniformity):
>>>>
>>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>>
>>>> and named graphs
>>>> ja:graph
>>>> [ ja:graphName <test:namedGraphExample> ;
>>>> ja:data <file:///some/data3.ttl> ] .
>>>>
>>>> I like the build-copy idea - these also add the core tasks of loading a file into a dataset in a direct way.
>>>>
>>>> Thoughts?
>>>>
>>>> Andy
>>>>
>>>>>
>>>>> --- A. Soroka The University of Virginia Library
>>>>
>>>> Prefixes !!!!!!!!!!!!!1
>>>>
>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>>
>>>> <test:simpleExample> a ja:MemoryDataset ;
>>>> ja:defaultGraph <test:defaultGraphDef> ;
>>>> ja:namedGraph <test:namedGraphDef> .
>>>>
>>>> <test:defaultGraphDef>
>>>> a ja:MemoryModel ;
>>>> ja:content [ ja:externalContent <file:///some/triples.nt> ] .
>>>>
>>>> <test:namedGraphDef> a ja:MemoryModel ;
>>>> ja:content
>>>> [ ja:externalContent <file:///some/other/triples.nt> ] ;
>>>> ja:graphName <test:namedGraphExample> .
>>>
>>
>
Re: dataset assembler for JENA-624
Posted by "A. Soroka" <aj...@virginia.edu>.
Yeah, I basically copied the “parallel semantics” of ja:graph and ja:defaultGraph from DatasetAssembler. Perhaps I misunderstood them?
I will definitely gin up some file-based examples. Is there a standard way to do that in Jena tests? I.e. should I put documents in the test classpath, or something else?
---
A. Soroka
The University of Virginia Library
> On Nov 5, 2015, at 10:35 AM, Andy Seaborne <an...@apache.org> wrote:
>
> Hi there,
>
> I got ja:data working but ja:graph alluded me. After some code digging, it seems it is a synonym for ja:defaultGraph but ja:defaultGraph goes to a full-blown model description.
>
> Tests from files would be good and they can be used in documentation examples as well.
>
> -----------------------
> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>
> <test:simpleExample> a ja:MemoryDataset ;
> ja:data <file:data.trig> ;
> ja:data <file:data1.ttl> ;
> ja:graph [ ja:data <file:data2.ttl> ] ;
> ja:graph
> [ ja:graphName <http://example/g3> ;
> ja:data <file:data3.ttl> ] ;
> .
> -----------------------
>
> and I tried:
>
> DatasetFactory.assemble("assembler.ttl") ;
>
> OK - wiring not in place yet
> then
>
> Model m = RDFDataMgr.loadModel("assembler.ttl") ;
> Resource r = m.createResource("test:simpleExample") ;
> Dataset ds = (Dataset)new InMemDatasetAssembler().open(r) ;
>
> I was expecting ja:graph to be a direct description - not needing a ja:MemoryModel description
>
> ja:graph is needed to associate a name with a triples file.
>
> I took a go at an implementation:
> ---------------------
>
> open( ...) {
> ...
> // Instead of: final Resource defaultGraphDef =
>
> multiValueResource(root, pGraph)
> .forEach(r->readGraphDesc(dataset, r));
> ...
>
>
>
> }
>
> // May need better checking for expected Resource/String things.
> private void readGraphDesc(Dataset dataset, Resource r) {
> String gn = null ;
> if ( r.hasProperty(pGraphName)) {
> Resource rgn = r.getProperty(pGraphName).getResource() ;
> gn = rgn.getURI() ;
> }
> String dataFn = getAsStringValue(r, data) ;
> if ( gn == null )
> RDFDataMgr.read(dataset.getDefaultModel(), dataFn);
> else
> RDFDataMgr.read(dataset.getNamedModel(gn), dataFn);
> }
> ---------------------
>
> Andy
>
> On 02/11/15 16:44, A. Soroka wrote:
>> I’ve added “direct data links” in the style Andy outlines below to this JENA-624 PR, with tests.
>>
>> Feedback eagerly desired!
>>
>> ---
>> A. Soroka
>> The University of Virginia Library
>>
>>> On Oct 30, 2015, at 3:05 PM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>>> Does this seem like a reasonable approach? It should allow users to
>>>> build up their data by whatever means they like, including using
>>>> inferencing models to generate assertions, then add them to the
>>>> in-memory container.
>>>
>>> That's an interesting possibility that hadn't occurred to me. It's a copy-in and the implications of that will need to be clear.
>>>
>>> Just relying on ja:MemoryModel and ja:externalContent for the cases of loading files risks a load-copy though. It could be "optimzied"
>>>
>>> What about allowing files to be directly pointed to from the assembler for the ja:MemoryDataset?
>>>
>>> Example:
>>>
>>> -----------------------------------
>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>
>>> <test:simpleExample> a ja:MemoryDataset ;
>>> ja:data <file:///some/data.trig> ;
>>> ja:data <file:///some/data.ttl> ;
>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>> ja:graph
>>> [ ja:graphName <test:namedGraphExample> ;
>>> ja:data <file:///some/data3.ttl> ] .
>>> -----------------------------------
>>>
>>> The ability to load trig, NQuads in the dataset:
>>>
>>> ja:data <file:///some/data.trig> ;
>>>
>>> If a triples form is given, it loads the default graph. This is what Jena does already. These are both RDFDataMgr.read(dataset, file) ;
>>>
>>> The ja:graph for specific graphs:
>>> Default graph (uniformity):
>>>
>>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>>
>>> and named graphs
>>> ja:graph
>>> [ ja:graphName <test:namedGraphExample> ;
>>> ja:data <file:///some/data3.ttl> ] .
>>>
>>> I like the build-copy idea - these also add the core tasks of loading a file into a dataset in a direct way.
>>>
>>> Thoughts?
>>>
>>> Andy
>>>
>>>>
>>>> --- A. Soroka The University of Virginia Library
>>>
>>> Prefixes !!!!!!!!!!!!!1
>>>
>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>>
>>> <test:simpleExample> a ja:MemoryDataset ;
>>> ja:defaultGraph <test:defaultGraphDef> ;
>>> ja:namedGraph <test:namedGraphDef> .
>>>
>>> <test:defaultGraphDef>
>>> a ja:MemoryModel ;
>>> ja:content [ ja:externalContent <file:///some/triples.nt> ] .
>>>
>>> <test:namedGraphDef> a ja:MemoryModel ;
>>> ja:content
>>> [ ja:externalContent <file:///some/other/triples.nt> ] ;
>>> ja:graphName <test:namedGraphExample> .
>>
>
Re: dataset assembler for JENA-624
Posted by Andy Seaborne <an...@apache.org>.
Hi there,
I got ja:data working but ja:graph alluded me. After some code digging,
it seems it is a synonym for ja:defaultGraph but ja:defaultGraph goes to
a full-blown model description.
Tests from files would be good and they can be used in documentation
examples as well.
-----------------------
@prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
<test:simpleExample> a ja:MemoryDataset ;
ja:data <file:data.trig> ;
ja:data <file:data1.ttl> ;
ja:graph [ ja:data <file:data2.ttl> ] ;
ja:graph
[ ja:graphName <http://example/g3> ;
ja:data <file:data3.ttl> ] ;
.
-----------------------
and I tried:
DatasetFactory.assemble("assembler.ttl") ;
OK - wiring not in place yet
then
Model m = RDFDataMgr.loadModel("assembler.ttl") ;
Resource r = m.createResource("test:simpleExample") ;
Dataset ds = (Dataset)new InMemDatasetAssembler().open(r) ;
I was expecting ja:graph to be a direct description - not needing a
ja:MemoryModel description
ja:graph is needed to associate a name with a triples file.
I took a go at an implementation:
---------------------
open( ...) {
...
// Instead of: final Resource defaultGraphDef =
multiValueResource(root, pGraph)
.forEach(r->readGraphDesc(dataset, r));
...
}
// May need better checking for expected Resource/String things.
private void readGraphDesc(Dataset dataset, Resource r) {
String gn = null ;
if ( r.hasProperty(pGraphName)) {
Resource rgn = r.getProperty(pGraphName).getResource() ;
gn = rgn.getURI() ;
}
String dataFn = getAsStringValue(r, data) ;
if ( gn == null )
RDFDataMgr.read(dataset.getDefaultModel(), dataFn);
else
RDFDataMgr.read(dataset.getNamedModel(gn), dataFn);
}
---------------------
Andy
On 02/11/15 16:44, A. Soroka wrote:
> I’ve added “direct data links” in the style Andy outlines below to this JENA-624 PR, with tests.
>
> Feedback eagerly desired!
>
> ---
> A. Soroka
> The University of Virginia Library
>
>> On Oct 30, 2015, at 3:05 PM, Andy Seaborne <an...@apache.org> wrote:
>>
>>> Does this seem like a reasonable approach? It should allow users to
>>> build up their data by whatever means they like, including using
>>> inferencing models to generate assertions, then add them to the
>>> in-memory container.
>>
>> That's an interesting possibility that hadn't occurred to me. It's a copy-in and the implications of that will need to be clear.
>>
>> Just relying on ja:MemoryModel and ja:externalContent for the cases of loading files risks a load-copy though. It could be "optimzied"
>>
>> What about allowing files to be directly pointed to from the assembler for the ja:MemoryDataset?
>>
>> Example:
>>
>> -----------------------------------
>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>
>> <test:simpleExample> a ja:MemoryDataset ;
>> ja:data <file:///some/data.trig> ;
>> ja:data <file:///some/data.ttl> ;
>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>> ja:graph
>> [ ja:graphName <test:namedGraphExample> ;
>> ja:data <file:///some/data3.ttl> ] .
>> -----------------------------------
>>
>> The ability to load trig, NQuads in the dataset:
>>
>> ja:data <file:///some/data.trig> ;
>>
>> If a triples form is given, it loads the default graph. This is what Jena does already. These are both RDFDataMgr.read(dataset, file) ;
>>
>> The ja:graph for specific graphs:
>> Default graph (uniformity):
>>
>> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>>
>> and named graphs
>> ja:graph
>> [ ja:graphName <test:namedGraphExample> ;
>> ja:data <file:///some/data3.ttl> ] .
>>
>> I like the build-copy idea - these also add the core tasks of loading a file into a dataset in a direct way.
>>
>> Thoughts?
>>
>> Andy
>>
>>>
>>> --- A. Soroka The University of Virginia Library
>>
>> Prefixes !!!!!!!!!!!!!1
>>
>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>>
>> <test:simpleExample> a ja:MemoryDataset ;
>> ja:defaultGraph <test:defaultGraphDef> ;
>> ja:namedGraph <test:namedGraphDef> .
>>
>> <test:defaultGraphDef>
>> a ja:MemoryModel ;
>> ja:content [ ja:externalContent <file:///some/triples.nt> ] .
>>
>> <test:namedGraphDef> a ja:MemoryModel ;
>> ja:content
>> [ ja:externalContent <file:///some/other/triples.nt> ] ;
>> ja:graphName <test:namedGraphExample> .
>
Re: dataset assembler for JENA-624
Posted by "A. Soroka" <aj...@virginia.edu>.
I’ve added “direct data links” in the style Andy outlines below to this JENA-624 PR, with tests.
Feedback eagerly desired!
---
A. Soroka
The University of Virginia Library
> On Oct 30, 2015, at 3:05 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> Does this seem like a reasonable approach? It should allow users to
>> build up their data by whatever means they like, including using
>> inferencing models to generate assertions, then add them to the
>> in-memory container.
>
> That's an interesting possibility that hadn't occurred to me. It's a copy-in and the implications of that will need to be clear.
>
> Just relying on ja:MemoryModel and ja:externalContent for the cases of loading files risks a load-copy though. It could be "optimzied"
>
> What about allowing files to be directly pointed to from the assembler for the ja:MemoryDataset?
>
> Example:
>
> -----------------------------------
> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>
> <test:simpleExample> a ja:MemoryDataset ;
> ja:data <file:///some/data.trig> ;
> ja:data <file:///some/data.ttl> ;
> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
> ja:graph
> [ ja:graphName <test:namedGraphExample> ;
> ja:data <file:///some/data3.ttl> ] .
> -----------------------------------
>
> The ability to load trig, NQuads in the dataset:
>
> ja:data <file:///some/data.trig> ;
>
> If a triples form is given, it loads the default graph. This is what Jena does already. These are both RDFDataMgr.read(dataset, file) ;
>
> The ja:graph for specific graphs:
> Default graph (uniformity):
>
> ja:graph [ ja:data <file:///some/data2.ttl> ] ;
>
> and named graphs
> ja:graph
> [ ja:graphName <test:namedGraphExample> ;
> ja:data <file:///some/data3.ttl> ] .
>
> I like the build-copy idea - these also add the core tasks of loading a file into a dataset in a direct way.
>
> Thoughts?
>
> Andy
>
>>
>> --- A. Soroka The University of Virginia Library
>
> Prefixes !!!!!!!!!!!!!1
>
> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
>
> <test:simpleExample> a ja:MemoryDataset ;
> ja:defaultGraph <test:defaultGraphDef> ;
> ja:namedGraph <test:namedGraphDef> .
>
> <test:defaultGraphDef>
> a ja:MemoryModel ;
> ja:content [ ja:externalContent <file:///some/triples.nt> ] .
>
> <test:namedGraphDef> a ja:MemoryModel ;
> ja:content
> [ ja:externalContent <file:///some/other/triples.nt> ] ;
> ja:graphName <test:namedGraphExample> .
Re: dataset assembler for JENA-624
Posted by Andy Seaborne <an...@apache.org>.
> Does this seem like a reasonable approach? It should allow users to
> build up their data by whatever means they like, including using
> inferencing models to generate assertions, then add them to the
> in-memory container.
That's an interesting possibility that hadn't occurred to me. It's a
copy-in and the implications of that will need to be clear.
Just relying on ja:MemoryModel and ja:externalContent for the cases of
loading files risks a load-copy though. It could be "optimzied"
What about allowing files to be directly pointed to from the assembler
for the ja:MemoryDataset?
Example:
-----------------------------------
@prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
<test:simpleExample> a ja:MemoryDataset ;
ja:data <file:///some/data.trig> ;
ja:data <file:///some/data.ttl> ;
ja:graph [ ja:data <file:///some/data2.ttl> ] ;
ja:graph
[ ja:graphName <test:namedGraphExample> ;
ja:data <file:///some/data3.ttl> ] .
-----------------------------------
The ability to load trig, NQuads in the dataset:
ja:data <file:///some/data.trig> ;
If a triples form is given, it loads the default graph. This is what
Jena does already. These are both RDFDataMgr.read(dataset, file) ;
The ja:graph for specific graphs:
Default graph (uniformity):
ja:graph [ ja:data <file:///some/data2.ttl> ] ;
and named graphs
ja:graph
[ ja:graphName <test:namedGraphExample> ;
ja:data <file:///some/data3.ttl> ] .
I like the build-copy idea - these also add the core tasks of loading a
file into a dataset in a direct way.
Thoughts?
Andy
>
> --- A. Soroka The University of Virginia Library
Prefixes !!!!!!!!!!!!!1
@prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
<test:simpleExample> a ja:MemoryDataset ;
ja:defaultGraph <test:defaultGraphDef> ;
ja:namedGraph <test:namedGraphDef> .
<test:defaultGraphDef>
a ja:MemoryModel ;
ja:content [ ja:externalContent <file:///some/triples.nt> ] .
<test:namedGraphDef> a ja:MemoryModel ;
ja:content
[ ja:externalContent <file:///some/other/triples.nt> ] ;
ja:graphName <test:namedGraphExample> .