You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Laura Morales <la...@mail.com> on 2017/12/14 06:19:41 UTC
TDB multiple locations
Hi all,
reading the TDB assembler doc [1], I was wondering how easy/difficult it would be to support multiple ja:graph for a ja:namedGraph, so that a large graph such as Wikidata could potentially be split into smaller graphs.
Alternatively, since SPARQL supports multiple FROM instructions, there could be an option to configure "aliases" so that if the user writes "FROM <wikidata>" this will be translated to "FROM <wikidata-store1> <wikidata-store2> ..." where each <wikidata-storeN> is its own ja:namedGraph.
I think this could be equivalent to tdb:unionDefaultGraph, but instead of being a union among *all* graphs, only a union among a limited selection of graphs. It kinda sounds easy in theory, since it seems like a particular case of tdb:unionDefaultGraph, but... I don't know in practice if this would be possible?
[1] https://jena.apache.org/documentation/tdb/assembler.html
Re: TDB multiple locations
Posted by Laura Morales <la...@mail.com>.
> Dick's Mosiac technology might be a better approach or not, depending on your use cases and comfort with the cutting edge.
Mosaic is not merged yet, is it?
SERVICE could work, but it seems like an inelegant hack since I'll have to define a new service for every graph that I wish to partition. It still seems to me that using a "partial union" would make more sense...
Re: TDB multiple locations
Posted by ajs6f <aj...@apache.org>.
You can use the SPARQL federate query mechanism for something like this:
https://www.w3.org/TR/sparql11-federated-query/
You would federate over a group of Fuseki instances or a single Fuseki instance with multiple TDB datasets loaded. Keep in mind that you will have to pay attention to how to shard triples into different stores to have performant query, but you would have to do that for any way of partitioning the data.
Dick's Mosiac technology might be a better approach or not, depending on your use cases and comfort with the cutting edge.
ajs6f
> On Dec 14, 2017, at 1:19 AM, Laura Morales <la...@mail.com> wrote:
>
> Hi all,
>
> reading the TDB assembler doc [1], I was wondering how easy/difficult it would be to support multiple ja:graph for a ja:namedGraph, so that a large graph such as Wikidata could potentially be split into smaller graphs.
> Alternatively, since SPARQL supports multiple FROM instructions, there could be an option to configure "aliases" so that if the user writes "FROM <wikidata>" this will be translated to "FROM <wikidata-store1> <wikidata-store2> ..." where each <wikidata-storeN> is its own ja:namedGraph.
>
> I think this could be equivalent to tdb:unionDefaultGraph, but instead of being a union among *all* graphs, only a union among a limited selection of graphs. It kinda sounds easy in theory, since it seems like a particular case of tdb:unionDefaultGraph, but... I don't know in practice if this would be possible?
>
> [1] https://jena.apache.org/documentation/tdb/assembler.html
Re: TDB multiple locations
Posted by "Andrew U. Frank" <fr...@geoinfo.tuwien.ac.at>.
you can achieve this with a file in your fuseki source directory
../run/configuration. i have found on the web an example which i used
from
https://github.com/jfmunozf/Jena-Fuseki-Reasoner-Inference/wiki/Configuring-Apache-Jena-Fuseki-2.4.1-inference-and-reasoning-support-using-SPARQL-1.1:-Jena-inference-rules,-RDFS-Entailment-Regimes-and-OWL-reasoning
adapt the filenames to your names!
@prefix : <http://base/#> .
@prefix tdb: <http://jena.hpl.hp.com/2008/tdb#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix fuseki: <http://jena.apache.org/fuseki#> .
#This line is a comment
:service1 a fuseki:Service ;
fuseki:dataset :dataset ;
fuseki:name "ElQuijote" ;
fuseki:serviceQuery "query" , "sparql" ;
fuseki:serviceReadGraphStore "get" ;
fuseki:serviceReadWriteGraphStore "data" ;
fuseki:serviceUpdate "update" ;
fuseki:serviceUpload "upload" .
:dataset rdf:type ja:RDFDataset ;
rdfs:label "ElQuijote" ;
ja:defaultGraph
[ rdfs:label "ElQuijote" ;
a ja:InfModel ;
#Reference to model.ttl file
ja:content [ja:externalContent <.../rdfsOntologyExample/model.ttl> ] ;
#Reference to data.ttl file
ja:content [ja:externalContent <.../rdfsOntologyExample/data.ttl> ] ;
#Disable OWL-based reasoner
ja:reasoner [ja:reasonerURL
<http://jena.hpl.hp.com/2003/OWLFBRuleReasoner> ] ;
#Disable RDFS-based reasoner
# ja:reasoner [ja:reasonerURL
<http://jena.hpl.hp.com/2003/RDFSExptRuleReasoner>] ;
#Enable Jena Rules-based reasoner and we point the location of
myrules.rules file
# ja:reasoner [
# ja:reasonerURL <http://jena.hpl.hp.com/2003/GenericRuleReasoner> ;
# ja:rulesFrom
</home/frank/Desktop/rdfsOntologyExample/myrules.rules> ;
# ] ;
] ;
.
On 12/14/2017 10:21 AM, Robert Nielsen wrote:
> Is it possible to start Fuseki with initial contents from a file (an OWL
> ontology, not an existing TDB), and then allow updates?
>
> When I start Fuseki with the following parameters:
>
> ./fuseki-server --file /Ontologies/MyOntology.owl --update /MyFuseki
>
> I get a message that the the resource /MyFuseki is running in read-only
> mode. It appears the --update parameter does nothing. But there is no
> message that says the parameter is ignored (and why). I can, of course,
> start the Fuseki server and then load the ontology file ... but it seems
> like I should be able to do it in one step. What am I missing?
>
> Running Apache Jena Fuseki 3.5.0 with Java 1.8.0_144 on Mac OS X 10.12.6
> x86_64.
>
> Robert Nielsen
>
--
em.o.Univ.Prof. Dr. sc.techn. Dr. h.c. Andrew U. Frank
+43 1 58801 12710 direct
Geoinformation, TU Wien +43 1 58801 12700 office
Gusshausstr. 27-29 +43 1 55801 12799 fax
1040 Wien Austria +43 676 419 25 72 mobil
Re: TDB multiple locations
Posted by Andy Seaborne <an...@apache.org>.
On 14/12/17 15:32, ajs6f wrote:
> Please start a new thread for a new topic.
Yes - please, a new topic or a JIRA ticket.
Something is wrong (I have checked and it as been for several versions.)
Andy
>
> Did you actually try an update? Did it work?
>
> ajs6f
>
>> On Dec 14, 2017, at 10:21 AM, Robert Nielsen <rn...@us.ibm.com> wrote:
>>
>>
>> Is it possible to start Fuseki with initial contents from a file (an OWL
>> ontology, not an existing TDB), and then allow updates?
>>
>> When I start Fuseki with the following parameters:
>>
>> ./fuseki-server --file /Ontologies/MyOntology.owl --update /MyFuseki
>>
>> I get a message that the the resource /MyFuseki is running in read-only
>> mode. It appears the --update parameter does nothing. But there is no
>> message that says the parameter is ignored (and why). I can, of course,
>> start the Fuseki server and then load the ontology file ... but it seems
>> like I should be able to do it in one step. What am I missing?
>>
>> Running Apache Jena Fuseki 3.5.0 with Java 1.8.0_144 on Mac OS X 10.12.6
>> x86_64.
>>
>> Robert Nielsen
>
Re: TDB multiple locations
Posted by ajs6f <aj...@apache.org>.
Please start a new thread for a new topic.
Did you actually try an update? Did it work?
ajs6f
> On Dec 14, 2017, at 10:21 AM, Robert Nielsen <rn...@us.ibm.com> wrote:
>
>
> Is it possible to start Fuseki with initial contents from a file (an OWL
> ontology, not an existing TDB), and then allow updates?
>
> When I start Fuseki with the following parameters:
>
> ./fuseki-server --file /Ontologies/MyOntology.owl --update /MyFuseki
>
> I get a message that the the resource /MyFuseki is running in read-only
> mode. It appears the --update parameter does nothing. But there is no
> message that says the parameter is ignored (and why). I can, of course,
> start the Fuseki server and then load the ontology file ... but it seems
> like I should be able to do it in one step. What am I missing?
>
> Running Apache Jena Fuseki 3.5.0 with Java 1.8.0_144 on Mac OS X 10.12.6
> x86_64.
>
> Robert Nielsen
Re: TDB multiple locations
Posted by Robert Nielsen <rn...@us.ibm.com>.
Is it possible to start Fuseki with initial contents from a file (an OWL
ontology, not an existing TDB), and then allow updates?
When I start Fuseki with the following parameters:
./fuseki-server --file /Ontologies/MyOntology.owl --update /MyFuseki
I get a message that the the resource /MyFuseki is running in read-only
mode. It appears the --update parameter does nothing. But there is no
message that says the parameter is ignored (and why). I can, of course,
start the Fuseki server and then load the ontology file ... but it seems
like I should be able to do it in one step. What am I missing?
Running Apache Jena Fuseki 3.5.0 with Java 1.8.0_144 on Mac OS X 10.12.6
x86_64.
Robert Nielsen
Re: TDB multiple locations
Posted by Laura Morales <la...@mail.com>.
> Have you seen the dynamic dataset feature?
Yes however how this works is with multiple FROM clauses, for example
SELECT ...
FROM <alice-foaf>
FROM <bob-foaf>
FROM <john-foaf>
FROM <mike-foaf>
...
which is very inconvenient if I've split my graph in multiple stores. For instance, wikidata would become
SELECT ...
FROM <wikidata-tdb1>
FROM <wikidata-tdb2>
FROM <wikidata-tdb3>
FROM <wikidata-tdb4>
...
what I was saying instead, is to define some kind of alias such that I can use
SELECT ... FROM <wikidata>
and internally Fuseki knows that I'm refering to <wikidata-tdb1> + <wikidata-tdb2> + <wikidata-tdb3> + <wikidata-tdb4> + ...
this would also be useful when using
SELECT ...
WHERE {
GRAPH <wikidata> {
}
}
Either this, or a way to declare a graph with multiple locations.
Re: TDB multiple locations
Posted by Andy Seaborne <an...@apache.org>.
On 14/12/17 06:19, Laura Morales wrote:
> Hi all,
>
> reading the TDB assembler doc [1], I was wondering how easy/difficult it would be to support multiple ja:graph for a ja:namedGraph,
That's not what ja:graph is for - it is the graph, and ja:namedGraph
adds the name.
How the graph is formed is down to the part of the assembler that
ja:graph points to. I don't know of anything for partial graphs - there
are some hard parts to that.
It you split a dataset across different storage units, query will slower
because the direct execution of TDB does not apply.
> so that a large graph such as Wikidata could potentially be split into smaller graphs.
> Alternatively, since SPARQL supports multiple FROM instructions, there could be an option to configure "aliases" so that if the user writes "FROM <wikidata>" this will be translated to "FROM <wikidata-store1> <wikidata-store2> ..." where each <wikidata-storeN> is its own ja:namedGraph.
>
> I think this could be equivalent to tdb:unionDefaultGraph, but instead of being a union among *all* graphs, only a union among a limited selection of graphs. It kinda sounds easy in theory, since it seems like a particular case of tdb:unionDefaultGraph, but... I don't know in practice if this would be possible?
tdb:unionDefaultGraph is quite specific for union and quite efficient.
Have you seen the dynamic dataset feature?
Arbitrary choices of graph mean tdb:unionDefaultGraph does not apply and
a more general, and slower, execution happens.
Andy
>
> [1] https://jena.apache.org/documentation/tdb/assembler.html
>