You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Martynas Jusevicius <ma...@graphity.org> on 2012/03/24 22:31:40 UTC
Controlling blank node IDs
Hey all,
what would be the easiest way to control the generation of blank node IDs?
What I need is to make them globally unique (I'm thinking UUIDs or
smth like that), because that's the approach taken by the triple store
I'm using.
Currently I load several documents into the store with the same
Jena-generated IDs (A0, A1, etc), and they sit nicely in their
separate named graphs. However when triples containing those bnodes
get re-serialized into a single RDF/XML result, the IDs are matching
where they shouldn't be.
Martynas
graphity.org
Re: Controlling blank node IDs
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi Martynas,
since I experienced some of your pains in a different context I want
to share my solution (it might be useful to you or others).
My problem was how to deal with blank nodes in MapReduce jobs or in a
sequence of MapReduce jobs. The solution I found works for N-Triples
and N-Quads formats so it might not be useful in your case, since you
want a single RDF/XML file.
Utils.createParserProfile(...) method creates a ParserProfile:
https://github.com/castagna/tdbloader4/blob/f5363fa49d16a04a362898c1a5084ade620ee81b/src/main/java/org/apache/jena/tdbloader4/Utils.java
The ParserProfile is using a custom LabelToNode object:
https://github.com/castagna/tdbloader4/blob/f5363fa49d16a04a362898c1a5084ade620ee81b/src/main/java/org/apache/jena/tdbloader4/io/MapReduceLabelToNode.java
MapReduceLabelToNode extends LabelToNode and uses a MapReduceAllocator
which implements Allocator<String, Node>.
This is not exactly what you want, but maybe you can do something
very similar. Have a look at LabelToNode and NodeToLabel in ARQ:
https://svn.apache.org/repos/asf/incubator/jena/Jena2/ARQ/tags/jena-arq-2.9.0-incubating/src/main/java/org/openjena/riot/lang/LabelToNode.java
https://svn.apache.org/repos/asf/incubator/jena/Jena2/ARQ/tags/jena-arq-2.9.0-incubating/src/main/java/org/openjena/riot/out/NodeToLabel.java
and how they are used in RIOT.
Paolo
Martynas Jusevicius wrote:
> Hey all,
>
> what would be the easiest way to control the generation of blank node IDs?
> What I need is to make them globally unique (I'm thinking UUIDs or
> smth like that), because that's the approach taken by the triple store
> I'm using.
> Currently I load several documents into the store with the same
> Jena-generated IDs (A0, A1, etc), and they sit nicely in their
> separate named graphs. However when triples containing those bnodes
> get re-serialized into a single RDF/XML result, the IDs are matching
> where they shouldn't be.
>
> Martynas
> graphity.org
Re: Controlling blank node IDs
Posted by Andy Seaborne <an...@apache.org>.
On 27/03/12 22:59, Martynas Jusevicius wrote:
> Andy, would you agree though that such feature (correspondence between
> blank nodes and HTML anchors) would make sense?
>
> However it seems pretty much impossible as you describe it, since I'm
> using RDF/XML as a gateway for all RDF serializations in the wild,
> which then gets transformed with XSLT.
>
> I was thinking maybe I could somehow generate the fragment IDs
> opaquely, without loading the RDF document. XSLT has a generate-id()
> function, but it wouldn't really help here as it works on document
> nodes.
> Maybe I could hash the whole URI and use that as the fragment ID
> (would probably look quite ugly). Hash the hash URI, so to speak?
>
> Martynas
> graphity.org
Blank nodes seem to me to be the wrong weapon of choice. If you want a
permanent identifier, then a URI is better. Even if you can force it
through Jena, then it will be unstable in any other toolkit.
UUIDs are cheap. That's effectively your hash.
If you want recoverability then encode in some way or
<http://host/path/doc#anchor> ?
Andy
Re: Controlling blank node IDs
Posted by Martynas Jusevicius <ma...@graphity.org>.
Andy, would you agree though that such feature (correspondence between
blank nodes and HTML anchors) would make sense?
However it seems pretty much impossible as you describe it, since I'm
using RDF/XML as a gateway for all RDF serializations in the wild,
which then gets transformed with XSLT.
I was thinking maybe I could somehow generate the fragment IDs
opaquely, without loading the RDF document. XSLT has a generate-id()
function, but it wouldn't really help here as it works on document
nodes.
Maybe I could hash the whole URI and use that as the fragment ID
(would probably look quite ugly). Hash the hash URI, so to speak?
Martynas
graphity.org
On Mon, Mar 26, 2012 at 10:54 PM, Andy Seaborne <an...@apache.org> wrote:
> On 25/03/12 22:01, Martynas Jusevicius wrote:
>>
>> I mean when serializing RDF/XML.
>>
>> On Sun, Mar 25, 2012 at 10:59 PM, Martynas Jusevicius
>> <ma...@graphity.org> wrote:
>>>
>>> Andy, is it possible however to have the original bnode IDs carried
>>> over to the output, or are they already lost when re-labelled with
>>> internal IDs?
>>>
>>> I was thinking about producing anchors (HTML #fragments) for that part
>>> of HTML which renders an according #bnode in RDF. But I realized I
>>> can't do it if I can't get the original IDs.
>
>
> They get relabelled - AFAIK this isn't changeable in ARP.
>
> On reading, the labels are treated as file-local and there is a mapping from
> file label to internal Ids but the file usage isn't recorded
>
> For RIOT, you can change the policy at he very lowest levels. But you end
> up with non-RDF semantics.
>
> Andy
>
>
Re: Controlling blank node IDs
Posted by Andy Seaborne <an...@apache.org>.
On 25/03/12 22:01, Martynas Jusevicius wrote:
> I mean when serializing RDF/XML.
>
> On Sun, Mar 25, 2012 at 10:59 PM, Martynas Jusevicius
> <ma...@graphity.org> wrote:
>> Andy, is it possible however to have the original bnode IDs carried
>> over to the output, or are they already lost when re-labelled with
>> internal IDs?
>>
>> I was thinking about producing anchors (HTML #fragments) for that part
>> of HTML which renders an according #bnode in RDF. But I realized I
>> can't do it if I can't get the original IDs.
They get relabelled - AFAIK this isn't changeable in ARP.
On reading, the labels are treated as file-local and there is a mapping
from file label to internal Ids but the file usage isn't recorded
For RIOT, you can change the policy at he very lowest levels. But you
end up with non-RDF semantics.
Andy
Re: Controlling blank node IDs
Posted by Martynas Jusevicius <ma...@graphity.org>.
I mean when serializing RDF/XML.
On Sun, Mar 25, 2012 at 10:59 PM, Martynas Jusevicius
<ma...@graphity.org> wrote:
> Andy, is it possible however to have the original bnode IDs carried
> over to the output, or are they already lost when re-labelled with
> internal IDs?
>
> I was thinking about producing anchors (HTML #fragments) for that part
> of HTML which renders an according #bnode in RDF. But I realized I
> can't do it if I can't get the original IDs.
>
> Martynas
> graphity.org
>
> On Sun, Mar 25, 2012 at 7:47 PM, Andy Seaborne <an...@apache.org> wrote:
>> On 24/03/12 23:55, Martynas Jusevicius wrote:
>>>
>>> Hey,
>>>
>>> I'll forward it to them :)
>>>
>>> Seems like I won't find an easy way out of this one. I'm using the
>>> DatasetGraphAccessorHTTP class which internally serializes into
>>> RDF/XML, so I guess I'll have to override that in order to try out the
>>> N-Triples option you mention.
>>
>>
>> yes, but only because RDF/XML is the only standard.
>>
>> N-triples will be standardised by RDF-WG (as well as Turtle and hopefully
>> TriG and NQuads if named graph debates pause enough to do the somewhat
>> mechanical work on named graph syntaxes).
>>
>> Andy
>>
>>
>>
>>
>>>
>>> Martynas
>>> graphity.org
>>>
>>> On Sun, Mar 25, 2012 at 12:13 AM, Andy Seaborne<an...@apache.org> wrote:
>>>>
>>>> On 24/03/12 22:47, Martynas Jusevicius wrote:
>>>>>
>>>>>
>>>>> Hey Andy,
>>>>>
>>>>> well yeah, I mean the serialized form. But that is what gets inserted
>>>>> (currently POSTed as RDF/XML over the graph store protocol) and
>>>>> becomes the bnode IDs in the store.
>>>>> The store in question is the cloud service http://dydra.com.
>>>>>
>>>>> Martynas
>>>>
>>>>
>>>>
>>>> Naughty Dydra, that's not what the specs say. Labels are scoped to the
>>>> file
>>>> as read -- i.e. you can conclude nothing about whether two files (or one
>>>> file read twice) refer to the same bNode from the serialization label
>>>> alone.
>>>>
>>>> Nasty things will happen because some bNodes are implicit and some
>>>> explicitly labels. I can see why they do it though. c.f. Talis stores
>>>> which convert bNodes to URIs.
>>>>
>>>> Try N-triples, the normal output uses the internal unique label to
>>>> generate
>>>> the _: name. if need be convert with RIOT
>>>> "riot D.rdf> "D.nt".
>>>>
>>>> Else you might try rewriting the RDF/XML output and do string
>>>> substitution.
>>>> rdf:nodeID="A0" is quite easy to spot.
>>>>
>>>> (The RDF-WG is fixing this with a skolemization scheme.)
>>>>
>>>> Andy
>>>>
>>
Re: Controlling blank node IDs
Posted by Martynas Jusevicius <ma...@graphity.org>.
Andy, is it possible however to have the original bnode IDs carried
over to the output, or are they already lost when re-labelled with
internal IDs?
I was thinking about producing anchors (HTML #fragments) for that part
of HTML which renders an according #bnode in RDF. But I realized I
can't do it if I can't get the original IDs.
Martynas
graphity.org
On Sun, Mar 25, 2012 at 7:47 PM, Andy Seaborne <an...@apache.org> wrote:
> On 24/03/12 23:55, Martynas Jusevicius wrote:
>>
>> Hey,
>>
>> I'll forward it to them :)
>>
>> Seems like I won't find an easy way out of this one. I'm using the
>> DatasetGraphAccessorHTTP class which internally serializes into
>> RDF/XML, so I guess I'll have to override that in order to try out the
>> N-Triples option you mention.
>
>
> yes, but only because RDF/XML is the only standard.
>
> N-triples will be standardised by RDF-WG (as well as Turtle and hopefully
> TriG and NQuads if named graph debates pause enough to do the somewhat
> mechanical work on named graph syntaxes).
>
> Andy
>
>
>
>
>>
>> Martynas
>> graphity.org
>>
>> On Sun, Mar 25, 2012 at 12:13 AM, Andy Seaborne<an...@apache.org> wrote:
>>>
>>> On 24/03/12 22:47, Martynas Jusevicius wrote:
>>>>
>>>>
>>>> Hey Andy,
>>>>
>>>> well yeah, I mean the serialized form. But that is what gets inserted
>>>> (currently POSTed as RDF/XML over the graph store protocol) and
>>>> becomes the bnode IDs in the store.
>>>> The store in question is the cloud service http://dydra.com.
>>>>
>>>> Martynas
>>>
>>>
>>>
>>> Naughty Dydra, that's not what the specs say. Labels are scoped to the
>>> file
>>> as read -- i.e. you can conclude nothing about whether two files (or one
>>> file read twice) refer to the same bNode from the serialization label
>>> alone.
>>>
>>> Nasty things will happen because some bNodes are implicit and some
>>> explicitly labels. I can see why they do it though. c.f. Talis stores
>>> which convert bNodes to URIs.
>>>
>>> Try N-triples, the normal output uses the internal unique label to
>>> generate
>>> the _: name. if need be convert with RIOT
>>> "riot D.rdf> "D.nt".
>>>
>>> Else you might try rewriting the RDF/XML output and do string
>>> substitution.
>>> rdf:nodeID="A0" is quite easy to spot.
>>>
>>> (The RDF-WG is fixing this with a skolemization scheme.)
>>>
>>> Andy
>>>
>
Re: Controlling blank node IDs
Posted by Andy Seaborne <an...@apache.org>.
On 24/03/12 23:55, Martynas Jusevicius wrote:
> Hey,
>
> I'll forward it to them :)
>
> Seems like I won't find an easy way out of this one. I'm using the
> DatasetGraphAccessorHTTP class which internally serializes into
> RDF/XML, so I guess I'll have to override that in order to try out the
> N-Triples option you mention.
yes, but only because RDF/XML is the only standard.
N-triples will be standardised by RDF-WG (as well as Turtle and
hopefully TriG and NQuads if named graph debates pause enough to do the
somewhat mechanical work on named graph syntaxes).
Andy
>
> Martynas
> graphity.org
>
> On Sun, Mar 25, 2012 at 12:13 AM, Andy Seaborne<an...@apache.org> wrote:
>> On 24/03/12 22:47, Martynas Jusevicius wrote:
>>>
>>> Hey Andy,
>>>
>>> well yeah, I mean the serialized form. But that is what gets inserted
>>> (currently POSTed as RDF/XML over the graph store protocol) and
>>> becomes the bnode IDs in the store.
>>> The store in question is the cloud service http://dydra.com.
>>>
>>> Martynas
>>
>>
>> Naughty Dydra, that's not what the specs say. Labels are scoped to the file
>> as read -- i.e. you can conclude nothing about whether two files (or one
>> file read twice) refer to the same bNode from the serialization label alone.
>>
>> Nasty things will happen because some bNodes are implicit and some
>> explicitly labels. I can see why they do it though. c.f. Talis stores
>> which convert bNodes to URIs.
>>
>> Try N-triples, the normal output uses the internal unique label to generate
>> the _: name. if need be convert with RIOT
>> "riot D.rdf> "D.nt".
>>
>> Else you might try rewriting the RDF/XML output and do string substitution.
>> rdf:nodeID="A0" is quite easy to spot.
>>
>> (The RDF-WG is fixing this with a skolemization scheme.)
>>
>> Andy
>>
Re: Controlling blank node IDs
Posted by Martynas Jusevicius <ma...@graphity.org>.
Hey,
I'll forward it to them :)
Seems like I won't find an easy way out of this one. I'm using the
DatasetGraphAccessorHTTP class which internally serializes into
RDF/XML, so I guess I'll have to override that in order to try out the
N-Triples option you mention.
Martynas
graphity.org
On Sun, Mar 25, 2012 at 12:13 AM, Andy Seaborne <an...@apache.org> wrote:
> On 24/03/12 22:47, Martynas Jusevicius wrote:
>>
>> Hey Andy,
>>
>> well yeah, I mean the serialized form. But that is what gets inserted
>> (currently POSTed as RDF/XML over the graph store protocol) and
>> becomes the bnode IDs in the store.
>> The store in question is the cloud service http://dydra.com.
>>
>> Martynas
>
>
> Naughty Dydra, that's not what the specs say. Labels are scoped to the file
> as read -- i.e. you can conclude nothing about whether two files (or one
> file read twice) refer to the same bNode from the serialization label alone.
>
> Nasty things will happen because some bNodes are implicit and some
> explicitly labels. I can see why they do it though. c.f. Talis stores
> which convert bNodes to URIs.
>
> Try N-triples, the normal output uses the internal unique label to generate
> the _: name. if need be convert with RIOT
> "riot D.rdf > "D.nt".
>
> Else you might try rewriting the RDF/XML output and do string substitution.
> rdf:nodeID="A0" is quite easy to spot.
>
> (The RDF-WG is fixing this with a skolemization scheme.)
>
> Andy
>
Re: Controlling blank node IDs
Posted by Andy Seaborne <an...@apache.org>.
On 24/03/12 22:47, Martynas Jusevicius wrote:
> Hey Andy,
>
> well yeah, I mean the serialized form. But that is what gets inserted
> (currently POSTed as RDF/XML over the graph store protocol) and
> becomes the bnode IDs in the store.
> The store in question is the cloud service http://dydra.com.
>
> Martynas
Naughty Dydra, that's not what the specs say. Labels are scoped to the
file as read -- i.e. you can conclude nothing about whether two files
(or one file read twice) refer to the same bNode from the serialization
label alone.
Nasty things will happen because some bNodes are implicit and some
explicitly labels. I can see why they do it though. c.f. Talis stores
which convert bNodes to URIs.
Try N-triples, the normal output uses the internal unique label to
generate the _: name. if need be convert with RIOT
"riot D.rdf > "D.nt".
Else you might try rewriting the RDF/XML output and do string
substitution. rdf:nodeID="A0" is quite easy to spot.
(The RDF-WG is fixing this with a skolemization scheme.)
Andy
Re: Controlling blank node IDs
Posted by Martynas Jusevicius <ma...@graphity.org>.
Hey Andy,
well yeah, I mean the serialized form. But that is what gets inserted
(currently POSTed as RDF/XML over the graph store protocol) and
becomes the bnode IDs in the store.
The store in question is the cloud service http://dydra.com.
Martynas
On Sat, Mar 24, 2012 at 11:38 PM, Andy Seaborne <an...@apache.org> wrote:
> On 24/03/12 21:31, Martynas Jusevicius wrote:
>>
>> Hey all,
>>
>> what would be the easiest way to control the generation of blank node IDs?
>> What I need is to make them globally unique (I'm thinking UUIDs or
>> smth like that), because that's the approach taken by the triple store
>> I'm using.
>
>
> Which one?
>
>
>> Currently I load several documents into the store with the same
>> Jena-generated IDs (A0, A1, etc),
>
>
> Jena IDs should not be of the form A0, A1, ... -- is this their serialized
> output (which is not the internal ID).
>
>
>> and they sit nicely in their
>> separate named graphs. However when triples containing those bnodes
>> get re-serialized into a single RDF/XML result, the IDs are matching
>> where they shouldn't be.
>
>
> Concrete example? Every reading of a file should produce different bNodes
> (unless you have reached into the internals of jena and changed the
> allocation scheme.)
>
> (The RIOT parsers are configurable but not exposed.)
>
> Andy
>
>>
>> Martynas
>> graphity.org
>
>
Re: Controlling blank node IDs
Posted by Andy Seaborne <an...@apache.org>.
On 24/03/12 21:31, Martynas Jusevicius wrote:
> Hey all,
>
> what would be the easiest way to control the generation of blank node IDs?
> What I need is to make them globally unique (I'm thinking UUIDs or
> smth like that), because that's the approach taken by the triple store
> I'm using.
Which one?
> Currently I load several documents into the store with the same
> Jena-generated IDs (A0, A1, etc),
Jena IDs should not be of the form A0, A1, ... -- is this their
serialized output (which is not the internal ID).
> and they sit nicely in their
> separate named graphs. However when triples containing those bnodes
> get re-serialized into a single RDF/XML result, the IDs are matching
> where they shouldn't be.
Concrete example? Every reading of a file should produce different
bNodes (unless you have reached into the internals of jena and changed
the allocation scheme.)
(The RIOT parsers are configurable but not exposed.)
Andy
>
> Martynas
> graphity.org