You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Laura Morales <la...@mail.com> on 2018/04/27 11:10:20 UTC

RIOT filter to remove bad triples

Here's the problem: I'm downloading a few .nt from the Internet and they contain some bad triples. "Bad" means that they are syntactically correct but logically incorrect; for example they contain triples with relative URLs instead of absolute, or maybe they contain an empty subject like this <>. I don't know if it's possible to ask RIOT to treat these as Warnings instead of Errors, but if I use RIOT --validate it only reports then as errors.
Before you say anything, I know that the original sources should be fixed and that it's not RIOT's job to fix any errors. But I don't really have control over those graphs, so I was wondering if it's possible to use RIOT as some sort of filter like this

1. read next triple
2. if logically correct, append to output
   if logically incorrect, do not append to output

Re: RIOT filter to remove bad triples

Posted by Lorenz Buehmann <bu...@informatik.uni-leipzig.de>.
AFAIK, that doesn't work for all kind of errors. Some are categorized as
"fatal", in which case the parser stops immediately. In any case you
could write your own parser that skips those. IIRC, there were at least
two discussion here on the mailing list some years ago how to keep track
of those bad triples [1] . Don't remember of the second thread.

There are different error handlers, probably only configurable via Java API:

ErrorHandlerFactory.errorHandlerWarn + ParserProfileStd +
RiotParsers.createIteratorNTriples


Clearly things might have been changed (I know that Rob Vesse did
something for the CLI). Probably there is a simpler way, but that's what
I remember and I don't have a computer at hand right now.


Cheers,
Lorenz


[1]
https://mail-archives.apache.org/mod_mbox/jena-users/201610.mbox/%3c580F4A74.6080002@helsinki.fi%3e


On 27.04.2018 13:49, Jean-Marc Vanel wrote:
> Hi Laura,
>
> I would do this:
>
>    - consider the .nt as a ttl file
>    - run riot with a base URI
>    - remove the bad lines with grep -v
>
> Example:
> cat relative.ttl
> <rel> <p> "v" .
> java -$CP riotcmd.riot --base=http://nogood.net/
> /home/jmv/data/TESTS/relative.ttl
> <http://nogood.net/rel> <http://nogood.net/p> "v" .
>
>
> 2018-04-27 13:10 GMT+02:00 Laura Morales <la...@mail.com>:
>
>> Here's the problem: I'm downloading a few .nt from the Internet and they
>> contain some bad triples. "Bad" means that they are syntactically correct
>> but logically incorrect; for example they contain triples with relative
>> URLs instead of absolute, or maybe they contain an empty subject like this
>> <>. I don't know if it's possible to ask RIOT to treat these as Warnings
>> instead of Errors, but if I use RIOT --validate it only reports then as
>> errors.
>> Before you say anything, I know that the original sources should be fixed
>> and that it's not RIOT's job to fix any errors. But I don't really have
>> control over those graphs, so I was wondering if it's possible to use RIOT
>> as some sort of filter like this
>>
>> 1. read next triple
>> 2. if logically correct, append to output
>>    if logically incorrect, do not append to output
>>
>
>



Re: RIOT filter to remove bad triples

Posted by Laura Morales <la...@mail.com>.
Thank you, sounds like a good solution!

 
 

Sent: Friday, April 27, 2018 at 1:49 PM
From: "Jean-Marc Vanel" <je...@gmail.com>
To: "Jena users" <us...@jena.apache.org>
Subject: Re: RIOT filter to remove bad triples
Hi Laura,

I would do this:

- consider the .nt as a ttl file
- run riot with a base URI
- remove the bad lines with grep -v

Example:
cat relative.ttl
<rel> <p> "v" .
java -$CP riotcmd.riot --base=http://nogood.net/
/home/jmv/data/TESTS/relative.ttl
<http://nogood.net/rel[http://nogood.net/rel]> <http://nogood.net/p[http://nogood.net/p]> "v" .


2018-04-27 13:10 GMT+02:00 Laura Morales <la...@mail.com>:

> Here's the problem: I'm downloading a few .nt from the Internet and they
> contain some bad triples. "Bad" means that they are syntactically correct
> but logically incorrect; for example they contain triples with relative
> URLs instead of absolute, or maybe they contain an empty subject like this
> <>. I don't know if it's possible to ask RIOT to treat these as Warnings
> instead of Errors, but if I use RIOT --validate it only reports then as
> errors.
> Before you say anything, I know that the original sources should be fixed
> and that it's not RIOT's job to fix any errors. But I don't really have
> control over those graphs, so I was wondering if it's possible to use RIOT
> as some sort of filter like this
>
> 1. read next triple
> 2. if logically correct, append to output
> if logically incorrect, do not append to output
>



--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject[http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject]
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me[http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me]>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui

Re: RIOT filter to remove bad triples

Posted by Jean-Marc Vanel <je...@gmail.com>.
Hi Laura,

I would do this:

   - consider the .nt as a ttl file
   - run riot with a base URI
   - remove the bad lines with grep -v

Example:
cat relative.ttl
<rel> <p> "v" .
java -$CP riotcmd.riot --base=http://nogood.net/
/home/jmv/data/TESTS/relative.ttl
<http://nogood.net/rel> <http://nogood.net/p> "v" .


2018-04-27 13:10 GMT+02:00 Laura Morales <la...@mail.com>:

> Here's the problem: I'm downloading a few .nt from the Internet and they
> contain some bad triples. "Bad" means that they are syntactically correct
> but logically incorrect; for example they contain triples with relative
> URLs instead of absolute, or maybe they contain an empty subject like this
> <>. I don't know if it's possible to ask RIOT to treat these as Warnings
> instead of Errors, but if I use RIOT --validate it only reports then as
> errors.
> Before you say anything, I know that the original sources should be fixed
> and that it's not RIOT's job to fix any errors. But I don't really have
> control over those graphs, so I was wondering if it's possible to use RIOT
> as some sort of filter like this
>
> 1. read next triple
> 2. if logically correct, append to output
>    if logically incorrect, do not append to output
>



-- 
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui