You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Abhishek Shivkumar <ab...@gmail.com> on 2012/12/27 19:42:40 UTC
Need help in loading a large RDF file into JENA TDB using TDBLoader
Hi,
I am trying to load a large (55 GB!) rdf file into JENA TDB for sparql
querying later. Here is a snapshot of the file at the end of this email:
When I am using TDBLoader from command line using the following command:
*c:\JENA\apache-jena-2.7.4\apache-jena-2.7.4\bat>tdbloader.bat -loc test
"C:\freebase-rdf-2012-12-09-00-00"*
I get this error:
*23:40:30 INFO loader :: -- Start triples data phase*
*23:40:30 INFO loader :: ** Load empty triples table*
*23:40:30 INFO loader :: -- Start quads data phase*
*23:40:30 INFO loader :: ** Load empty quads table*
*23:40:30 INFO loader :: Load: C:\Users\IBM_ADMIN\My
Documents\dow*
*n\freebase-rdf-2012-12-09-00-00\freebase-rdf-2012-12-09-00-00 --
2012/12/27 23:4*
*0:30 IST*
*23:40:30 ERROR riot :: [line: 1, col: 1 ] Expected BNode
or IRI:*
* Got: [DIRECTIVE:prefix]*
*Exception in thread "main" org.openjena.riot.RiotException: [line: 1, col:
1 ] E*
*xpected BNode or IRI: Got: [DIRECTIVE:prefix]*
* at
org.openjena.riot.ErrorHandlerFactory$ErrorHandlerStd.fatal(ErrorHand*
*lerFactory.java:130)*
* at
org.openjena.riot.lang.LangEngine.raiseException(LangEngine.java:169)*
*
*
* at
org.openjena.riot.lang.LangEngine.exceptionDirect(LangEngine.java:162*
*)*
* at org.openjena.riot.lang.LangEngine.exception(LangEngine.java:155)
*
* at
org.openjena.riot.lang.LangNTuple.checkIRIOrBNode(LangNTuple.java:107*
*)*
* at org.openjena.riot.lang.LangNQuads.parseOne(LangNQuads.java:84)*
* at org.openjena.riot.lang.LangNQuads.parseOne(LangNQuads.java:34)*
* at org.openjena.riot.lang.LangNTuple.runParser(LangNTuple.java:69)*
* at org.openjena.riot.lang.LangBase.parse(LangBase.java:43)*
* at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:134)*
* at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:121)*
* at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:107)*
* at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadQuads$(BulkLoader*
*.java:160)*
* at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadDataset(BulkLoade*
*r.java:121)*
* at com.hp.hpl.jena.tdb.TDBLoader.loadDataset$(TDBLoader.java:283)*
* at com.hp.hpl.jena.tdb.TDBLoader.loadDataset(TDBLoader.java:196)*
* at com.hp.hpl.jena.tdb.TDBLoader.load(TDBLoader.java:75)*
* at tdb.tdbloader.loadQuads(tdbloader.java:163)*
* at tdb.tdbloader.exec(tdbloader.java:122)*
* at arq.cmdline.CmdMain.mainMethod(CmdMain.java:101)*
* at arq.cmdline.CmdMain.mainRun(CmdMain.java:63)*
* at arq.cmdline.CmdMain.mainRun(CmdMain.java:50)*
* at tdb.tdbloader.main(tdbloader.java:53)*
I need help in understanding this error and how to solve it. Is there a
problem with the input file?
@prefix ns: <http://rdf.freebase.com/ns/>.
@prefix key: <http://rdf.freebase.com/key/>.
@prefix owl: <http://www.w3.org/2002/07/owl#>.
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
ns:m.012rkqx ns:type.object.type ns:common.topic.
ns:m.012rkqx ns:type.object.name "High Fidelity"@en.
ns:m.012rkqx ns:type.object.type ns:music.single.
ns:m.012rkqx ns:type.object.key ns:authority.musicbrainz.name.TRACK3987054.
ns:m.012rkqx ns:type.object.type ns:music.recording.
ns:m.012rkqx key:authority.musicbrainz "258c45bd-4437-4580-8988-b3f3be975f9c".
ns:m.012rkqx key:authority.musicbrainz.name "TRACK3987054".
ns:m.012rkqx rdf:label "High Fidelity"@en.
ns:m.012rkqx rdf:type ns:common.topic.
ns:m.012rkqx rdf:type ns:music.single.
ns:m.012rkqx rdf:type ns:music.recording.
Need help in loading a large RDF file into JENA TDB using TDBLoader
Posted by Abhishek Shivkumar <ab...@gmail.com>.
Hi,
I am trying to load a large (55 GB!) rdf file into JENA TDB for sparql
querying later. Here is a snapshot of the file at the end of this email:
When I am using TDBLoader from command line using the following command:
*c:\JENA\apache-jena-2.7.4\apache-jena-2.7.4\bat>tdbloader.bat -loc test
"C:\freebase-rdf-2012-12-09-00-00"*
I get this error:
*23:40:30 INFO loader :: -- Start triples data phase*
*23:40:30 INFO loader :: ** Load empty triples table*
*23:40:30 INFO loader :: -- Start quads data phase*
*23:40:30 INFO loader :: ** Load empty quads table*
*23:40:30 INFO loader :: Load: C:\Users\IBM_ADMIN\My
Documents\dow*
*n\freebase-rdf-2012-12-09-00-00\freebase-rdf-2012-12-09-00-00 --
2012/12/27 23:4*
*0:30 IST*
*23:40:30 ERROR riot :: [line: 1, col: 1 ] Expected BNode
or IRI:*
* Got: [DIRECTIVE:prefix]*
*Exception in thread "main" org.openjena.riot.RiotException: [line: 1, col:
1 ] E*
*xpected BNode or IRI: Got: [DIRECTIVE:prefix]*
* at
org.openjena.riot.ErrorHandlerFactory$ErrorHandlerStd.fatal(ErrorHand*
*lerFactory.java:130)*
* at
org.openjena.riot.lang.LangEngine.raiseException(LangEngine.java:169)*
*
*
* at
org.openjena.riot.lang.LangEngine.exceptionDirect(LangEngine.java:162*
*)*
* at org.openjena.riot.lang.LangEngine.exception(LangEngine.java:155)
*
* at
org.openjena.riot.lang.LangNTuple.checkIRIOrBNode(LangNTuple.java:107*
*)*
* at org.openjena.riot.lang.LangNQuads.parseOne(LangNQuads.java:84)*
* at org.openjena.riot.lang.LangNQuads.parseOne(LangNQuads.java:34)*
* at org.openjena.riot.lang.LangNTuple.runParser(LangNTuple.java:69)*
* at org.openjena.riot.lang.LangBase.parse(LangBase.java:43)*
* at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:134)*
* at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:121)*
* at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:107)*
* at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadQuads$(BulkLoader*
*.java:160)*
* at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadDataset(BulkLoade*
*r.java:121)*
* at com.hp.hpl.jena.tdb.TDBLoader.loadDataset$(TDBLoader.java:283)*
* at com.hp.hpl.jena.tdb.TDBLoader.loadDataset(TDBLoader.java:196)*
* at com.hp.hpl.jena.tdb.TDBLoader.load(TDBLoader.java:75)*
* at tdb.tdbloader.loadQuads(tdbloader.java:163)*
* at tdb.tdbloader.exec(tdbloader.java:122)*
* at arq.cmdline.CmdMain.mainMethod(CmdMain.java:101)*
* at arq.cmdline.CmdMain.mainRun(CmdMain.java:63)*
* at arq.cmdline.CmdMain.mainRun(CmdMain.java:50)*
* at tdb.tdbloader.main(tdbloader.java:53)*
I need help in understanding this error and how to solve it. Is there a
problem with the input file?
@prefix ns: <http://rdf.freebase.com/ns/>.
@prefix key: <http://rdf.freebase.com/key/>.
@prefix owl: <http://www.w3.org/2002/07/owl#>.
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
ns:m.012rkqx ns:type.object.type ns:common.topic.
ns:m.012rkqx ns:type.object.name "High Fidelity"@en.
ns:m.012rkqx ns:type.object.type ns:music.single.
ns:m.012rkqx ns:type.object.key ns:authority.musicbrainz.name.TRACK3987054.
ns:m.012rkqx ns:type.object.type ns:music.recording.
ns:m.012rkqx key:authority.musicbrainz "258c45bd-4437-4580-8988-b3f3be975f9c".
ns:m.012rkqx key:authority.musicbrainz.name "TRACK3987054".
ns:m.012rkqx rdf:label "High Fidelity"@en.
ns:m.012rkqx rdf:type ns:common.topic.
ns:m.012rkqx rdf:type ns:music.single.
ns:m.012rkqx rdf:type ns:music.recording.
Re: Parsing a freebase RDF dump.
Posted by Andy Seaborne <an...@apache.org>.
On 27/08/13 00:21, Yuhan Zhang wrote:
> thanks for the suggestion, Andy. I'll convert the turtle file and retry it.
It seems their encoding of any odd characters is $xxxx, I'm not sure
where that came from. After the report from the 20121223 I thought
they'd clean this up - maybe a new one was get through. It's worth a
report to the freebase list.
Andy
>
> Yuhan Zhang
> Senior Software Engineer
> OneScreen Inc.
> www.onescreen.com
> (949) 525-4825 Ext: 177
> yzhang@onescreen.com <eh...@onescreen.com>
>
>
> On Mon, Aug 26, 2013 at 1:29 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 26/08/13 18:41, Yuhan Zhang wrote:
>>
>>> Hi Andy,
>>>
>>> the line 4643044 looks like this:
>>>
>>> ns:m.04ln83j key:user.robert.world$0027s_**tallest.building "preceded_by"
>>>
>>
>> Raw $ is not allowed in a prefixed name.
>>
>> But I guess $0027 is intended (which is ') so use %27.
>>
>> - - - - - -
>>
>> You can use \$ (in which case the URI will have a real $ in it) but some
>> tools may have problems, or use %24 (in which case the URI will have the 3
>> chars %-2-4 in it)
>>
>> [172s] PN_LOCAL_ESC ::= '\' ('_' | '~' | '.' | '-' | '!' | '$' |
>> '&' | "'" | '(' | ')' | '*' | '+' | ',' | ';' | '=' | '/' | '?' | '#' | '@'
>> | '%')
>>
>> http://www.w3.org/TR/turtle/#**sec-grammar-grammar<http://www.w3.org/TR/turtle/#sec-grammar-grammar>
>>
>> Andy
>>
>>
>>>
>>> Yuhan Zhang
>>> Senior Software Engineer
>>> OneScreen Inc.
>>> www.onescreen.com
>>> (949) 525-4825 Ext: 177
>>> yzhang@onescreen.com <eh...@onescreen.com>
>>>
>>>
>>> On Sat, Aug 24, 2013 at 2:50 AM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> On 24/08/13 03:29, Yuhan Zhang wrote:
>>>>
>>>> I reached a similar error with jena-2.10.1 but with a different
>>>>> character
>>>>> when parsing a more recent version of freebase-rdf-2013-08-04-00-00.
>>>>>
>>>>>
>>>> Yes - they keep changing things and don't check much.
>>>>
>>>>
>>>> WARN [line: 4632165, col: 55] Bad IRI: <
>>>>
>>>>> http://croctail.corpwatch.org/****#cw_506630,cw_{key}<http://croctail.corpwatch.org/**#cw_506630,cw_%7Bkey%7D>
>>>>> <http://**croctail.corpwatch.org/#cw_**506630,cw_%7Bkey%7D<http://croctail.corpwatch.org/#cw_506630,cw_%7Bkey%7D>
>>>>>>>
>>>>>
>>>>> Code: 4/UNWISE_CHARACTER
>>>>> in FRAGMENT: The character matches no grammar rules of URIs/IRIs. These
>>>>> characters are permitted in RDF URI References, XML system identifiers,
>>>>> and
>>>>> XML Schema anyURIs.
>>>>>
>>>>>
>>>> Only a warning - no '{' or '}' in IRIs
>>>>
>>>>
>>>> ERROR [line: 4643044, col: 35] Unknown char: $(36;0x0024)
>>>>
>>>>>
>>>>>
>>>> What's that line? $ is illegal in some places, but legal in others.
>>>>
>>>> Andy
>>>>
>>>>
>>>> Yuhan Zhang
>>>>> Senior Software Engineer
>>>>> OneScreen Inc.
>>>>> www.onescreen.com
>>>>> (949) 525-4825 Ext: 177
>>>>> yzhang@onescreen.com <eh...@onescreen.com>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Jan 8, 2013 at 4:24 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>>
>>>>> On 08/01/13 11:49, Rob Vesse wrote:
>>>>>
>>>>>>
>>>>>> 2.10.0 is the current development snapshot, you can get this via
>>>>>> maven
>>>>>>
>>>>>>> by
>>>>>>> setting the version for your Jena dependencies to 2.10.0-SNAPSHOT
>>>>>>>
>>>>>>>
>>>>>>> If you need to download the JARs (I.e. non-maven builds) you can find
>>>>>>> them
>>>>>>> on the Apache artifactory at
>>>>>>> https://repository.apache.org/******index.html#nexus-search;**<https://repository.apache.org/****index.html#nexus-search;**>
>>>>>>> quick~**jena<https://**repository.apache.org/**index.**
>>>>>>> html#nexus-search;quick~**jena<https://repository.apache.org/**index.html#nexus-search;quick~**jena>
>>>>>>> **>
>>>>>>> <https://**repository.apache.**org/index.**html#nexus-search;**
>>>>>>> quick~jena<http://repository.apache.org/index.**html#nexus-search;quick~jena>
>>>>>>> <https://repository.**apache.org/index.html#nexus-**search;quick~jena<https://repository.apache.org/index.html#nexus-search;quick~jena>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> You need to click on Show All Versions for the module you want in
>>>>>>> order
>>>>>>> to
>>>>>>> see download links for snapshots
>>>>>>>
>>>>>>> Rob
>>>>>>>
>>>>>>>
>>>>>>> And the download is available at:
>>>>>>
>>>>>> https://repository.apache.org/******content/repositories/**<https://repository.apache.org/****content/repositories/**>
>>>>>> <ht**tps://repository.apache.org/****content/repositories/**<https://repository.apache.org/**content/repositories/**>
>>>>>>>
>>>>>> snapshots/org/apache/jena/******apache-jena/<https://**
>>>>>> repository.apache.org/content/****repositories/snapshots/org/****<http://repository.apache.org/content/**repositories/snapshots/org/**>
>>>>>> apache/jena/apache-jena/<https**://repository.apache.org/**
>>>>>> content/repositories/**snapshots/org/apache/jena/**apache-jena/<https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena/>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>
>>>>>> (cough - see message of 28/Dec in this thread)
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 1/8/13 11:45 AM, "Abhishek Shivkumar" <ab...@gmail.com>
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> 1. I am using the correct version of rdf file that you have.
>>>>>>>
>>>>>>> 2. This error of unknown char (\92) is appearing in all the files at
>>>>>>>> different line numbers. I am not sure what this unknown char \(92)
>>>>>>>> is.
>>>>>>>> Tried to look in the surrounding of the line number in the file
>>>>>>>> contents
>>>>>>>> but can't find it :(
>>>>>>>> 3. I can only find version 2.7.4 at
>>>>>>>> http://www.apache.org/dist/******jena/binaries/<http://www.apache.org/dist/****jena/binaries/>
>>>>>>>> <http://www.**apache.org/dist/**jena/**binaries/<http://www.apache.org/dist/**jena/binaries/>
>>>>>>>>>
>>>>>>>> <http://www.**apache.org/dist/**jena/binaries/<http://apache.org/dist/jena/binaries/>
>>>>>>>> <http://www.**apache.org/dist/jena/binaries/<http://www.apache.org/dist/jena/binaries/>
>>>>>>>> **>
>>>>>>>> **>.
>>>>>>>>
>>>>>>>>
>>>>>>>> May be THIS is the reason. Do
>>>>>>>> you know where I can download the 2.10.0 version?
>>>>>>>>
>>>>>>>> Thanks much!
>>>>>>>>
>>>>>>>> Thank you!
>>>>>>>>
>>>>>>>> With Regards,
>>>>>>>> Abhishek S
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hi Andy,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I am using the script to correct the errors. When I run the
>>>>>>>>>> script
>>>>>>>>>> dwim
>>>>>>>>>> on all the part files, it shows error messages, and continues
>>>>>>>>>> processing.
>>>>>>>>>> Are these errors that are corrected, or still existing that need
>>>>>>>>>> attention?
>>>>>>>>>> Sample error message is:
>>>>>>>>>>
>>>>>>>>>> ERROR [line:25335, col:25] Unknown char: \(92)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> What's on the lines around there?
>>>>>>>>>>
>>>>>>>>> And if you've split the dump, which file?
>>>>>>>>>
>>>>>>>>> That needs correcting in the source. I can pare the first 30k lines
>>>>>>>>> of
>>>>>>>>> the file with Jena with no fixups.
>>>>>>>>>
>>>>>>>>> Maybe you don't have exactly the version of Freebase that I did
>>>>>>>>> freebase-rdf-2012-12-23-00-00.********gz. There is no suspect
>>>>>>>>> forms
>>>>>>>>>
>>>>>>>>> around
>>>>>>>>>
>>>>>>>>> line 25K of my copy.
>>>>>>>>>
>>>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
>>>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk.
>>>>>>>>> <---25335
>>>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>>>>>>>>>
>>>>>>>>> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Just wanted to know if we can ignore these messages while running
>>>>>>>>> the
>>>>>>>>>
>>>>>>>>> dwim
>>>>>>>>>> script.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> You can ignore WARN. ERRORs usually stop the parser as they
>>>>>>>>>>
>>>>>>>>> indicate
>>>>>>>>> structural problems.
>>>>>>>>>
>>>>>>>>> Andy
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thank you!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> With Regards,
>>>>>>>>>> Abhishek S
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> If you want to parse the Freebase dump, try this:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> http://people.apache.org/~**********andy/Freebase20121223/**
>>>>>>>>>>> Notes.**<http://people.apache.org/~********andy/Freebase20121223/Notes.**>
>>>>>>>>>>> ******txt<http://people.**apache.org/~******andy/**
>>>>>>>>>>> Freebase20121223/Notes.********txt<http://people.apache.org/~******andy/Freebase20121223/Notes.******txt>
>>>>>>>>>>>>
>>>>>>>>>>> <http://people.**apache.org/~******andy/**<http://apache.org/~****andy/**>
>>>>>>>>>>> Freebase20121223/Notes.******txt<http://people.apache.org/~**
>>>>>>>>>>> ****andy/Freebase20121223/**Notes.****txt<http://people.apache.org/~****andy/Freebase20121223/Notes.****txt>
>>>>>>>>>>>>
>>>>>>>>>>> **>
>>>>>>>>>>> <http:
>>>>>>>>>>> //people.apache.org/%7E**andy/******Freebase20121223/Notes.****
>>>>>>>>>>> txt<http://people.apache.org/%7E**andy/****Freebase20121223/Notes.**txt>
>>>>>>>>>>> <http://people.apache.org/%**7E**andy/**Freebase20121223/**
>>>>>>>>>>> Notes.**txt<http://people.apache.org/%7E**andy/**Freebase20121223/Notes.**txt>
>>>>>>>>>>>>
>>>>>>>>>>> **<http://people.apache.org/%**7E****andy/Freebase20121223/**
>>>>>>>>>>> Notes.***<http://people.apache.org/%7E****andy/Freebase20121223/Notes.***>
>>>>>>>>>>> *txt<http://people.apache.org/**%7E**andy/Freebase20121223/**
>>>>>>>>>>> Notes.**txt<http://people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <http://people.apache.org/%********7Eandy/Freebase20121223/**
>>>>>>>>>>> Notes.****txt
>>>>>>>>>>> <http:/
>>>>>>>>>>> /people.apache.org/%7Eandy/******Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/****Freebase20121223/Notes.txt>
>>>>>>>>>>> <ht**tp://people.apache.org/%**7Eandy/**Freebase20121223/**
>>>>>>>>>>> Notes.txt<http://people.apache.org/%7Eandy/**Freebase20121223/Notes.txt>
>>>>>>>>>>>>
>>>>>>>>>>> <htt**p://people.apache.org/%**7Eandy/**Freebase20121223/**
>>>>>>>>>>> Notes.txt<http://people.apache.org/%7Eandy/**Freebase20121223/Notes.txt>
>>>>>>>>>>> <http://people.**apache.org/%7Eandy/**Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> It takes about 90 minutes on my home desktop machine to fix and
>>>>>>>>>>> parse
>>>>>>>>>>> the
>>>>>>>>>>> data.
>>>>>>>>>>>
>>>>>>>>>>> To load it, get a very large machine - it has been reported [1]
>>>>>>>>>>> that a
>>>>>>>>>>> previous dump has been loaded into TDB.
>>>>>>>>>>>
>>>>>>>>>>> Andy
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>> http://lists.freebase.com/**********pipermail/freebase-**
>>>>>>>>>>> discuss/****<http://lists.freebase.com/********pipermail/freebase-discuss/****>
>>>>>>>>>>> <http://lists.**freebase.com/******pipermail/**
>>>>>>>>>>> freebase-discuss/**<http://lists.freebase.com/******pipermail/freebase-discuss/**>
>>>>>>>>>>>>
>>>>>>>>>>> <http://lists.freebase.com/********pipermail/freebase-discuss/**
>>>>>>>>>>> ****<http://lists.freebase.com/******pipermail/freebase-discuss/****>
>>>>>>>>>>> <http://lists.freebase.**com/****pipermail/freebase-**discuss/**<http://lists.freebase.com/****pipermail/freebase-discuss/**>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <**http://list <http://list>
>>>>>>>>>>> s.freebase.com/**pipermail/******freebase-discuss/**<http://s.freebase.com/**pipermail/****freebase-discuss/**>
>>>>>>>>>>> <http://s.**freebase.com/**pipermail/****freebase-discuss/**<http://s.freebase.com/**pipermail/**freebase-discuss/**>
>>>>>>>>>>>>
>>>>>>>>>>> <http://s.**freebase.com/****pipermail/**freebase-discuss/****<http://freebase.com/**pipermail/**freebase-discuss/**>
>>>>>>>>>>> <http://s.freebase.com/****pipermail/freebase-discuss/**<http://s.freebase.com/**pipermail/freebase-discuss/**>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> 2012-December/010169.html<******http**://lists.freebase.com/**
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> pipermail/freebase-discuss/********2012-December/010169.html<***
>>>>>>>>>>> *http**
>>>>>>>>>>> ://lists.fre <http://lists.fre>
>>>>>>>>>>> ebase.com/pipermail/freebase-******discuss/2012-December/**
>>>>>>>>>>> 010169.**<http://ebase.com/pipermail/freebase-****discuss/2012-December/010169.**>
>>>>>>>>>>> **html<http://ebase.com/**pipermail/freebase-**discuss/**
>>>>>>>>>>> 2012-December/010169.**html<http://ebase.com/pipermail/freebase-**discuss/2012-December/010169.**html>
>>>>>>>>>>>>
>>>>>>>>>>> <http://ebase.com/**pipermail/**freebase-discuss/**<http://ebase.com/**pipermail/freebase-discuss/**>
>>>>>>>>>>> 2012-December/010169.html<http**://ebase.com/pipermail/**
>>>>>>>>>>> freebase-discuss/2012-**December/010169.html<http://ebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Parsing a freebase RDF dump.
Posted by Yuhan Zhang <yz...@onescreen.com>.
thanks for the suggestion, Andy. I'll convert the turtle file and retry it.
Yuhan Zhang
Senior Software Engineer
OneScreen Inc.
www.onescreen.com
(949) 525-4825 Ext: 177
yzhang@onescreen.com <eh...@onescreen.com>
On Mon, Aug 26, 2013 at 1:29 PM, Andy Seaborne <an...@apache.org> wrote:
> On 26/08/13 18:41, Yuhan Zhang wrote:
>
>> Hi Andy,
>>
>> the line 4643044 looks like this:
>>
>> ns:m.04ln83j key:user.robert.world$0027s_**tallest.building "preceded_by"
>>
>
> Raw $ is not allowed in a prefixed name.
>
> But I guess $0027 is intended (which is ') so use %27.
>
> - - - - - -
>
> You can use \$ (in which case the URI will have a real $ in it) but some
> tools may have problems, or use %24 (in which case the URI will have the 3
> chars %-2-4 in it)
>
> [172s] PN_LOCAL_ESC ::= '\' ('_' | '~' | '.' | '-' | '!' | '$' |
> '&' | "'" | '(' | ')' | '*' | '+' | ',' | ';' | '=' | '/' | '?' | '#' | '@'
> | '%')
>
> http://www.w3.org/TR/turtle/#**sec-grammar-grammar<http://www.w3.org/TR/turtle/#sec-grammar-grammar>
>
> Andy
>
>
>>
>> Yuhan Zhang
>> Senior Software Engineer
>> OneScreen Inc.
>> www.onescreen.com
>> (949) 525-4825 Ext: 177
>> yzhang@onescreen.com <eh...@onescreen.com>
>>
>>
>> On Sat, Aug 24, 2013 at 2:50 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>> On 24/08/13 03:29, Yuhan Zhang wrote:
>>>
>>> I reached a similar error with jena-2.10.1 but with a different
>>>> character
>>>> when parsing a more recent version of freebase-rdf-2013-08-04-00-00.
>>>>
>>>>
>>> Yes - they keep changing things and don't check much.
>>>
>>>
>>> WARN [line: 4632165, col: 55] Bad IRI: <
>>>
>>>> http://croctail.corpwatch.org/****#cw_506630,cw_{key}<http://croctail.corpwatch.org/**#cw_506630,cw_%7Bkey%7D>
>>>> <http://**croctail.corpwatch.org/#cw_**506630,cw_%7Bkey%7D<http://croctail.corpwatch.org/#cw_506630,cw_%7Bkey%7D>
>>>> >>
>>>>
>>>> Code: 4/UNWISE_CHARACTER
>>>> in FRAGMENT: The character matches no grammar rules of URIs/IRIs. These
>>>> characters are permitted in RDF URI References, XML system identifiers,
>>>> and
>>>> XML Schema anyURIs.
>>>>
>>>>
>>> Only a warning - no '{' or '}' in IRIs
>>>
>>>
>>> ERROR [line: 4643044, col: 35] Unknown char: $(36;0x0024)
>>>
>>>>
>>>>
>>> What's that line? $ is illegal in some places, but legal in others.
>>>
>>> Andy
>>>
>>>
>>> Yuhan Zhang
>>>> Senior Software Engineer
>>>> OneScreen Inc.
>>>> www.onescreen.com
>>>> (949) 525-4825 Ext: 177
>>>> yzhang@onescreen.com <eh...@onescreen.com>
>>>>
>>>>
>>>>
>>>> On Tue, Jan 8, 2013 at 4:24 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>> On 08/01/13 11:49, Rob Vesse wrote:
>>>>
>>>>>
>>>>> 2.10.0 is the current development snapshot, you can get this via
>>>>> maven
>>>>>
>>>>>> by
>>>>>> setting the version for your Jena dependencies to 2.10.0-SNAPSHOT
>>>>>>
>>>>>>
>>>>>> If you need to download the JARs (I.e. non-maven builds) you can find
>>>>>> them
>>>>>> on the Apache artifactory at
>>>>>> https://repository.apache.org/******index.html#nexus-search;**<https://repository.apache.org/****index.html#nexus-search;**>
>>>>>> quick~**jena<https://**repository.apache.org/**index.**
>>>>>> html#nexus-search;quick~**jena<https://repository.apache.org/**index.html#nexus-search;quick~**jena>
>>>>>> **>
>>>>>> <https://**repository.apache.**org/index.**html#nexus-search;**
>>>>>> quick~jena<http://repository.apache.org/index.**html#nexus-search;quick~jena>
>>>>>> <https://repository.**apache.org/index.html#nexus-**search;quick~jena<https://repository.apache.org/index.html#nexus-search;quick~jena>
>>>>>> >
>>>>>>
>>>>>>
>>>>>>>
>>>>>>
>>>>>> You need to click on Show All Versions for the module you want in
>>>>>> order
>>>>>> to
>>>>>> see download links for snapshots
>>>>>>
>>>>>> Rob
>>>>>>
>>>>>>
>>>>>> And the download is available at:
>>>>>
>>>>> https://repository.apache.org/******content/repositories/**<https://repository.apache.org/****content/repositories/**>
>>>>> <ht**tps://repository.apache.org/****content/repositories/**<https://repository.apache.org/**content/repositories/**>
>>>>> >
>>>>> snapshots/org/apache/jena/******apache-jena/<https://**
>>>>> repository.apache.org/content/****repositories/snapshots/org/****<http://repository.apache.org/content/**repositories/snapshots/org/**>
>>>>> apache/jena/apache-jena/<https**://repository.apache.org/**
>>>>> content/repositories/**snapshots/org/apache/jena/**apache-jena/<https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena/>
>>>>> >
>>>>>
>>>>>
>>>>>>
>>>>>
>>>>> (cough - see message of 28/Dec in this thread)
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>>
>>>>> On 1/8/13 11:45 AM, "Abhishek Shivkumar" <ab...@gmail.com>
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> 1. I am using the correct version of rdf file that you have.
>>>>>>
>>>>>> 2. This error of unknown char (\92) is appearing in all the files at
>>>>>>> different line numbers. I am not sure what this unknown char \(92)
>>>>>>> is.
>>>>>>> Tried to look in the surrounding of the line number in the file
>>>>>>> contents
>>>>>>> but can't find it :(
>>>>>>> 3. I can only find version 2.7.4 at
>>>>>>> http://www.apache.org/dist/******jena/binaries/<http://www.apache.org/dist/****jena/binaries/>
>>>>>>> <http://www.**apache.org/dist/**jena/**binaries/<http://www.apache.org/dist/**jena/binaries/>
>>>>>>> >
>>>>>>> <http://www.**apache.org/dist/**jena/binaries/<http://apache.org/dist/jena/binaries/>
>>>>>>> <http://www.**apache.org/dist/jena/binaries/<http://www.apache.org/dist/jena/binaries/>
>>>>>>> **>
>>>>>>> **>.
>>>>>>>
>>>>>>>
>>>>>>> May be THIS is the reason. Do
>>>>>>> you know where I can download the 2.10.0 version?
>>>>>>>
>>>>>>> Thanks much!
>>>>>>>
>>>>>>> Thank you!
>>>>>>>
>>>>>>> With Regards,
>>>>>>> Abhishek S
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>>>>>>>
>>>>>>>
>>>>>>>> Hi Andy,
>>>>>>>>
>>>>>>>>
>>>>>>>>> I am using the script to correct the errors. When I run the
>>>>>>>>> script
>>>>>>>>> dwim
>>>>>>>>> on all the part files, it shows error messages, and continues
>>>>>>>>> processing.
>>>>>>>>> Are these errors that are corrected, or still existing that need
>>>>>>>>> attention?
>>>>>>>>> Sample error message is:
>>>>>>>>>
>>>>>>>>> ERROR [line:25335, col:25] Unknown char: \(92)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What's on the lines around there?
>>>>>>>>>
>>>>>>>> And if you've split the dump, which file?
>>>>>>>>
>>>>>>>> That needs correcting in the source. I can pare the first 30k lines
>>>>>>>> of
>>>>>>>> the file with Jena with no fixups.
>>>>>>>>
>>>>>>>> Maybe you don't have exactly the version of Freebase that I did
>>>>>>>> freebase-rdf-2012-12-23-00-00.********gz. There is no suspect
>>>>>>>> forms
>>>>>>>>
>>>>>>>> around
>>>>>>>>
>>>>>>>> line 25K of my copy.
>>>>>>>>
>>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
>>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk.
>>>>>>>> <---25335
>>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>>>>>>>>
>>>>>>>> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Just wanted to know if we can ignore these messages while running
>>>>>>>> the
>>>>>>>>
>>>>>>>> dwim
>>>>>>>>> script.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> You can ignore WARN. ERRORs usually stop the parser as they
>>>>>>>>>
>>>>>>>> indicate
>>>>>>>> structural problems.
>>>>>>>>
>>>>>>>> Andy
>>>>>>>>
>>>>>>>>
>>>>>>>> Thank you!
>>>>>>>>
>>>>>>>>
>>>>>>>>> With Regards,
>>>>>>>>> Abhishek S
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> If you want to parse the Freebase dump, try this:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> http://people.apache.org/~**********andy/Freebase20121223/**
>>>>>>>>>> Notes.**<http://people.apache.org/~********andy/Freebase20121223/Notes.**>
>>>>>>>>>> ******txt<http://people.**apache.org/~******andy/**
>>>>>>>>>> Freebase20121223/Notes.********txt<http://people.apache.org/~******andy/Freebase20121223/Notes.******txt>
>>>>>>>>>> >
>>>>>>>>>> <http://people.**apache.org/~******andy/**<http://apache.org/~****andy/**>
>>>>>>>>>> Freebase20121223/Notes.******txt<http://people.apache.org/~**
>>>>>>>>>> ****andy/Freebase20121223/**Notes.****txt<http://people.apache.org/~****andy/Freebase20121223/Notes.****txt>
>>>>>>>>>> >
>>>>>>>>>> **>
>>>>>>>>>> <http:
>>>>>>>>>> //people.apache.org/%7E**andy/******Freebase20121223/Notes.****
>>>>>>>>>> txt<http://people.apache.org/%7E**andy/****Freebase20121223/Notes.**txt>
>>>>>>>>>> <http://people.apache.org/%**7E**andy/**Freebase20121223/**
>>>>>>>>>> Notes.**txt<http://people.apache.org/%7E**andy/**Freebase20121223/Notes.**txt>
>>>>>>>>>> >
>>>>>>>>>> **<http://people.apache.org/%**7E****andy/Freebase20121223/**
>>>>>>>>>> Notes.***<http://people.apache.org/%7E****andy/Freebase20121223/Notes.***>
>>>>>>>>>> *txt<http://people.apache.org/**%7E**andy/Freebase20121223/**
>>>>>>>>>> Notes.**txt<http://people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> <http://people.apache.org/%********7Eandy/Freebase20121223/**
>>>>>>>>>> Notes.****txt
>>>>>>>>>> <http:/
>>>>>>>>>> /people.apache.org/%7Eandy/******Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/****Freebase20121223/Notes.txt>
>>>>>>>>>> <ht**tp://people.apache.org/%**7Eandy/**Freebase20121223/**
>>>>>>>>>> Notes.txt<http://people.apache.org/%7Eandy/**Freebase20121223/Notes.txt>
>>>>>>>>>> >
>>>>>>>>>> <htt**p://people.apache.org/%**7Eandy/**Freebase20121223/**
>>>>>>>>>> Notes.txt<http://people.apache.org/%7Eandy/**Freebase20121223/Notes.txt>
>>>>>>>>>> <http://people.**apache.org/%7Eandy/**Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It takes about 90 minutes on my home desktop machine to fix and
>>>>>>>>>> parse
>>>>>>>>>> the
>>>>>>>>>> data.
>>>>>>>>>>
>>>>>>>>>> To load it, get a very large machine - it has been reported [1]
>>>>>>>>>> that a
>>>>>>>>>> previous dump has been loaded into TDB.
>>>>>>>>>>
>>>>>>>>>> Andy
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> http://lists.freebase.com/**********pipermail/freebase-**
>>>>>>>>>> discuss/****<http://lists.freebase.com/********pipermail/freebase-discuss/****>
>>>>>>>>>> <http://lists.**freebase.com/******pipermail/**
>>>>>>>>>> freebase-discuss/**<http://lists.freebase.com/******pipermail/freebase-discuss/**>
>>>>>>>>>> >
>>>>>>>>>> <http://lists.freebase.com/********pipermail/freebase-discuss/**
>>>>>>>>>> ****<http://lists.freebase.com/******pipermail/freebase-discuss/****>
>>>>>>>>>> <http://lists.freebase.**com/****pipermail/freebase-**discuss/**<http://lists.freebase.com/****pipermail/freebase-discuss/**>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> <**http://list <http://list>
>>>>>>>>>> s.freebase.com/**pipermail/******freebase-discuss/**<http://s.freebase.com/**pipermail/****freebase-discuss/**>
>>>>>>>>>> <http://s.**freebase.com/**pipermail/****freebase-discuss/**<http://s.freebase.com/**pipermail/**freebase-discuss/**>
>>>>>>>>>> >
>>>>>>>>>> <http://s.**freebase.com/****pipermail/**freebase-discuss/****<http://freebase.com/**pipermail/**freebase-discuss/**>
>>>>>>>>>> <http://s.freebase.com/****pipermail/freebase-discuss/**<http://s.freebase.com/**pipermail/freebase-discuss/**>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 2012-December/010169.html<******http**://lists.freebase.com/**
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> pipermail/freebase-discuss/********2012-December/010169.html<***
>>>>>>>>>> *http**
>>>>>>>>>> ://lists.fre <http://lists.fre>
>>>>>>>>>> ebase.com/pipermail/freebase-******discuss/2012-December/**
>>>>>>>>>> 010169.**<http://ebase.com/pipermail/freebase-****discuss/2012-December/010169.**>
>>>>>>>>>> **html<http://ebase.com/**pipermail/freebase-**discuss/**
>>>>>>>>>> 2012-December/010169.**html<http://ebase.com/pipermail/freebase-**discuss/2012-December/010169.**html>
>>>>>>>>>> >
>>>>>>>>>> <http://ebase.com/**pipermail/**freebase-discuss/**<http://ebase.com/**pipermail/freebase-discuss/**>
>>>>>>>>>> 2012-December/010169.html<http**://ebase.com/pipermail/**
>>>>>>>>>> freebase-discuss/2012-**December/010169.html<http://ebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
--
The information contained in this e-mail is for the exclusive use of the
intended recipient(s) and may be confidential, proprietary, and/or legally
privileged. Inadvertent disclosure of this message does not constitute a
waiver of any privilege. If you receive this message in error, please do
not directly or indirectly print, copy, retransmit, disseminate, or
otherwise use the information. In addition, please delete this e-mail and
all copies and notify the sender.
Re: Parsing a freebase RDF dump.
Posted by Andy Seaborne <an...@apache.org>.
On 26/08/13 18:41, Yuhan Zhang wrote:
> Hi Andy,
>
> the line 4643044 looks like this:
>
> ns:m.04ln83j key:user.robert.world$0027s_tallest.building "preceded_by"
Raw $ is not allowed in a prefixed name.
But I guess $0027 is intended (which is ') so use %27.
- - - - - -
You can use \$ (in which case the URI will have a real $ in it) but some
tools may have problems, or use %24 (in which case the URI will have the
3 chars %-2-4 in it)
[172s] PN_LOCAL_ESC ::= '\' ('_' | '~' | '.' | '-' | '!' | '$' | '&'
| "'" | '(' | ')' | '*' | '+' | ',' | ';' | '=' | '/' | '?' | '#' | '@'
| '%')
http://www.w3.org/TR/turtle/#sec-grammar-grammar
Andy
>
>
> Yuhan Zhang
> Senior Software Engineer
> OneScreen Inc.
> www.onescreen.com
> (949) 525-4825 Ext: 177
> yzhang@onescreen.com <eh...@onescreen.com>
>
>
> On Sat, Aug 24, 2013 at 2:50 AM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 24/08/13 03:29, Yuhan Zhang wrote:
>>
>>> I reached a similar error with jena-2.10.1 but with a different character
>>> when parsing a more recent version of freebase-rdf-2013-08-04-00-00.
>>>
>>
>> Yes - they keep changing things and don't check much.
>>
>>
>> WARN [line: 4632165, col: 55] Bad IRI: <
>>> http://croctail.corpwatch.org/**#cw_506630,cw_{key}<http://croctail.corpwatch.org/#cw_506630,cw_%7Bkey%7D>>
>>> Code: 4/UNWISE_CHARACTER
>>> in FRAGMENT: The character matches no grammar rules of URIs/IRIs. These
>>> characters are permitted in RDF URI References, XML system identifiers,
>>> and
>>> XML Schema anyURIs.
>>>
>>
>> Only a warning - no '{' or '}' in IRIs
>>
>>
>> ERROR [line: 4643044, col: 35] Unknown char: $(36;0x0024)
>>>
>>
>> What's that line? $ is illegal in some places, but legal in others.
>>
>> Andy
>>
>>
>>> Yuhan Zhang
>>> Senior Software Engineer
>>> OneScreen Inc.
>>> www.onescreen.com
>>> (949) 525-4825 Ext: 177
>>> yzhang@onescreen.com <eh...@onescreen.com>
>>>
>>>
>>>
>>> On Tue, Jan 8, 2013 at 4:24 AM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> On 08/01/13 11:49, Rob Vesse wrote:
>>>>
>>>> 2.10.0 is the current development snapshot, you can get this via maven
>>>>> by
>>>>> setting the version for your Jena dependencies to 2.10.0-SNAPSHOT
>>>>>
>>>>>
>>>>> If you need to download the JARs (I.e. non-maven builds) you can find
>>>>> them
>>>>> on the Apache artifactory at
>>>>> https://repository.apache.org/****index.html#nexus-search;**
>>>>> quick~**jena<https://repository.apache.org/**index.html#nexus-search;quick~**jena>
>>>>> <https://**repository.apache.org/index.**html#nexus-search;quick~jena<https://repository.apache.org/index.html#nexus-search;quick~jena>
>>>>>>
>>>>>
>>>>>
>>>>> You need to click on Show All Versions for the module you want in order
>>>>> to
>>>>> see download links for snapshots
>>>>>
>>>>> Rob
>>>>>
>>>>>
>>>> And the download is available at:
>>>>
>>>> https://repository.apache.org/****content/repositories/**<https://repository.apache.org/**content/repositories/**>
>>>> snapshots/org/apache/jena/****apache-jena/<https://**
>>>> repository.apache.org/content/**repositories/snapshots/org/**
>>>> apache/jena/apache-jena/<https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena/>
>>>>>
>>>>
>>>>
>>>> (cough - see message of 28/Dec in this thread)
>>>>
>>>> Andy
>>>>
>>>>
>>>>
>>>> On 1/8/13 11:45 AM, "Abhishek Shivkumar" <ab...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> 1. I am using the correct version of rdf file that you have.
>>>>>
>>>>>> 2. This error of unknown char (\92) is appearing in all the files at
>>>>>> different line numbers. I am not sure what this unknown char \(92) is.
>>>>>> Tried to look in the surrounding of the line number in the file
>>>>>> contents
>>>>>> but can't find it :(
>>>>>> 3. I can only find version 2.7.4 at
>>>>>> http://www.apache.org/dist/****jena/binaries/<http://www.apache.org/dist/**jena/binaries/>
>>>>>> <http://www.**apache.org/dist/jena/binaries/<http://www.apache.org/dist/jena/binaries/>
>>>>>> **>.
>>>>>>
>>>>>> May be THIS is the reason. Do
>>>>>> you know where I can download the 2.10.0 version?
>>>>>>
>>>>>> Thanks much!
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>> With Regards,
>>>>>> Abhishek S
>>>>>>
>>>>>>
>>>>>> On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>>>
>>>>>> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi Andy,
>>>>>>>
>>>>>>>>
>>>>>>>> I am using the script to correct the errors. When I run the
>>>>>>>> script
>>>>>>>> dwim
>>>>>>>> on all the part files, it shows error messages, and continues
>>>>>>>> processing.
>>>>>>>> Are these errors that are corrected, or still existing that need
>>>>>>>> attention?
>>>>>>>> Sample error message is:
>>>>>>>>
>>>>>>>> ERROR [line:25335, col:25] Unknown char: \(92)
>>>>>>>>
>>>>>>>>
>>>>>>>> What's on the lines around there?
>>>>>>> And if you've split the dump, which file?
>>>>>>>
>>>>>>> That needs correcting in the source. I can pare the first 30k lines
>>>>>>> of
>>>>>>> the file with Jena with no fixups.
>>>>>>>
>>>>>>> Maybe you don't have exactly the version of Freebase that I did
>>>>>>> freebase-rdf-2012-12-23-00-00.******gz. There is no suspect forms
>>>>>>> around
>>>>>>>
>>>>>>> line 25K of my copy.
>>>>>>>
>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk.
>>>>>>> <---25335
>>>>>>> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>>>>>>>
>>>>>>> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Just wanted to know if we can ignore these messages while running
>>>>>>> the
>>>>>>>
>>>>>>>> dwim
>>>>>>>> script.
>>>>>>>>
>>>>>>>>
>>>>>>>> You can ignore WARN. ERRORs usually stop the parser as they
>>>>>>> indicate
>>>>>>> structural problems.
>>>>>>>
>>>>>>> Andy
>>>>>>>
>>>>>>>
>>>>>>> Thank you!
>>>>>>>
>>>>>>>>
>>>>>>>> With Regards,
>>>>>>>> Abhishek S
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> If you want to parse the Freebase dump, try this:
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> http://people.apache.org/~********andy/Freebase20121223/Notes.**
>>>>>>>>> ******txt<http://people.apache.org/~******andy/Freebase20121223/Notes.******txt>
>>>>>>>>> <http://people.**apache.org/~****andy/**
>>>>>>>>> Freebase20121223/Notes.****txt<http://people.apache.org/~****andy/Freebase20121223/Notes.****txt>
>>>>>>>>> **>
>>>>>>>>> <http:
>>>>>>>>> //people.apache.org/%7E**andy/****Freebase20121223/Notes.**txt<http://people.apache.org/%7E**andy/**Freebase20121223/Notes.**txt>
>>>>>>>>> **<http://people.apache.org/%7E****andy/Freebase20121223/Notes.***
>>>>>>>>> *txt<http://people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> <http://people.apache.org/%******7Eandy/Freebase20121223/**
>>>>>>>>> Notes.****txt
>>>>>>>>> <http:/
>>>>>>>>> /people.apache.org/%7Eandy/****Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/**Freebase20121223/Notes.txt>
>>>>>>>>> <htt**p://people.apache.org/%7Eandy/**Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> It takes about 90 minutes on my home desktop machine to fix and
>>>>>>>>> parse
>>>>>>>>> the
>>>>>>>>> data.
>>>>>>>>>
>>>>>>>>> To load it, get a very large machine - it has been reported [1]
>>>>>>>>> that a
>>>>>>>>> previous dump has been loaded into TDB.
>>>>>>>>>
>>>>>>>>> Andy
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> http://lists.freebase.com/********pipermail/freebase-discuss/****<http://lists.freebase.com/******pipermail/freebase-discuss/**>
>>>>>>>>> <http://lists.freebase.com/******pipermail/freebase-discuss/****<http://lists.freebase.com/****pipermail/freebase-discuss/**>
>>>>>>>>>>
>>>>>>>>> <**http://list <http://list>
>>>>>>>>> s.freebase.com/**pipermail/****freebase-discuss/**<http://s.freebase.com/**pipermail/**freebase-discuss/**>
>>>>>>>>> <http://s.**freebase.com/**pipermail/**freebase-discuss/**<http://s.freebase.com/**pipermail/freebase-discuss/**>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2012-December/010169.html<****http**://lists.freebase.com/**
>>>>>>>>>
>>>>>>>>> pipermail/freebase-discuss/******2012-December/010169.html<**http**
>>>>>>>>> ://lists.fre <http://lists.fre>
>>>>>>>>> ebase.com/pipermail/freebase-****discuss/2012-December/010169.**
>>>>>>>>> **html<http://ebase.com/pipermail/freebase-**discuss/2012-December/010169.**html>
>>>>>>>>> <http://ebase.com/**pipermail/freebase-discuss/**
>>>>>>>>> 2012-December/010169.html<http://ebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Parsing a freebase RDF dump.
Posted by Yuhan Zhang <yz...@onescreen.com>.
Hi Andy,
the line 4643044 looks like this:
ns:m.04ln83j key:user.robert.world$0027s_tallest.building "preceded_by"
Yuhan Zhang
Senior Software Engineer
OneScreen Inc.
www.onescreen.com
(949) 525-4825 Ext: 177
yzhang@onescreen.com <eh...@onescreen.com>
On Sat, Aug 24, 2013 at 2:50 AM, Andy Seaborne <an...@apache.org> wrote:
> On 24/08/13 03:29, Yuhan Zhang wrote:
>
>> I reached a similar error with jena-2.10.1 but with a different character
>> when parsing a more recent version of freebase-rdf-2013-08-04-00-00.
>>
>
> Yes - they keep changing things and don't check much.
>
>
> WARN [line: 4632165, col: 55] Bad IRI: <
>> http://croctail.corpwatch.org/**#cw_506630,cw_{key}<http://croctail.corpwatch.org/#cw_506630,cw_%7Bkey%7D>>
>> Code: 4/UNWISE_CHARACTER
>> in FRAGMENT: The character matches no grammar rules of URIs/IRIs. These
>> characters are permitted in RDF URI References, XML system identifiers,
>> and
>> XML Schema anyURIs.
>>
>
> Only a warning - no '{' or '}' in IRIs
>
>
> ERROR [line: 4643044, col: 35] Unknown char: $(36;0x0024)
>>
>
> What's that line? $ is illegal in some places, but legal in others.
>
> Andy
>
>
>> Yuhan Zhang
>> Senior Software Engineer
>> OneScreen Inc.
>> www.onescreen.com
>> (949) 525-4825 Ext: 177
>> yzhang@onescreen.com <eh...@onescreen.com>
>>
>>
>>
>> On Tue, Jan 8, 2013 at 4:24 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>> On 08/01/13 11:49, Rob Vesse wrote:
>>>
>>> 2.10.0 is the current development snapshot, you can get this via maven
>>>> by
>>>> setting the version for your Jena dependencies to 2.10.0-SNAPSHOT
>>>>
>>>>
>>>> If you need to download the JARs (I.e. non-maven builds) you can find
>>>> them
>>>> on the Apache artifactory at
>>>> https://repository.apache.org/****index.html#nexus-search;**
>>>> quick~**jena<https://repository.apache.org/**index.html#nexus-search;quick~**jena>
>>>> <https://**repository.apache.org/index.**html#nexus-search;quick~jena<https://repository.apache.org/index.html#nexus-search;quick~jena>
>>>> >
>>>>
>>>>
>>>> You need to click on Show All Versions for the module you want in order
>>>> to
>>>> see download links for snapshots
>>>>
>>>> Rob
>>>>
>>>>
>>> And the download is available at:
>>>
>>> https://repository.apache.org/****content/repositories/**<https://repository.apache.org/**content/repositories/**>
>>> snapshots/org/apache/jena/****apache-jena/<https://**
>>> repository.apache.org/content/**repositories/snapshots/org/**
>>> apache/jena/apache-jena/<https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena/>
>>> >
>>>
>>>
>>> (cough - see message of 28/Dec in this thread)
>>>
>>> Andy
>>>
>>>
>>>
>>> On 1/8/13 11:45 AM, "Abhishek Shivkumar" <ab...@gmail.com>
>>>> wrote:
>>>>
>>>> 1. I am using the correct version of rdf file that you have.
>>>>
>>>>> 2. This error of unknown char (\92) is appearing in all the files at
>>>>> different line numbers. I am not sure what this unknown char \(92) is.
>>>>> Tried to look in the surrounding of the line number in the file
>>>>> contents
>>>>> but can't find it :(
>>>>> 3. I can only find version 2.7.4 at
>>>>> http://www.apache.org/dist/****jena/binaries/<http://www.apache.org/dist/**jena/binaries/>
>>>>> <http://www.**apache.org/dist/jena/binaries/<http://www.apache.org/dist/jena/binaries/>
>>>>> **>.
>>>>>
>>>>> May be THIS is the reason. Do
>>>>> you know where I can download the 2.10.0 version?
>>>>>
>>>>> Thanks much!
>>>>>
>>>>> Thank you!
>>>>>
>>>>> With Regards,
>>>>> Abhishek S
>>>>>
>>>>>
>>>>> On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>>
>>>>> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>>>>>
>>>>>>
>>>>>> Hi Andy,
>>>>>>
>>>>>>>
>>>>>>> I am using the script to correct the errors. When I run the
>>>>>>> script
>>>>>>> dwim
>>>>>>> on all the part files, it shows error messages, and continues
>>>>>>> processing.
>>>>>>> Are these errors that are corrected, or still existing that need
>>>>>>> attention?
>>>>>>> Sample error message is:
>>>>>>>
>>>>>>> ERROR [line:25335, col:25] Unknown char: \(92)
>>>>>>>
>>>>>>>
>>>>>>> What's on the lines around there?
>>>>>> And if you've split the dump, which file?
>>>>>>
>>>>>> That needs correcting in the source. I can pare the first 30k lines
>>>>>> of
>>>>>> the file with Jena with no fixups.
>>>>>>
>>>>>> Maybe you don't have exactly the version of Freebase that I did
>>>>>> freebase-rdf-2012-12-23-00-00.******gz. There is no suspect forms
>>>>>> around
>>>>>>
>>>>>> line 25K of my copy.
>>>>>>
>>>>>> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
>>>>>> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk.
>>>>>> <---25335
>>>>>> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>>>>>>
>>>>>> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>>>>>>
>>>>>>
>>>>>>
>>>>>> Just wanted to know if we can ignore these messages while running
>>>>>> the
>>>>>>
>>>>>>> dwim
>>>>>>> script.
>>>>>>>
>>>>>>>
>>>>>>> You can ignore WARN. ERRORs usually stop the parser as they
>>>>>> indicate
>>>>>> structural problems.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>>>
>>>>>>> With Regards,
>>>>>>> Abhishek S
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> If you want to parse the Freebase dump, try this:
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> http://people.apache.org/~********andy/Freebase20121223/Notes.**
>>>>>>>> ******txt<http://people.apache.org/~******andy/Freebase20121223/Notes.******txt>
>>>>>>>> <http://people.**apache.org/~****andy/**
>>>>>>>> Freebase20121223/Notes.****txt<http://people.apache.org/~****andy/Freebase20121223/Notes.****txt>
>>>>>>>> **>
>>>>>>>> <http:
>>>>>>>> //people.apache.org/%7E**andy/****Freebase20121223/Notes.**txt<http://people.apache.org/%7E**andy/**Freebase20121223/Notes.**txt>
>>>>>>>> **<http://people.apache.org/%7E****andy/Freebase20121223/Notes.***
>>>>>>>> *txt<http://people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>>>>>>> >
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> <http://people.apache.org/%******7Eandy/Freebase20121223/**
>>>>>>>> Notes.****txt
>>>>>>>> <http:/
>>>>>>>> /people.apache.org/%7Eandy/****Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/**Freebase20121223/Notes.txt>
>>>>>>>> <htt**p://people.apache.org/%7Eandy/**Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> It takes about 90 minutes on my home desktop machine to fix and
>>>>>>>> parse
>>>>>>>> the
>>>>>>>> data.
>>>>>>>>
>>>>>>>> To load it, get a very large machine - it has been reported [1]
>>>>>>>> that a
>>>>>>>> previous dump has been loaded into TDB.
>>>>>>>>
>>>>>>>> Andy
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> http://lists.freebase.com/********pipermail/freebase-discuss/****<http://lists.freebase.com/******pipermail/freebase-discuss/**>
>>>>>>>> <http://lists.freebase.com/******pipermail/freebase-discuss/****<http://lists.freebase.com/****pipermail/freebase-discuss/**>
>>>>>>>> >
>>>>>>>> <**http://list <http://list>
>>>>>>>> s.freebase.com/**pipermail/****freebase-discuss/**<http://s.freebase.com/**pipermail/**freebase-discuss/**>
>>>>>>>> <http://s.**freebase.com/**pipermail/**freebase-discuss/**<http://s.freebase.com/**pipermail/freebase-discuss/**>
>>>>>>>> >
>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2012-December/010169.html<****http**://lists.freebase.com/**
>>>>>>>>
>>>>>>>> pipermail/freebase-discuss/******2012-December/010169.html<**http**
>>>>>>>> ://lists.fre <http://lists.fre>
>>>>>>>> ebase.com/pipermail/freebase-****discuss/2012-December/010169.**
>>>>>>>> **html<http://ebase.com/pipermail/freebase-**discuss/2012-December/010169.**html>
>>>>>>>> <http://ebase.com/**pipermail/freebase-discuss/**
>>>>>>>> 2012-December/010169.html<http://ebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>>>>>>> >
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>
--
The information contained in this e-mail is for the exclusive use of the
intended recipient(s) and may be confidential, proprietary, and/or legally
privileged. Inadvertent disclosure of this message does not constitute a
waiver of any privilege. If you receive this message in error, please do
not directly or indirectly print, copy, retransmit, disseminate, or
otherwise use the information. In addition, please delete this e-mail and
all copies and notify the sender.
Re: Parsing a freebase RDF dump.
Posted by Andy Seaborne <an...@apache.org>.
On 24/08/13 03:29, Yuhan Zhang wrote:
> I reached a similar error with jena-2.10.1 but with a different character
> when parsing a more recent version of freebase-rdf-2013-08-04-00-00.
Yes - they keep changing things and don't check much.
> WARN [line: 4632165, col: 55] Bad IRI: <
> http://croctail.corpwatch.org/#cw_506630,cw_{key}> Code: 4/UNWISE_CHARACTER
> in FRAGMENT: The character matches no grammar rules of URIs/IRIs. These
> characters are permitted in RDF URI References, XML system identifiers, and
> XML Schema anyURIs.
Only a warning - no '{' or '}' in IRIs
> ERROR [line: 4643044, col: 35] Unknown char: $(36;0x0024)
What's that line? $ is illegal in some places, but legal in others.
Andy
>
> Yuhan Zhang
> Senior Software Engineer
> OneScreen Inc.
> www.onescreen.com
> (949) 525-4825 Ext: 177
> yzhang@onescreen.com <eh...@onescreen.com>
>
>
> On Tue, Jan 8, 2013 at 4:24 AM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 08/01/13 11:49, Rob Vesse wrote:
>>
>>> 2.10.0 is the current development snapshot, you can get this via maven by
>>> setting the version for your Jena dependencies to 2.10.0-SNAPSHOT
>>>
>>>
>>> If you need to download the JARs (I.e. non-maven builds) you can find them
>>> on the Apache artifactory at
>>> https://repository.apache.org/**index.html#nexus-search;quick~**jena<https://repository.apache.org/index.html#nexus-search;quick~jena>
>>>
>>> You need to click on Show All Versions for the module you want in order to
>>> see download links for snapshots
>>>
>>> Rob
>>>
>>
>> And the download is available at:
>>
>> https://repository.apache.org/**content/repositories/**
>> snapshots/org/apache/jena/**apache-jena/<https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena/>
>>
>> (cough - see message of 28/Dec in this thread)
>>
>> Andy
>>
>>
>>
>>> On 1/8/13 11:45 AM, "Abhishek Shivkumar" <ab...@gmail.com>
>>> wrote:
>>>
>>> 1. I am using the correct version of rdf file that you have.
>>>> 2. This error of unknown char (\92) is appearing in all the files at
>>>> different line numbers. I am not sure what this unknown char \(92) is.
>>>> Tried to look in the surrounding of the line number in the file contents
>>>> but can't find it :(
>>>> 3. I can only find version 2.7.4 at
>>>> http://www.apache.org/dist/**jena/binaries/<http://www.apache.org/dist/jena/binaries/>.
>>>> May be THIS is the reason. Do
>>>> you know where I can download the 2.10.0 version?
>>>>
>>>> Thanks much!
>>>>
>>>> Thank you!
>>>>
>>>> With Regards,
>>>> Abhishek S
>>>>
>>>>
>>>> On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>>>>>
>>>>> Hi Andy,
>>>>>>
>>>>>> I am using the script to correct the errors. When I run the script
>>>>>> dwim
>>>>>> on all the part files, it shows error messages, and continues
>>>>>> processing.
>>>>>> Are these errors that are corrected, or still existing that need
>>>>>> attention?
>>>>>> Sample error message is:
>>>>>>
>>>>>> ERROR [line:25335, col:25] Unknown char: \(92)
>>>>>>
>>>>>>
>>>>> What's on the lines around there?
>>>>> And if you've split the dump, which file?
>>>>>
>>>>> That needs correcting in the source. I can pare the first 30k lines of
>>>>> the file with Jena with no fixups.
>>>>>
>>>>> Maybe you don't have exactly the version of Freebase that I did
>>>>> freebase-rdf-2012-12-23-00-00.****gz. There is no suspect forms around
>>>>> line 25K of my copy.
>>>>>
>>>>> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
>>>>> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk. <---25335
>>>>> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>>>>>
>>>>> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>>>>>
>>>>>
>>>>>
>>>>> Just wanted to know if we can ignore these messages while running the
>>>>>> dwim
>>>>>> script.
>>>>>>
>>>>>>
>>>>> You can ignore WARN. ERRORs usually stop the parser as they indicate
>>>>> structural problems.
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>> Thank you!
>>>>>>
>>>>>> With Regards,
>>>>>> Abhishek S
>>>>>>
>>>>>>
>>>>>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> If you want to parse the Freebase dump, try this:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> http://people.apache.org/~******andy/Freebase20121223/Notes.******txt<http://people.apache.org/~****andy/Freebase20121223/Notes.****txt>
>>>>>>> <http:
>>>>>>> //people.apache.org/%7E**andy/**Freebase20121223/Notes.**txt<http://people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>>>>>>>
>>>>>>>
>>>>>>> <http://people.apache.org/%****7Eandy/Freebase20121223/Notes.****txt
>>>>>>> <http:/
>>>>>>> /people.apache.org/%7Eandy/**Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> It takes about 90 minutes on my home desktop machine to fix and parse
>>>>>>> the
>>>>>>> data.
>>>>>>>
>>>>>>> To load it, get a very large machine - it has been reported [1] that a
>>>>>>> previous dump has been loaded into TDB.
>>>>>>>
>>>>>>> Andy
>>>>>>>
>>>>>>> [1]
>>>>>>> http://lists.freebase.com/******pipermail/freebase-discuss/**<http://lists.freebase.com/****pipermail/freebase-discuss/**>
>>>>>>> <**http://list <http://list>
>>>>>>> s.freebase.com/**pipermail/**freebase-discuss/**<http://s.freebase.com/**pipermail/freebase-discuss/**>
>>>>>>>>
>>>>>>> 2012-December/010169.html<**http**://lists.freebase.com/**
>>>>>>>
>>>>>>> pipermail/freebase-discuss/****2012-December/010169.html<http**
>>>>>>> ://lists.fre <http://lists.fre>
>>>>>>> ebase.com/pipermail/freebase-**discuss/2012-December/010169.**html<http://ebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>
>
Re: Parsing a freebase RDF dump.
Posted by Yuhan Zhang <yz...@onescreen.com>.
I reached a similar error with jena-2.10.1 but with a different character
when parsing a more recent version of freebase-rdf-2013-08-04-00-00.
WARN [line: 4632165, col: 55] Bad IRI: <
http://croctail.corpwatch.org/#cw_506630,cw_{key}> Code: 4/UNWISE_CHARACTER
in FRAGMENT: The character matches no grammar rules of URIs/IRIs. These
characters are permitted in RDF URI References, XML system identifiers, and
XML Schema anyURIs.
ERROR [line: 4643044, col: 35] Unknown char: $(36;0x0024)
Yuhan Zhang
Senior Software Engineer
OneScreen Inc.
www.onescreen.com
(949) 525-4825 Ext: 177
yzhang@onescreen.com <eh...@onescreen.com>
On Tue, Jan 8, 2013 at 4:24 AM, Andy Seaborne <an...@apache.org> wrote:
> On 08/01/13 11:49, Rob Vesse wrote:
>
>> 2.10.0 is the current development snapshot, you can get this via maven by
>> setting the version for your Jena dependencies to 2.10.0-SNAPSHOT
>>
>>
>> If you need to download the JARs (I.e. non-maven builds) you can find them
>> on the Apache artifactory at
>> https://repository.apache.org/**index.html#nexus-search;quick~**jena<https://repository.apache.org/index.html#nexus-search;quick~jena>
>>
>> You need to click on Show All Versions for the module you want in order to
>> see download links for snapshots
>>
>> Rob
>>
>
> And the download is available at:
>
> https://repository.apache.org/**content/repositories/**
> snapshots/org/apache/jena/**apache-jena/<https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena/>
>
> (cough - see message of 28/Dec in this thread)
>
> Andy
>
>
>
>> On 1/8/13 11:45 AM, "Abhishek Shivkumar" <ab...@gmail.com>
>> wrote:
>>
>> 1. I am using the correct version of rdf file that you have.
>>> 2. This error of unknown char (\92) is appearing in all the files at
>>> different line numbers. I am not sure what this unknown char \(92) is.
>>> Tried to look in the surrounding of the line number in the file contents
>>> but can't find it :(
>>> 3. I can only find version 2.7.4 at
>>> http://www.apache.org/dist/**jena/binaries/<http://www.apache.org/dist/jena/binaries/>.
>>> May be THIS is the reason. Do
>>> you know where I can download the 2.10.0 version?
>>>
>>> Thanks much!
>>>
>>> Thank you!
>>>
>>> With Regards,
>>> Abhishek S
>>>
>>>
>>> On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>>>>
>>>> Hi Andy,
>>>>>
>>>>> I am using the script to correct the errors. When I run the script
>>>>> dwim
>>>>> on all the part files, it shows error messages, and continues
>>>>> processing.
>>>>> Are these errors that are corrected, or still existing that need
>>>>> attention?
>>>>> Sample error message is:
>>>>>
>>>>> ERROR [line:25335, col:25] Unknown char: \(92)
>>>>>
>>>>>
>>>> What's on the lines around there?
>>>> And if you've split the dump, which file?
>>>>
>>>> That needs correcting in the source. I can pare the first 30k lines of
>>>> the file with Jena with no fixups.
>>>>
>>>> Maybe you don't have exactly the version of Freebase that I did
>>>> freebase-rdf-2012-12-23-00-00.****gz. There is no suspect forms around
>>>> line 25K of my copy.
>>>>
>>>> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
>>>> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk. <---25335
>>>> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>>>>
>>>> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>>>>
>>>>
>>>>
>>>> Just wanted to know if we can ignore these messages while running the
>>>>> dwim
>>>>> script.
>>>>>
>>>>>
>>>> You can ignore WARN. ERRORs usually stop the parser as they indicate
>>>> structural problems.
>>>>
>>>> Andy
>>>>
>>>>
>>>> Thank you!
>>>>>
>>>>> With Regards,
>>>>> Abhishek S
>>>>>
>>>>>
>>>>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org>
>>>>> wrote:
>>>>>
>>>>> If you want to parse the Freebase dump, try this:
>>>>>
>>>>>>
>>>>>>
>>>>>> http://people.apache.org/~******andy/Freebase20121223/Notes.******txt<http://people.apache.org/~****andy/Freebase20121223/Notes.****txt>
>>>>>> <http:
>>>>>> //people.apache.org/%7E**andy/**Freebase20121223/Notes.**txt<http://people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>>>>> >
>>>>>>
>>>>>> <http://people.apache.org/%****7Eandy/Freebase20121223/Notes.****txt
>>>>>> <http:/
>>>>>> /people.apache.org/%7Eandy/**Freebase20121223/Notes.txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>>>>> >
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> It takes about 90 minutes on my home desktop machine to fix and parse
>>>>>> the
>>>>>> data.
>>>>>>
>>>>>> To load it, get a very large machine - it has been reported [1] that a
>>>>>> previous dump has been loaded into TDB.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>> [1]
>>>>>> http://lists.freebase.com/******pipermail/freebase-discuss/**<http://lists.freebase.com/****pipermail/freebase-discuss/**>
>>>>>> <**http://list <http://list>
>>>>>> s.freebase.com/**pipermail/**freebase-discuss/**<http://s.freebase.com/**pipermail/freebase-discuss/**>
>>>>>> >
>>>>>> 2012-December/010169.html<**http**://lists.freebase.com/**
>>>>>>
>>>>>> pipermail/freebase-discuss/****2012-December/010169.html<http**
>>>>>> ://lists.fre <http://lists.fre>
>>>>>> ebase.com/pipermail/freebase-**discuss/2012-December/010169.**html<http://ebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>>>>> >
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>
--
The information contained in this e-mail is for the exclusive use of the
intended recipient(s) and may be confidential, proprietary, and/or legally
privileged. Inadvertent disclosure of this message does not constitute a
waiver of any privilege. If you receive this message in error, please do
not directly or indirectly print, copy, retransmit, disseminate, or
otherwise use the information. In addition, please delete this e-mail and
all copies and notify the sender.
Re: Parsing a freebase RDF dump.
Posted by Andy Seaborne <an...@apache.org>.
On 08/01/13 11:49, Rob Vesse wrote:
> 2.10.0 is the current development snapshot, you can get this via maven by
> setting the version for your Jena dependencies to 2.10.0-SNAPSHOT
>
>
> If you need to download the JARs (I.e. non-maven builds) you can find them
> on the Apache artifactory at
> https://repository.apache.org/index.html#nexus-search;quick~jena
>
> You need to click on Show All Versions for the module you want in order to
> see download links for snapshots
>
> Rob
And the download is available at:
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena/
(cough - see message of 28/Dec in this thread)
Andy
>
> On 1/8/13 11:45 AM, "Abhishek Shivkumar" <ab...@gmail.com> wrote:
>
>> 1. I am using the correct version of rdf file that you have.
>> 2. This error of unknown char (\92) is appearing in all the files at
>> different line numbers. I am not sure what this unknown char \(92) is.
>> Tried to look in the surrounding of the line number in the file contents
>> but can't find it :(
>> 3. I can only find version 2.7.4 at
>> http://www.apache.org/dist/jena/binaries/. May be THIS is the reason. Do
>> you know where I can download the 2.10.0 version?
>>
>> Thanks much!
>>
>> Thank you!
>>
>> With Regards,
>> Abhishek S
>>
>>
>> On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>>> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>>>
>>>> Hi Andy,
>>>>
>>>> I am using the script to correct the errors. When I run the script
>>>> dwim
>>>> on all the part files, it shows error messages, and continues
>>>> processing.
>>>> Are these errors that are corrected, or still existing that need
>>>> attention?
>>>> Sample error message is:
>>>>
>>>> ERROR [line:25335, col:25] Unknown char: \(92)
>>>>
>>>
>>> What's on the lines around there?
>>> And if you've split the dump, which file?
>>>
>>> That needs correcting in the source. I can pare the first 30k lines of
>>> the file with Jena with no fixups.
>>>
>>> Maybe you don't have exactly the version of Freebase that I did
>>> freebase-rdf-2012-12-23-00-00.**gz. There is no suspect forms around
>>> line 25K of my copy.
>>>
>>> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
>>> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk. <---25335
>>> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>>>
>>> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>>>
>>>
>>>
>>>> Just wanted to know if we can ignore these messages while running the
>>>> dwim
>>>> script.
>>>>
>>>
>>> You can ignore WARN. ERRORs usually stop the parser as they indicate
>>> structural problems.
>>>
>>> Andy
>>>
>>>
>>>> Thank you!
>>>>
>>>> With Regards,
>>>> Abhishek S
>>>>
>>>>
>>>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>> If you want to parse the Freebase dump, try this:
>>>>>
>>>>>
>>>>> http://people.apache.org/~****andy/Freebase20121223/Notes.****txt<http:
>>>>> //people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>>>>
>>>>> <http://people.apache.org/%**7Eandy/Freebase20121223/Notes.**txt<http:/
>>>>> /people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>>>>>
>>>>>
>>>>>
>>>>> It takes about 90 minutes on my home desktop machine to fix and parse
>>>>> the
>>>>> data.
>>>>>
>>>>> To load it, get a very large machine - it has been reported [1] that a
>>>>> previous dump has been loaded into TDB.
>>>>>
>>>>> Andy
>>>>>
>>>>> [1]
>>>>> http://lists.freebase.com/****pipermail/freebase-discuss/**<http://list
>>>>> s.freebase.com/**pipermail/freebase-discuss/**>
>>>>> 2012-December/010169.html<http**://lists.freebase.com/**
>>>>>
>>>>> pipermail/freebase-discuss/**2012-December/010169.html<http://lists.fre
>>>>> ebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>
Re: Parsing a freebase RDF dump.
Posted by Rob Vesse <rv...@yarcdata.com>.
2.10.0 is the current development snapshot, you can get this via maven by
setting the version for your Jena dependencies to 2.10.0-SNAPSHOT
If you need to download the JARs (I.e. non-maven builds) you can find them
on the Apache artifactory at
https://repository.apache.org/index.html#nexus-search;quick~jena
You need to click on Show All Versions for the module you want in order to
see download links for snapshots
Rob
On 1/8/13 11:45 AM, "Abhishek Shivkumar" <ab...@gmail.com> wrote:
>1. I am using the correct version of rdf file that you have.
>2. This error of unknown char (\92) is appearing in all the files at
>different line numbers. I am not sure what this unknown char \(92) is.
>Tried to look in the surrounding of the line number in the file contents
>but can't find it :(
>3. I can only find version 2.7.4 at
>http://www.apache.org/dist/jena/binaries/. May be THIS is the reason. Do
>you know where I can download the 2.10.0 version?
>
>Thanks much!
>
>Thank you!
>
>With Regards,
>Abhishek S
>
>
>On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>>
>>> Hi Andy,
>>>
>>> I am using the script to correct the errors. When I run the script
>>> dwim
>>> on all the part files, it shows error messages, and continues
>>>processing.
>>> Are these errors that are corrected, or still existing that need
>>> attention?
>>> Sample error message is:
>>>
>>> ERROR [line:25335, col:25] Unknown char: \(92)
>>>
>>
>> What's on the lines around there?
>> And if you've split the dump, which file?
>>
>> That needs correcting in the source. I can pare the first 30k lines of
>> the file with Jena with no fixups.
>>
>> Maybe you don't have exactly the version of Freebase that I did
>> freebase-rdf-2012-12-23-00-00.**gz. There is no suspect forms around
>> line 25K of my copy.
>>
>> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
>> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk. <---25335
>> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>>
>> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>>
>>
>>
>>> Just wanted to know if we can ignore these messages while running the
>>>dwim
>>> script.
>>>
>>
>> You can ignore WARN. ERRORs usually stop the parser as they indicate
>> structural problems.
>>
>> Andy
>>
>>
>>> Thank you!
>>>
>>> With Regards,
>>> Abhishek S
>>>
>>>
>>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> If you want to parse the Freebase dump, try this:
>>>>
>>>>
>>>>http://people.apache.org/~****andy/Freebase20121223/Notes.****txt<http:
>>>>//people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>>>
>>>><http://people.apache.org/%**7Eandy/Freebase20121223/Notes.**txt<http:/
>>>>/people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>>> >
>>>>
>>>>
>>>> It takes about 90 minutes on my home desktop machine to fix and parse
>>>>the
>>>> data.
>>>>
>>>> To load it, get a very large machine - it has been reported [1] that a
>>>> previous dump has been loaded into TDB.
>>>>
>>>> Andy
>>>>
>>>> [1]
>>>>http://lists.freebase.com/****pipermail/freebase-discuss/**<http://list
>>>>s.freebase.com/**pipermail/freebase-discuss/**>
>>>> 2012-December/010169.html<http**://lists.freebase.com/**
>>>>
>>>>pipermail/freebase-discuss/**2012-December/010169.html<http://lists.fre
>>>>ebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>>> >
>>>>
>>>>
>>>
>>
Re: Parsing a freebase RDF dump.
Posted by Abhishek Shivkumar <ab...@gmail.com>.
1. I am using the correct version of rdf file that you have.
2. This error of unknown char (\92) is appearing in all the files at
different line numbers. I am not sure what this unknown char \(92) is.
Tried to look in the surrounding of the line number in the file contents
but can't find it :(
3. I can only find version 2.7.4 at
http://www.apache.org/dist/jena/binaries/. May be THIS is the reason. Do
you know where I can download the 2.10.0 version?
Thanks much!
Thank you!
With Regards,
Abhishek S
On Tue, Jan 8, 2013 at 5:26 AM, Andy Seaborne <an...@apache.org> wrote:
> On 08/01/13 11:00, Abhishek Shivkumar wrote:
>
>> Hi Andy,
>>
>> I am using the script to correct the errors. When I run the script
>> dwim
>> on all the part files, it shows error messages, and continues processing.
>> Are these errors that are corrected, or still existing that need
>> attention?
>> Sample error message is:
>>
>> ERROR [line:25335, col:25] Unknown char: \(92)
>>
>
> What's on the lines around there?
> And if you've split the dump, which file?
>
> That needs correcting in the source. I can pare the first 30k lines of
> the file with Jena with no fixups.
>
> Maybe you don't have exactly the version of Freebase that I did
> freebase-rdf-2012-12-23-00-00.**gz. There is no suspect forms around
> line 25K of my copy.
>
> ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
> ns:award.award_winner ns:type.type.instance ns:m.05x3tbk. <---25335
> ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
>
> You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>
>
>
>> Just wanted to know if we can ignore these messages while running the dwim
>> script.
>>
>
> You can ignore WARN. ERRORs usually stop the parser as they indicate
> structural problems.
>
> Andy
>
>
>> Thank you!
>>
>> With Regards,
>> Abhishek S
>>
>>
>> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org> wrote:
>>
>> If you want to parse the Freebase dump, try this:
>>>
>>> http://people.apache.org/~****andy/Freebase20121223/Notes.****txt<http://people.apache.org/%7E**andy/Freebase20121223/Notes.**txt>
>>> <http://people.apache.org/%**7Eandy/Freebase20121223/Notes.**txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>> >
>>>
>>>
>>> It takes about 90 minutes on my home desktop machine to fix and parse the
>>> data.
>>>
>>> To load it, get a very large machine - it has been reported [1] that a
>>> previous dump has been loaded into TDB.
>>>
>>> Andy
>>>
>>> [1] http://lists.freebase.com/****pipermail/freebase-discuss/**<http://lists.freebase.com/**pipermail/freebase-discuss/**>
>>> 2012-December/010169.html<http**://lists.freebase.com/**
>>> pipermail/freebase-discuss/**2012-December/010169.html<http://lists.freebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>> >
>>>
>>>
>>
>
Re: Parsing a freebase RDF dump.
Posted by Andy Seaborne <an...@apache.org>.
On 08/01/13 11:00, Abhishek Shivkumar wrote:
> Hi Andy,
>
> I am using the script to correct the errors. When I run the script dwim
> on all the part files, it shows error messages, and continues processing.
> Are these errors that are corrected, or still existing that need attention?
> Sample error message is:
>
> ERROR [line:25335, col:25] Unknown char: \(92)
What's on the lines around there?
And if you've split the dump, which file?
That needs correcting in the source. I can pare the first 30k lines of
the file with Jena with no fixups.
Maybe you don't have exactly the version of Freebase that I did
freebase-rdf-2012-12-23-00-00.gz. There is no suspect forms around line
25K of my copy.
ns:award.award_winner ns:type.type.instance ns:m.03cpgmq.
ns:award.award_winner ns:type.type.instance ns:m.05x3tbk. <---25335
ns:award.award_winner ns:type.type.instance ns:m.05q_rp.
You also need the latest version of Jena (recent 2.10.0 SNAPSHOT).
>
> Just wanted to know if we can ignore these messages while running the dwim
> script.
You can ignore WARN. ERRORs usually stop the parser as they indicate
structural problems.
Andy
>
> Thank you!
>
> With Regards,
> Abhishek S
>
>
> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> If you want to parse the Freebase dump, try this:
>>
>> http://people.apache.org/~**andy/Freebase20121223/Notes.**txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>
>> It takes about 90 minutes on my home desktop machine to fix and parse the
>> data.
>>
>> To load it, get a very large machine - it has been reported [1] that a
>> previous dump has been loaded into TDB.
>>
>> Andy
>>
>> [1] http://lists.freebase.com/**pipermail/freebase-discuss/**
>> 2012-December/010169.html<http://lists.freebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>
>
Re: Parsing a freebase RDF dump.
Posted by Olivier Rossel <ol...@gmail.com>.
isn't BaseKB supposed to be a clean RDF version of Freebase?
http://basekb.com/
On Tue, Jan 8, 2013 at 12:00 PM, Abhishek Shivkumar
<ab...@gmail.com> wrote:
> Hi Andy,
>
> I am using the script to correct the errors. When I run the script dwim
> on all the part files, it shows error messages, and continues processing.
> Are these errors that are corrected, or still existing that need attention?
> Sample error message is:
>
> ERROR [line:25335, col:25] Unknown char: \(92)
>
> Just wanted to know if we can ignore these messages while running the dwim
> script.
>
> Thank you!
>
> With Regards,
> Abhishek S
>
>
> On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> If you want to parse the Freebase dump, try this:
>>
>> http://people.apache.org/~**andy/Freebase20121223/Notes.**txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>>
>> It takes about 90 minutes on my home desktop machine to fix and parse the
>> data.
>>
>> To load it, get a very large machine - it has been reported [1] that a
>> previous dump has been loaded into TDB.
>>
>> Andy
>>
>> [1] http://lists.freebase.com/**pipermail/freebase-discuss/**
>> 2012-December/010169.html<http://lists.freebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>>
Re: Parsing a freebase RDF dump.
Posted by Abhishek Shivkumar <ab...@gmail.com>.
Hi Andy,
I am using the script to correct the errors. When I run the script dwim
on all the part files, it shows error messages, and continues processing.
Are these errors that are corrected, or still existing that need attention?
Sample error message is:
ERROR [line:25335, col:25] Unknown char: \(92)
Just wanted to know if we can ignore these messages while running the dwim
script.
Thank you!
With Regards,
Abhishek S
On Sat, Dec 29, 2012 at 1:58 PM, Andy Seaborne <an...@apache.org> wrote:
> If you want to parse the Freebase dump, try this:
>
> http://people.apache.org/~**andy/Freebase20121223/Notes.**txt<http://people.apache.org/%7Eandy/Freebase20121223/Notes.txt>
>
> It takes about 90 minutes on my home desktop machine to fix and parse the
> data.
>
> To load it, get a very large machine - it has been reported [1] that a
> previous dump has been loaded into TDB.
>
> Andy
>
> [1] http://lists.freebase.com/**pipermail/freebase-discuss/**
> 2012-December/010169.html<http://lists.freebase.com/pipermail/freebase-discuss/2012-December/010169.html>
>
Parsing a freebase RDF dump.
Posted by Andy Seaborne <an...@apache.org>.
If you want to parse the Freebase dump, try this:
http://people.apache.org/~andy/Freebase20121223/Notes.txt
It takes about 90 minutes on my home desktop machine to fix and parse
the data.
To load it, get a very large machine - it has been reported [1] that a
previous dump has been loaded into TDB.
Andy
[1]
http://lists.freebase.com/pipermail/freebase-discuss/2012-December/010169.html
Re: Need help in loading a large RDF file into JENA TDB using TDBLoader
Posted by Andy Seaborne <an...@apache.org>.
On 28/12/12 13:26, Abhishek Shivkumar wrote:
> Thanks Andy. Awesome!
>
> So, I am downloading the latest dump of Freebase RDF ->
> freebase-rdf-2012-12-23-00-00.gz
>
> Let me check with that and use tdbloader to see if it has been corrected.
>
> Also, when will JENA 2.10.0 with this correction, be released?
Development builds are always available:
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
We don't have a date for the release but
(1) the development builds should pass all the tests - the CI Jenkins
installation will tell the dev mailing list if not!
(2) expect a test/release cycle for 2.10.0 when we ask the community to
test things before a release, and not after, so it that will take a while.
So while not as highly tested, the dev builds are generally pretty good
and contain no known issues that would block as release.
I wrote to freebase-discuss about the 13 issue but running
sed -e 's/\.$/ ./'
might be a good idea.
To load TDB, you're going to need a big RAM machine and patience.
So checking the data first, despite the delay, is going to be a good idea.
tdbloader2 is likely to be faster. There are some tuning parameters as
well - see the script for details. Adding --parallel=3 is good if your
sort(1) supports it.
I'll download the 32-12 data but I don't have access to a large machine
at the moment. I'm going to run "split -l 10000000" to make finding
errors easier.
Andy
>
> Thank you!
>
> With Regards,
> Abhishek S
>
>
> On Fri, Dec 28, 2012 at 6:32 PM, Andy Seaborne <andy@apache.org
> <ma...@apache.org>> wrote:
>
> On 28/12/12 07:42, Abhishek Shivkumar wrote:
>
> Hi Andy,
>
> Here are the triples from the neighborhood of line 270608. i
> tried
> finding the error but couldn't. Do you see any by chance?
> I printed the line number too on the left just in case. Ex:
> "line num
> 270591-"
>
>
> Not quite the right line but close ... this may be the problem:
>
> Line:
> -----------------
>
> ns:m.01gqn1
> ns:base.braziliangovt.__brazilian_political_party.__number 13.
> -----------------
>
> and the problem is the 13.
>
> The WG spec in development has:
>
> [21] DECIMAL ::= [+-]? [0-9]* '.' [0-9]+
>
> so a decimal must have a trailing digit, and "13." is integer 13
> followed by a DOT (terminates the triples).
>
> But in the W3C submission has a know problem in this area:
>
> [18] decimal ::= ('-' | '+')? ( [0-9]+ '.' [0-9]* |
> '.' ([0-9])+ | ([0-9])+ )
>
> and 13. is ambiguous. Is it 13 and a DOT or a decimal with lexical
> form "13." The normal way to tokenize is to choose the longest
> match (so ":abc" isn't ":a" then "bc") and that means you need a
> space to the tokens '13' and DOT
>
> Jena 2.7.4 follows the submission and "13." is a decimal and the
> needs a trailing DOT.
>
> In fact, using space-DOT everywhere would be very sensible.
> Trailing dots on prefix names may confuse some older parsers.
>
> Jena development (2.10.0) follows the W3C WG spec and it's 13
> integer and a trailing DOT and parses.
>
> Do you have a corrected version of freebase-rdf-2012-12-09-00-00? I
> downloaded it but there are other things to fix up before it gets to
> that point.
>
> Andy
>
>
>
>
Re: Need help in loading a large RDF file into JENA TDB using TDBLoader
Posted by Abhishek Shivkumar <ab...@gmail.com>.
Thanks Andy. Awesome!
So, I am downloading the latest dump of Freebase RDF ->
freebase-rdf-2012-12-23-00-00.gz
Let me check with that and use tdbloader to see if it has been corrected.
Also, when will JENA 2.10.0 with this correction, be released?
Thank you!
With Regards,
Abhishek S
On Fri, Dec 28, 2012 at 6:32 PM, Andy Seaborne <an...@apache.org> wrote:
> On 28/12/12 07:42, Abhishek Shivkumar wrote:
>
>> Hi Andy,
>>
>> Here are the triples from the neighborhood of line 270608. i tried
>> finding the error but couldn't. Do you see any by chance?
>> I printed the line number too on the left just in case. Ex: "line num
>> 270591-"
>>
>
> Not quite the right line but close ... this may be the problem:
>
> Line:
> -----------------
>
> ns:m.01gqn1 ns:base.braziliangovt.**brazilian_political_party.**number
> 13.
> -----------------
>
> and the problem is the 13.
>
> The WG spec in development has:
>
> [21] DECIMAL ::= [+-]? [0-9]* '.' [0-9]+
>
> so a decimal must have a trailing digit, and "13." is integer 13 followed
> by a DOT (terminates the triples).
>
> But in the W3C submission has a know problem in this area:
>
> [18] decimal ::= ('-' | '+')? ( [0-9]+ '.' [0-9]* | '.'
> ([0-9])+ | ([0-9])+ )
>
> and 13. is ambiguous. Is it 13 and a DOT or a decimal with lexical form
> "13." The normal way to tokenize is to choose the longest match (so ":abc"
> isn't ":a" then "bc") and that means you need a space to the tokens '13'
> and DOT
>
> Jena 2.7.4 follows the submission and "13." is a decimal and the needs a
> trailing DOT.
>
> In fact, using space-DOT everywhere would be very sensible. Trailing dots
> on prefix names may confuse some older parsers.
>
> Jena development (2.10.0) follows the W3C WG spec and it's 13 integer and
> a trailing DOT and parses.
>
> Do you have a corrected version of freebase-rdf-2012-12-09-00-00? I
> downloaded it but there are other things to fix up before it gets to that
> point.
>
> Andy
>
>
>
>
Re: Need help in loading a large RDF file into JENA TDB using TDBLoader
Posted by Andy Seaborne <an...@apache.org>.
On 28/12/12 07:42, Abhishek Shivkumar wrote:
> Hi Andy,
>
> Here are the triples from the neighborhood of line 270608. i tried
> finding the error but couldn't. Do you see any by chance?
> I printed the line number too on the left just in case. Ex: "line num
> 270591-"
Not quite the right line but close ... this may be the problem:
Line:
-----------------
ns:m.01gqn1 ns:base.braziliangovt.brazilian_political_party.number 13.
-----------------
and the problem is the 13.
The WG spec in development has:
[21] DECIMAL ::= [+-]? [0-9]* '.' [0-9]+
so a decimal must have a trailing digit, and "13." is integer 13
followed by a DOT (terminates the triples).
But in the W3C submission has a know problem in this area:
[18] decimal ::= ('-' | '+')? ( [0-9]+ '.' [0-9]* | '.' ([0-9])+ |
([0-9])+ )
and 13. is ambiguous. Is it 13 and a DOT or a decimal with lexical form
"13." The normal way to tokenize is to choose the longest match (so
":abc" isn't ":a" then "bc") and that means you need a space to the
tokens '13' and DOT
Jena 2.7.4 follows the submission and "13." is a decimal and the needs a
trailing DOT.
In fact, using space-DOT everywhere would be very sensible. Trailing
dots on prefix names may confuse some older parsers.
Jena development (2.10.0) follows the W3C WG spec and it's 13 integer
and a trailing DOT and parses.
Do you have a corrected version of freebase-rdf-2012-12-09-00-00? I
downloaded it but there are other things to fix up before it gets to
that point.
Andy
Re: Need help in loading a large RDF file into JENA TDB using TDBLoader
Posted by Abhishek Shivkumar <ab...@gmail.com>.
Hi Andy,
Here are the triples from the neighborhood of line 270608. i tried
finding the error but couldn't. Do you see any by chance?
I printed the line number too on the left just in case. Ex: "line num
270591-"
------------------------------
line num: 270591- ns:m.01gqn1 ns:type.object.type
ns:organization.organization.
line num: 270592- ns:m.01gqn1 ns:type.object.key
"/wikipedia/en/Worker$0027s_Party_of_Brazil".
line num: 270593- ns:m.01gqn1 ns:type.object.name "Workers' Party"@en.
line num: 270594- ns:m.01gqn1 ns:type.object.name "Partido dos
Trabalhadores"@ca.
line num: 270595- ns:m.01gqn1 ns:type.object.key
"/wikipedia/en/Partido_dos_Trabalhadores".
line num: 270596- ns:m.01gqn1 ns:type.object.name "Partido dos
Trabalhadores"@de.
line num: 270597- ns:m.01gqn1 ns:type.object.key
"/wikipedia/pt_title/Partido_dos_Trabalhadores".
line num: 270598- ns:m.01gqn1 ns:type.object.key
"/wikipedia/it_title/Partito_dei_Lavoratori_$0028Brasile$0029".
line num: 270599- ns:m.01gqn1 ns:type.object.key
"/wikipedia/ja_title/$52B4$50CD$8005$515A_$0028$30D6$30E9$30B8$30EB$0029".
line num: 270600- ns:m.01gqn1
ns:base.braziliangovt.brazilian_political_party.president ns:m.02ql58w.
line num: 270601- ns:m.01gqn1 key:wikipedia.fr_id "742582".
line num: 270602- ns:m.01gqn1 key:wikipedia.ja_id "1747452".
line num: 270603- ns:m.01gqn1 ns:type.object.key "/wikipedia/es_id/281246".
line num: 270604- ns:m.01gqn1 ns:type.object.key
"/wikipedia/en/Brazilian_Worker$0027s_Party".
line num: 270605- ns:m.01gqn1 ns:common.topic.topical_webpage <
http://tse.gov.br/internet/partidos/partidos_politicos/pt.htm>.
line num: 270606- ns:m.01gqn1
ns:base.braziliangovt.brazilian_political_party.number 13.
line num: 270607- ns:m.01gqn1 ns:type.object.key "/en/workers_party".
line num: 270608- ns:m.01gqn1 ns:type.object.key
"/wikipedia/en/Brazilian_Workers_Party".
line num: 270609- ns:m.01gqn1 ns:type.object.type
ns:government.political_party.
line num: 270610- ns:m.01gqn1 ns:common.topic.image ns:m.03s8rh8.
line num: 270611- ns:m.01gqn1 ns:common.topic.official_website <
http://www.pt.org.br/>.
line num: 270612- ns:m.01gqn1 ns:type.object.type ns:business.employer.
line num: 270613- ns:m.01gqn1 ns:type.object.key "/wikipedia/ru_id/1230551".
line num: 270614- ns:m.01gqn1 ns:type.object.key "/wikipedia/en_id/224622".
line num: 270615- ns:m.01gqn1 ns:type.object.type
ns:base.braziliangovt.brazilian_political_party.
line num: 270616- ns:m.01gqn1 ns:type.object.name "Partido de los
Trabajadores"@es.
line num: 270617- ns:m.01gqn1 ns:type.object.key
"/wikipedia/ja/$52B4$50CD$8005$515A_$0028$30D6$30E9$30B8$30EB$0029".
line num: 270618- ns:m.01gqn1 ns:common.topic.article ns:m.01gqn9.
line num: 270619- ns:m.01gqn1 ns:common.topic.webpage ns:m.04yvgzd.
line num: 270620- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://de.wikipedia.org/wiki/Partido_dos_Trabalhadores>.
line num: 270621- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://fr.wikipedia.org/wiki/Parti_des_travailleurs_(Brésil)>.
line num: 270622- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://it.wikipedia.org/wiki/index.html?curid=616917>.
line num: 270623- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://ru.wikipedia.org/wiki/index.html?curid=1230551>.
line num: 270624- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://ja.wikipedia.org/wiki/index.html?curid=1747452>.
line num: 270625- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://de.wikipedia.org/wiki/index.html?curid=408018>.
line num: 270626- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://ru.wikipedia.org/wiki/ПÐ
°Ñ€Ñ‚иÑ?_трудÑ?щихÑ?Ñ?_(БразилиÑ?)>.
line num: 270627- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://es.wikipedia.org/wiki/Partido_de_los_Trabajadores_(Brasil)>.
line num: 270628- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://es.wikipedia.org/wiki/index.html?curid=281246>.
line num: 270629- ns:m.01gqn1 ns:common.topic.topic_equivalent_webpage <
http://it.wikipedia.org/wiki/Partito_dei_Lavoratori_(Brasile)>.
----------------------------
Thank you!
With Regards,
Abhishek S
On Fri, Dec 28, 2012 at 1:15 AM, Andy Seaborne <an...@apache.org> wrote:
> On 27/12/12 19:17, Abhishek Shivkumar wrote:
>
>> Thanks Andy. It runs for few lines and then throws an error like below
>> saying that the triples are not ending with a DOT. I am assuming the file
>> doesn't end every triple with a "." . Is there a work around for this?
>>
>> I read somewhere in the mailing list that this has been fixed in SVN.
>> Anyways, I downloaded the jena from http://www.apache.org/dist/**jena/<http://www.apache.org/dist/jena/>
>>
>
> 1/ the warning for es-419 is fixed.
>
> 2/
> [[
> The Bad IRI: <http://*lv.wikipedia.org/**wiki/Riode<http://lv.wikipedia.org/wiki/Riode>
> ₧aneiro_"Fluminense"**> Code: 4/UNWISE_CHARACTER in PATH: The character
> matches no grammar rules of URIs/IRIs. These characters are permitted in
> RDF URI References, XML system identifiers, and XML Schema anyURIs.
> ]]
>
> Can't have " in URIs. The ₧ looks like an ISO-8859-1/UTF-8 encoding error.
>
> 3/ The file has bad syntax - you need to look at line 270608 or so and fix
> it up. It's a bug in the freebase data.
>
> Shame they haven't fixed it - it was wrong previously as well. It may be
> something like unmatched quotes, or an encoding error, so it may look right
> but isn't.
>
> What are the lines around 270608?
>
>
> (Another good reason for parsing data before loading!)
>
> Andy
>
>
>
>> Thanks much!
>>
>> *00:39:13 INFO loader :: Add: 150,000 triples (Batch:
>> 40,950
>> / Avg*
>> *: 23,648)*
>> *00:39:14 INFO loader :: Add: 200,000 triples (Batch:
>> 43,898
>> / Avg*
>> *: 26,730)*
>> *00:39:14 WARN riot :: [line: 209572, col: 54] Bad IRI:
>> <http://*
>> *lv.wikipedia.org/wiki/Riode₧**aneiro_"Fluminense"> Code:
>> 4/UNWISE_CHARACTER
>> in PAT*
>> *H: The character matches no grammar rules of URIs/IRIs. These characters
>> are per*
>> *mitted in RDF URI References, XML system identifiers, and XML Schema
>> anyURIs.*
>> *00:39:14 WARN riot :: [line: 219452, col: 33] Language
>> not vali*
>> *d: es-419*
>> *00:39:14 WARN riot :: [line: 219560, col: 24] Language
>> not vali*
>> *d: es-419*
>> *00:39:15 INFO loader :: Add: 250,000 triples (Batch:
>> 43,975
>> / Avg*
>> *: 29,005)*
>> *00:39:15 ERROR riot :: [line: 270608, col: 1 ] Triples
>> not
>> termi*
>> *nated by DOT*
>> *Exception in thread "main" org.openjena.riot.**RiotException: [line:
>> 270608,
>> col:*
>> *1 ] Triples not terminated by DOT*
>>
>> * at
>> org.openjena.riot.**ErrorHandlerFactory$**ErrorHandlerStd.fatal(**
>> ErrorHand*
>> *lerFactory.java:130)*
>> * at
>> org.openjena.riot.lang.**LangEngine.raiseException(**
>> LangEngine.java:169)*
>> *
>> *
>> * at
>> org.openjena.riot.lang.**LangEngine.exceptionDirect(**
>> LangEngine.java:162*
>> *)*
>> * at org.openjena.riot.lang.**LangEngine.exception(**
>> LangEngine.java:155)
>> *
>> * at org.openjena.riot.lang.**LangEngine.expect(LangEngine.**
>> java:147)*
>> * at
>> org.openjena.riot.lang.**LangEngine.expectOrEOF(**LangEngine.java:138)*
>> * at
>> org.openjena.riot.lang.**LangTurtle.expectEndOfTriples(**
>> LangTurtle.java:*
>> *57)*
>> * at
>> org.openjena.riot.lang.**LangTurtleBase.triples(**
>> LangTurtleBase.java:285*
>> *)*
>> * at
>> org.openjena.riot.lang.**LangTurtleBase.**triplesSameSubject(**
>> LangTurtleBa*
>> *se.java:223)*
>> * at
>> org.openjena.riot.lang.**LangTurtle.oneTopLevelElement(**
>> LangTurtle.java:*
>> *46)*
>> * at
>> org.openjena.riot.lang.**LangTurtleBase.runParser(**
>> LangTurtleBase.java:1*
>> *44)*
>> * at org.openjena.riot.lang.**LangBase.parse(LangBase.java:**43)*
>> * at org.openjena.riot.RiotReader.**parseTriples(RiotReader.java:*
>> *97)*
>> * at org.openjena.riot.RiotReader.**parseTriples(RiotReader.java:*
>> *83)*
>> * at org.openjena.riot.RiotReader.**parseTriples(RiotReader.java:*
>> *56)*
>> * at
>> com.hp.hpl.jena.tdb.store.**bulkloader.BulkLoader.**
>> loadTriples$(BulkLoad*
>> *er.java:139)*
>> * at
>> com.hp.hpl.jena.tdb.store.**bulkloader.BulkLoader.**
>> loadDefaultGraph(Bulk*
>> *Loader.java:87)*
>> * at
>> com.hp.hpl.jena.tdb.TDBLoader.**loadDefaultGraph$(TDBLoader.**java:261)*
>> * at com.hp.hpl.jena.tdb.TDBLoader.**
>> loadGraph$(TDBLoader.java:244)***
>> * at com.hp.hpl.jena.tdb.TDBLoader.**
>> loadGraph(TDBLoader.java:177)*
>> * at com.hp.hpl.jena.tdb.TDBLoader.**load(TDBLoader.java:112)*
>> * at tdb.tdbloader.**loadDefaultGraph(tdbloader.**java:150)*
>> * at tdb.tdbloader.exec(tdbloader.**java:116)*
>>
>> * at arq.cmdline.CmdMain.**mainMethod(CmdMain.java:101)*
>> * at arq.cmdline.CmdMain.mainRun(**CmdMain.java:63)*
>> * at arq.cmdline.CmdMain.mainRun(**CmdMain.java:50)*
>> * at tdb.tdbloader.main(tdbloader.**java:53)*
>>
>> Thank you!
>>
>> With Regards,
>> Abhishek S
>>
>>
>> On Fri, Dec 28, 2012 at 12:35 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>> On 27/12/12 18:42, Abhishek Shivkumar wrote:
>>>
>>> Hi,
>>>>
>>>> I am trying to load a large (55 GB!) rdf file into JENA TDB for
>>>> sparql
>>>> querying later. Here is a snapshot of the file at the end of this email:
>>>>
>>>> When I am using TDBLoader from command line using the following command:
>>>>
>>>> *c:\JENA\apache-jena-2.7.4\****apache-jena-2.7.4\bat>****tdbloader.bat
>>>> -loc
>>>> test
>>>> "C:\freebase-rdf-2012-12-09-****00-00"*
>>>>
>>>>
>>> The TDB loader has no clue, via file extension, as to the syntax. The
>>> default is n-quads/n-triples.
>>>
>>> But it's turtle, hence a syntax error.
>>>
>>> So either:
>>>
>>> 1/ Run "riotcmd.turtle FILE > data.nt"
>>>
>>> This is preferred because:
>>> A/ It check the file is valid before loading.
>>> B/ The NT loads faster.
>>>
>>> 2/ Rename the file to "something.ttl"
>>>
>>> Andy
>>>
>>>
>>> I get this error:
>>>>
>>>> *23:40:30 INFO loader :: -- Start triples data phase*
>>>> *23:40:30 INFO loader :: ** Load empty triples table*
>>>> *23:40:30 INFO loader :: -- Start quads data phase*
>>>> *23:40:30 INFO loader :: ** Load empty quads table*
>>>> *23:40:30 INFO loader :: Load: C:\Users\IBM_ADMIN\My
>>>> Documents\dow*
>>>> *n\freebase-rdf-2012-12-09-00-****00\freebase-rdf-2012-12-09-**00-**00
>>>> --
>>>>
>>>> 2012/12/27 23:4*
>>>> *0:30 IST*
>>>> *23:40:30 ERROR riot :: [line: 1, col: 1 ] Expected
>>>> BNode
>>>>
>>>> or IRI:*
>>>> * Got: [DIRECTIVE:prefix]*
>>>> *Exception in thread "main" org.openjena.riot.****RiotException:
>>>> [line: 1,
>>>>
>>>> col:
>>>>
>>>> 1 ] E*
>>>> *xpected BNode or IRI: Got: [DIRECTIVE:prefix]*
>>>> * at
>>>> org.openjena.riot.****ErrorHandlerFactory$****ErrorHandlerStd.fatal(**
>>>> ErrorHand*
>>>> *lerFactory.java:130)*
>>>> * at
>>>> org.openjena.riot.lang.****LangEngine.raiseException(**
>>>>
>>>> LangEngine.java:169)*
>>>> *
>>>> *
>>>> * at
>>>> org.openjena.riot.lang.****LangEngine.exceptionDirect(**
>>>> LangEngine.java:162*
>>>> *)*
>>>> * at org.openjena.riot.lang.****LangEngine.exception(**
>>>> LangEngine.java:155)
>>>> *
>>>> * at
>>>> org.openjena.riot.lang.****LangNTuple.checkIRIOrBNode(**
>>>> LangNTuple.java:107*
>>>> *)*
>>>> * at org.openjena.riot.lang.****LangNQuads.parseOne(**
>>>> LangNQuads.java:84)*
>>>> * at org.openjena.riot.lang.****LangNQuads.parseOne(**
>>>> LangNQuads.java:34)*
>>>> * at org.openjena.riot.lang.****LangNTuple.runParser(**
>>>> LangNTuple.java:69)*
>>>> * at org.openjena.riot.lang.****LangBase.parse(LangBase.java:***
>>>> *43)*
>>>> * at org.openjena.riot.RiotReader.***
>>>> *parseQuads(RiotReader.java:**
>>>> 134)*
>>>> * at org.openjena.riot.RiotReader.***
>>>> *parseQuads(RiotReader.java:**
>>>> 121)*
>>>> * at org.openjena.riot.RiotReader.***
>>>> *parseQuads(RiotReader.java:**
>>>> 107)*
>>>> * at
>>>> com.hp.hpl.jena.tdb.store.****bulkloader.BulkLoader.**
>>>> loadQuads$(BulkLoader*
>>>> *.java:160)*
>>>> * at
>>>> com.hp.hpl.jena.tdb.store.****bulkloader.BulkLoader.**
>>>> loadDataset(BulkLoade*
>>>> *r.java:121)*
>>>> * at com.hp.hpl.jena.tdb.TDBLoader.**
>>>> **loadDataset$(TDBLoader.java:***
>>>> *283)*
>>>> * at com.hp.hpl.jena.tdb.TDBLoader.**
>>>> **loadDataset(TDBLoader.java:****
>>>> 196)*
>>>> * at com.hp.hpl.jena.tdb.TDBLoader.****load(TDBLoader.java:75)*
>>>> * at tdb.tdbloader.loadQuads(****tdbloader.java:163)*
>>>> * at tdb.tdbloader.exec(tdbloader.****java:122)*
>>>> * at arq.cmdline.CmdMain.****mainMethod(CmdMain.java:101)*
>>>> * at arq.cmdline.CmdMain.mainRun(****CmdMain.java:63)*
>>>> * at arq.cmdline.CmdMain.mainRun(****CmdMain.java:50)*
>>>> * at tdb.tdbloader.main(tdbloader.****java:53)*
>>>>
>>>>
>>>>
>>>> I need help in understanding this error and how to solve it. Is there a
>>>> problem with the input file?
>>>>
>>>>
>>>> @prefix ns: <http://rdf.freebase.com/ns/>.
>>>> @prefix key: <http://rdf.freebase.com/key/>****.
>>>> @prefix owl: <http://www.w3.org/2002/07/****owl#<http://www.w3.org/2002/07/**owl#>
>>>> <http://www.w3.org/2002/**07/owl# <http://www.w3.org/2002/07/owl#>>
>>>>
>>>>> .
>>>>>
>>>> @prefix rdfs: <http://www.w3.org/2000/01/****rdf-schema#<http://www.w3.org/2000/01/**rdf-schema#>
>>>> <http://www.w3.org/**2000/01/rdf-schema#<http://www.w3.org/2000/01/rdf-schema#>
>>>> >
>>>>
>>>>> .
>>>>>
>>>> @prefix xsd: <http://www.w3.org/2001/****XMLSchema#<http://www.w3.org/2001/**XMLSchema#>
>>>> <http://www.w3.org/**2001/XMLSchema#<http://www.w3.org/2001/XMLSchema#>
>>>> >
>>>>
>>>> .
>>>>>
>>>>
>>>> ns:m.012rkqx ns:type.object.type ns:common.topic.
>>>> ns:m.012rkqx ns:type.object.name "High Fidelity"@en.
>>>> ns:m.012rkqx ns:type.object.type ns:music.single.
>>>> ns:m.012rkqx ns:type.object.key ns:authority.musicbrainz.name.*
>>>> ***
>>>>
>>>> TRACK3987054.
>>>> ns:m.012rkqx ns:type.object.type ns:music.recording.
>>>> ns:m.012rkqx key:authority.musicbrainz "258c45bd-
>>>> 4437-4580-8988
>>>> -b3f3be975f9c".
>>>> ns:m.012rkqx key:authority.musicbrainz.name "TRACK3987054".
>>>> ns:m.012rkqx rdf:label "High Fidelity"@en.
>>>> ns:m.012rkqx rdf:type ns:common.topic.
>>>> ns:m.012rkqx rdf:type ns:music.single.
>>>> ns:m.012rkqx rdf:type ns:music.recording.
>>>>
>>>>
>>>>
>>>
>>
>
Re: Need help in loading a large RDF file into JENA TDB using TDBLoader
Posted by Andy Seaborne <an...@apache.org>.
On 27/12/12 19:17, Abhishek Shivkumar wrote:
> Thanks Andy. It runs for few lines and then throws an error like below
> saying that the triples are not ending with a DOT. I am assuming the file
> doesn't end every triple with a "." . Is there a work around for this?
>
> I read somewhere in the mailing list that this has been fixed in SVN.
> Anyways, I downloaded the jena from http://www.apache.org/dist/jena/
1/ the warning for es-419 is fixed.
2/
[[
The Bad IRI: <http://*lv.wikipedia.org/wiki/Riode₧aneiro_"Fluminense">
Code: 4/UNWISE_CHARACTER in PATH: The character matches no grammar rules
of URIs/IRIs. These characters are permitted in RDF URI References, XML
system identifiers, and XML Schema anyURIs.
]]
Can't have " in URIs. The ₧ looks like an ISO-8859-1/UTF-8 encoding error.
3/ The file has bad syntax - you need to look at line 270608 or so and
fix it up. It's a bug in the freebase data.
Shame they haven't fixed it - it was wrong previously as well. It may
be something like unmatched quotes, or an encoding error, so it may look
right but isn't.
What are the lines around 270608?
(Another good reason for parsing data before loading!)
Andy
>
> Thanks much!
>
> *00:39:13 INFO loader :: Add: 150,000 triples (Batch: 40,950
> / Avg*
> *: 23,648)*
> *00:39:14 INFO loader :: Add: 200,000 triples (Batch: 43,898
> / Avg*
> *: 26,730)*
> *00:39:14 WARN riot :: [line: 209572, col: 54] Bad IRI:
> <http://*
> *lv.wikipedia.org/wiki/Riode₧aneiro_"Fluminense"> Code: 4/UNWISE_CHARACTER
> in PAT*
> *H: The character matches no grammar rules of URIs/IRIs. These characters
> are per*
> *mitted in RDF URI References, XML system identifiers, and XML Schema
> anyURIs.*
> *00:39:14 WARN riot :: [line: 219452, col: 33] Language
> not vali*
> *d: es-419*
> *00:39:14 WARN riot :: [line: 219560, col: 24] Language
> not vali*
> *d: es-419*
> *00:39:15 INFO loader :: Add: 250,000 triples (Batch: 43,975
> / Avg*
> *: 29,005)*
> *00:39:15 ERROR riot :: [line: 270608, col: 1 ] Triples not
> termi*
> *nated by DOT*
> *Exception in thread "main" org.openjena.riot.RiotException: [line: 270608,
> col:*
> *1 ] Triples not terminated by DOT*
> * at
> org.openjena.riot.ErrorHandlerFactory$ErrorHandlerStd.fatal(ErrorHand*
> *lerFactory.java:130)*
> * at
> org.openjena.riot.lang.LangEngine.raiseException(LangEngine.java:169)*
> *
> *
> * at
> org.openjena.riot.lang.LangEngine.exceptionDirect(LangEngine.java:162*
> *)*
> * at org.openjena.riot.lang.LangEngine.exception(LangEngine.java:155)
> *
> * at org.openjena.riot.lang.LangEngine.expect(LangEngine.java:147)*
> * at
> org.openjena.riot.lang.LangEngine.expectOrEOF(LangEngine.java:138)*
> * at
> org.openjena.riot.lang.LangTurtle.expectEndOfTriples(LangTurtle.java:*
> *57)*
> * at
> org.openjena.riot.lang.LangTurtleBase.triples(LangTurtleBase.java:285*
> *)*
> * at
> org.openjena.riot.lang.LangTurtleBase.triplesSameSubject(LangTurtleBa*
> *se.java:223)*
> * at
> org.openjena.riot.lang.LangTurtle.oneTopLevelElement(LangTurtle.java:*
> *46)*
> * at
> org.openjena.riot.lang.LangTurtleBase.runParser(LangTurtleBase.java:1*
> *44)*
> * at org.openjena.riot.lang.LangBase.parse(LangBase.java:43)*
> * at org.openjena.riot.RiotReader.parseTriples(RiotReader.java:97)*
> * at org.openjena.riot.RiotReader.parseTriples(RiotReader.java:83)*
> * at org.openjena.riot.RiotReader.parseTriples(RiotReader.java:56)*
> * at
> com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadTriples$(BulkLoad*
> *er.java:139)*
> * at
> com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadDefaultGraph(Bulk*
> *Loader.java:87)*
> * at
> com.hp.hpl.jena.tdb.TDBLoader.loadDefaultGraph$(TDBLoader.java:261)*
> * at com.hp.hpl.jena.tdb.TDBLoader.loadGraph$(TDBLoader.java:244)*
> * at com.hp.hpl.jena.tdb.TDBLoader.loadGraph(TDBLoader.java:177)*
> * at com.hp.hpl.jena.tdb.TDBLoader.load(TDBLoader.java:112)*
> * at tdb.tdbloader.loadDefaultGraph(tdbloader.java:150)*
> * at tdb.tdbloader.exec(tdbloader.java:116)*
> * at arq.cmdline.CmdMain.mainMethod(CmdMain.java:101)*
> * at arq.cmdline.CmdMain.mainRun(CmdMain.java:63)*
> * at arq.cmdline.CmdMain.mainRun(CmdMain.java:50)*
> * at tdb.tdbloader.main(tdbloader.java:53)*
>
> Thank you!
>
> With Regards,
> Abhishek S
>
>
> On Fri, Dec 28, 2012 at 12:35 AM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 27/12/12 18:42, Abhishek Shivkumar wrote:
>>
>>> Hi,
>>>
>>> I am trying to load a large (55 GB!) rdf file into JENA TDB for sparql
>>> querying later. Here is a snapshot of the file at the end of this email:
>>>
>>> When I am using TDBLoader from command line using the following command:
>>>
>>> *c:\JENA\apache-jena-2.7.4\**apache-jena-2.7.4\bat>**tdbloader.bat -loc
>>> test
>>> "C:\freebase-rdf-2012-12-09-**00-00"*
>>>
>>
>> The TDB loader has no clue, via file extension, as to the syntax. The
>> default is n-quads/n-triples.
>>
>> But it's turtle, hence a syntax error.
>>
>> So either:
>>
>> 1/ Run "riotcmd.turtle FILE > data.nt"
>>
>> This is preferred because:
>> A/ It check the file is valid before loading.
>> B/ The NT loads faster.
>>
>> 2/ Rename the file to "something.ttl"
>>
>> Andy
>>
>>
>>> I get this error:
>>>
>>> *23:40:30 INFO loader :: -- Start triples data phase*
>>> *23:40:30 INFO loader :: ** Load empty triples table*
>>> *23:40:30 INFO loader :: -- Start quads data phase*
>>> *23:40:30 INFO loader :: ** Load empty quads table*
>>> *23:40:30 INFO loader :: Load: C:\Users\IBM_ADMIN\My
>>> Documents\dow*
>>> *n\freebase-rdf-2012-12-09-00-**00\freebase-rdf-2012-12-09-00-**00 --
>>> 2012/12/27 23:4*
>>> *0:30 IST*
>>> *23:40:30 ERROR riot :: [line: 1, col: 1 ] Expected BNode
>>>
>>> or IRI:*
>>> * Got: [DIRECTIVE:prefix]*
>>> *Exception in thread "main" org.openjena.riot.**RiotException: [line: 1,
>>> col:
>>>
>>> 1 ] E*
>>> *xpected BNode or IRI: Got: [DIRECTIVE:prefix]*
>>> * at
>>> org.openjena.riot.**ErrorHandlerFactory$**ErrorHandlerStd.fatal(**
>>> ErrorHand*
>>> *lerFactory.java:130)*
>>> * at
>>> org.openjena.riot.lang.**LangEngine.raiseException(**
>>> LangEngine.java:169)*
>>> *
>>> *
>>> * at
>>> org.openjena.riot.lang.**LangEngine.exceptionDirect(**
>>> LangEngine.java:162*
>>> *)*
>>> * at org.openjena.riot.lang.**LangEngine.exception(**
>>> LangEngine.java:155)
>>> *
>>> * at
>>> org.openjena.riot.lang.**LangNTuple.checkIRIOrBNode(**
>>> LangNTuple.java:107*
>>> *)*
>>> * at org.openjena.riot.lang.**LangNQuads.parseOne(**
>>> LangNQuads.java:84)*
>>> * at org.openjena.riot.lang.**LangNQuads.parseOne(**
>>> LangNQuads.java:34)*
>>> * at org.openjena.riot.lang.**LangNTuple.runParser(**
>>> LangNTuple.java:69)*
>>> * at org.openjena.riot.lang.**LangBase.parse(LangBase.java:**43)*
>>> * at org.openjena.riot.RiotReader.**parseQuads(RiotReader.java:**
>>> 134)*
>>> * at org.openjena.riot.RiotReader.**parseQuads(RiotReader.java:**
>>> 121)*
>>> * at org.openjena.riot.RiotReader.**parseQuads(RiotReader.java:**
>>> 107)*
>>> * at
>>> com.hp.hpl.jena.tdb.store.**bulkloader.BulkLoader.**
>>> loadQuads$(BulkLoader*
>>> *.java:160)*
>>> * at
>>> com.hp.hpl.jena.tdb.store.**bulkloader.BulkLoader.**
>>> loadDataset(BulkLoade*
>>> *r.java:121)*
>>> * at com.hp.hpl.jena.tdb.TDBLoader.**loadDataset$(TDBLoader.java:*
>>> *283)*
>>> * at com.hp.hpl.jena.tdb.TDBLoader.**loadDataset(TDBLoader.java:**
>>> 196)*
>>> * at com.hp.hpl.jena.tdb.TDBLoader.**load(TDBLoader.java:75)*
>>> * at tdb.tdbloader.loadQuads(**tdbloader.java:163)*
>>> * at tdb.tdbloader.exec(tdbloader.**java:122)*
>>> * at arq.cmdline.CmdMain.**mainMethod(CmdMain.java:101)*
>>> * at arq.cmdline.CmdMain.mainRun(**CmdMain.java:63)*
>>> * at arq.cmdline.CmdMain.mainRun(**CmdMain.java:50)*
>>> * at tdb.tdbloader.main(tdbloader.**java:53)*
>>>
>>>
>>> I need help in understanding this error and how to solve it. Is there a
>>> problem with the input file?
>>>
>>>
>>> @prefix ns: <http://rdf.freebase.com/ns/>.
>>> @prefix key: <http://rdf.freebase.com/key/>**.
>>> @prefix owl: <http://www.w3.org/2002/07/**owl#<http://www.w3.org/2002/07/owl#>
>>>> .
>>> @prefix rdfs: <http://www.w3.org/2000/01/**rdf-schema#<http://www.w3.org/2000/01/rdf-schema#>
>>>> .
>>> @prefix xsd: <http://www.w3.org/2001/**XMLSchema#<http://www.w3.org/2001/XMLSchema#>
>>>> .
>>>
>>> ns:m.012rkqx ns:type.object.type ns:common.topic.
>>> ns:m.012rkqx ns:type.object.name "High Fidelity"@en.
>>> ns:m.012rkqx ns:type.object.type ns:music.single.
>>> ns:m.012rkqx ns:type.object.key ns:authority.musicbrainz.name.**
>>> TRACK3987054.
>>> ns:m.012rkqx ns:type.object.type ns:music.recording.
>>> ns:m.012rkqx key:authority.musicbrainz "258c45bd-4437-4580-8988
>>> -b3f3be975f9c".
>>> ns:m.012rkqx key:authority.musicbrainz.name "TRACK3987054".
>>> ns:m.012rkqx rdf:label "High Fidelity"@en.
>>> ns:m.012rkqx rdf:type ns:common.topic.
>>> ns:m.012rkqx rdf:type ns:music.single.
>>> ns:m.012rkqx rdf:type ns:music.recording.
>>>
>>>
>>
>
Re: Need help in loading a large RDF file into JENA TDB using TDBLoader
Posted by Abhishek Shivkumar <ab...@gmail.com>.
Thanks Andy. It runs for few lines and then throws an error like below
saying that the triples are not ending with a DOT. I am assuming the file
doesn't end every triple with a "." . Is there a work around for this?
I read somewhere in the mailing list that this has been fixed in SVN.
Anyways, I downloaded the jena from http://www.apache.org/dist/jena/
Thanks much!
*00:39:13 INFO loader :: Add: 150,000 triples (Batch: 40,950
/ Avg*
*: 23,648)*
*00:39:14 INFO loader :: Add: 200,000 triples (Batch: 43,898
/ Avg*
*: 26,730)*
*00:39:14 WARN riot :: [line: 209572, col: 54] Bad IRI:
<http://*
*lv.wikipedia.org/wiki/Riode₧aneiro_"Fluminense"> Code: 4/UNWISE_CHARACTER
in PAT*
*H: The character matches no grammar rules of URIs/IRIs. These characters
are per*
*mitted in RDF URI References, XML system identifiers, and XML Schema
anyURIs.*
*00:39:14 WARN riot :: [line: 219452, col: 33] Language
not vali*
*d: es-419*
*00:39:14 WARN riot :: [line: 219560, col: 24] Language
not vali*
*d: es-419*
*00:39:15 INFO loader :: Add: 250,000 triples (Batch: 43,975
/ Avg*
*: 29,005)*
*00:39:15 ERROR riot :: [line: 270608, col: 1 ] Triples not
termi*
*nated by DOT*
*Exception in thread "main" org.openjena.riot.RiotException: [line: 270608,
col:*
*1 ] Triples not terminated by DOT*
* at
org.openjena.riot.ErrorHandlerFactory$ErrorHandlerStd.fatal(ErrorHand*
*lerFactory.java:130)*
* at
org.openjena.riot.lang.LangEngine.raiseException(LangEngine.java:169)*
*
*
* at
org.openjena.riot.lang.LangEngine.exceptionDirect(LangEngine.java:162*
*)*
* at org.openjena.riot.lang.LangEngine.exception(LangEngine.java:155)
*
* at org.openjena.riot.lang.LangEngine.expect(LangEngine.java:147)*
* at
org.openjena.riot.lang.LangEngine.expectOrEOF(LangEngine.java:138)*
* at
org.openjena.riot.lang.LangTurtle.expectEndOfTriples(LangTurtle.java:*
*57)*
* at
org.openjena.riot.lang.LangTurtleBase.triples(LangTurtleBase.java:285*
*)*
* at
org.openjena.riot.lang.LangTurtleBase.triplesSameSubject(LangTurtleBa*
*se.java:223)*
* at
org.openjena.riot.lang.LangTurtle.oneTopLevelElement(LangTurtle.java:*
*46)*
* at
org.openjena.riot.lang.LangTurtleBase.runParser(LangTurtleBase.java:1*
*44)*
* at org.openjena.riot.lang.LangBase.parse(LangBase.java:43)*
* at org.openjena.riot.RiotReader.parseTriples(RiotReader.java:97)*
* at org.openjena.riot.RiotReader.parseTriples(RiotReader.java:83)*
* at org.openjena.riot.RiotReader.parseTriples(RiotReader.java:56)*
* at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadTriples$(BulkLoad*
*er.java:139)*
* at
com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadDefaultGraph(Bulk*
*Loader.java:87)*
* at
com.hp.hpl.jena.tdb.TDBLoader.loadDefaultGraph$(TDBLoader.java:261)*
* at com.hp.hpl.jena.tdb.TDBLoader.loadGraph$(TDBLoader.java:244)*
* at com.hp.hpl.jena.tdb.TDBLoader.loadGraph(TDBLoader.java:177)*
* at com.hp.hpl.jena.tdb.TDBLoader.load(TDBLoader.java:112)*
* at tdb.tdbloader.loadDefaultGraph(tdbloader.java:150)*
* at tdb.tdbloader.exec(tdbloader.java:116)*
* at arq.cmdline.CmdMain.mainMethod(CmdMain.java:101)*
* at arq.cmdline.CmdMain.mainRun(CmdMain.java:63)*
* at arq.cmdline.CmdMain.mainRun(CmdMain.java:50)*
* at tdb.tdbloader.main(tdbloader.java:53)*
Thank you!
With Regards,
Abhishek S
On Fri, Dec 28, 2012 at 12:35 AM, Andy Seaborne <an...@apache.org> wrote:
> On 27/12/12 18:42, Abhishek Shivkumar wrote:
>
>> Hi,
>>
>> I am trying to load a large (55 GB!) rdf file into JENA TDB for sparql
>> querying later. Here is a snapshot of the file at the end of this email:
>>
>> When I am using TDBLoader from command line using the following command:
>>
>> *c:\JENA\apache-jena-2.7.4\**apache-jena-2.7.4\bat>**tdbloader.bat -loc
>> test
>> "C:\freebase-rdf-2012-12-09-**00-00"*
>>
>
> The TDB loader has no clue, via file extension, as to the syntax. The
> default is n-quads/n-triples.
>
> But it's turtle, hence a syntax error.
>
> So either:
>
> 1/ Run "riotcmd.turtle FILE > data.nt"
>
> This is preferred because:
> A/ It check the file is valid before loading.
> B/ The NT loads faster.
>
> 2/ Rename the file to "something.ttl"
>
> Andy
>
>
>> I get this error:
>>
>> *23:40:30 INFO loader :: -- Start triples data phase*
>> *23:40:30 INFO loader :: ** Load empty triples table*
>> *23:40:30 INFO loader :: -- Start quads data phase*
>> *23:40:30 INFO loader :: ** Load empty quads table*
>> *23:40:30 INFO loader :: Load: C:\Users\IBM_ADMIN\My
>> Documents\dow*
>> *n\freebase-rdf-2012-12-09-00-**00\freebase-rdf-2012-12-09-00-**00 --
>> 2012/12/27 23:4*
>> *0:30 IST*
>> *23:40:30 ERROR riot :: [line: 1, col: 1 ] Expected BNode
>>
>> or IRI:*
>> * Got: [DIRECTIVE:prefix]*
>> *Exception in thread "main" org.openjena.riot.**RiotException: [line: 1,
>> col:
>>
>> 1 ] E*
>> *xpected BNode or IRI: Got: [DIRECTIVE:prefix]*
>> * at
>> org.openjena.riot.**ErrorHandlerFactory$**ErrorHandlerStd.fatal(**
>> ErrorHand*
>> *lerFactory.java:130)*
>> * at
>> org.openjena.riot.lang.**LangEngine.raiseException(**
>> LangEngine.java:169)*
>> *
>> *
>> * at
>> org.openjena.riot.lang.**LangEngine.exceptionDirect(**
>> LangEngine.java:162*
>> *)*
>> * at org.openjena.riot.lang.**LangEngine.exception(**
>> LangEngine.java:155)
>> *
>> * at
>> org.openjena.riot.lang.**LangNTuple.checkIRIOrBNode(**
>> LangNTuple.java:107*
>> *)*
>> * at org.openjena.riot.lang.**LangNQuads.parseOne(**
>> LangNQuads.java:84)*
>> * at org.openjena.riot.lang.**LangNQuads.parseOne(**
>> LangNQuads.java:34)*
>> * at org.openjena.riot.lang.**LangNTuple.runParser(**
>> LangNTuple.java:69)*
>> * at org.openjena.riot.lang.**LangBase.parse(LangBase.java:**43)*
>> * at org.openjena.riot.RiotReader.**parseQuads(RiotReader.java:**
>> 134)*
>> * at org.openjena.riot.RiotReader.**parseQuads(RiotReader.java:**
>> 121)*
>> * at org.openjena.riot.RiotReader.**parseQuads(RiotReader.java:**
>> 107)*
>> * at
>> com.hp.hpl.jena.tdb.store.**bulkloader.BulkLoader.**
>> loadQuads$(BulkLoader*
>> *.java:160)*
>> * at
>> com.hp.hpl.jena.tdb.store.**bulkloader.BulkLoader.**
>> loadDataset(BulkLoade*
>> *r.java:121)*
>> * at com.hp.hpl.jena.tdb.TDBLoader.**loadDataset$(TDBLoader.java:*
>> *283)*
>> * at com.hp.hpl.jena.tdb.TDBLoader.**loadDataset(TDBLoader.java:**
>> 196)*
>> * at com.hp.hpl.jena.tdb.TDBLoader.**load(TDBLoader.java:75)*
>> * at tdb.tdbloader.loadQuads(**tdbloader.java:163)*
>> * at tdb.tdbloader.exec(tdbloader.**java:122)*
>> * at arq.cmdline.CmdMain.**mainMethod(CmdMain.java:101)*
>> * at arq.cmdline.CmdMain.mainRun(**CmdMain.java:63)*
>> * at arq.cmdline.CmdMain.mainRun(**CmdMain.java:50)*
>> * at tdb.tdbloader.main(tdbloader.**java:53)*
>>
>>
>> I need help in understanding this error and how to solve it. Is there a
>> problem with the input file?
>>
>>
>> @prefix ns: <http://rdf.freebase.com/ns/>.
>> @prefix key: <http://rdf.freebase.com/key/>**.
>> @prefix owl: <http://www.w3.org/2002/07/**owl#<http://www.w3.org/2002/07/owl#>
>> >.
>> @prefix rdfs: <http://www.w3.org/2000/01/**rdf-schema#<http://www.w3.org/2000/01/rdf-schema#>
>> >.
>> @prefix xsd: <http://www.w3.org/2001/**XMLSchema#<http://www.w3.org/2001/XMLSchema#>
>> >.
>>
>> ns:m.012rkqx ns:type.object.type ns:common.topic.
>> ns:m.012rkqx ns:type.object.name "High Fidelity"@en.
>> ns:m.012rkqx ns:type.object.type ns:music.single.
>> ns:m.012rkqx ns:type.object.key ns:authority.musicbrainz.name.**
>> TRACK3987054.
>> ns:m.012rkqx ns:type.object.type ns:music.recording.
>> ns:m.012rkqx key:authority.musicbrainz "258c45bd-4437-4580-8988
>> -b3f3be975f9c".
>> ns:m.012rkqx key:authority.musicbrainz.name "TRACK3987054".
>> ns:m.012rkqx rdf:label "High Fidelity"@en.
>> ns:m.012rkqx rdf:type ns:common.topic.
>> ns:m.012rkqx rdf:type ns:music.single.
>> ns:m.012rkqx rdf:type ns:music.recording.
>>
>>
>
Re: Need help in loading a large RDF file into JENA TDB using TDBLoader
Posted by Andy Seaborne <an...@apache.org>.
On 27/12/12 18:42, Abhishek Shivkumar wrote:
> Hi,
>
> I am trying to load a large (55 GB!) rdf file into JENA TDB for sparql
> querying later. Here is a snapshot of the file at the end of this email:
>
> When I am using TDBLoader from command line using the following command:
>
> *c:\JENA\apache-jena-2.7.4\apache-jena-2.7.4\bat>tdbloader.bat -loc test
> "C:\freebase-rdf-2012-12-09-00-00"*
The TDB loader has no clue, via file extension, as to the syntax. The
default is n-quads/n-triples.
But it's turtle, hence a syntax error.
So either:
1/ Run "riotcmd.turtle FILE > data.nt"
This is preferred because:
A/ It check the file is valid before loading.
B/ The NT loads faster.
2/ Rename the file to "something.ttl"
Andy
>
> I get this error:
>
> *23:40:30 INFO loader :: -- Start triples data phase*
> *23:40:30 INFO loader :: ** Load empty triples table*
> *23:40:30 INFO loader :: -- Start quads data phase*
> *23:40:30 INFO loader :: ** Load empty quads table*
> *23:40:30 INFO loader :: Load: C:\Users\IBM_ADMIN\My
> Documents\dow*
> *n\freebase-rdf-2012-12-09-00-00\freebase-rdf-2012-12-09-00-00 --
> 2012/12/27 23:4*
> *0:30 IST*
> *23:40:30 ERROR riot :: [line: 1, col: 1 ] Expected BNode
> or IRI:*
> * Got: [DIRECTIVE:prefix]*
> *Exception in thread "main" org.openjena.riot.RiotException: [line: 1, col:
> 1 ] E*
> *xpected BNode or IRI: Got: [DIRECTIVE:prefix]*
> * at
> org.openjena.riot.ErrorHandlerFactory$ErrorHandlerStd.fatal(ErrorHand*
> *lerFactory.java:130)*
> * at
> org.openjena.riot.lang.LangEngine.raiseException(LangEngine.java:169)*
> *
> *
> * at
> org.openjena.riot.lang.LangEngine.exceptionDirect(LangEngine.java:162*
> *)*
> * at org.openjena.riot.lang.LangEngine.exception(LangEngine.java:155)
> *
> * at
> org.openjena.riot.lang.LangNTuple.checkIRIOrBNode(LangNTuple.java:107*
> *)*
> * at org.openjena.riot.lang.LangNQuads.parseOne(LangNQuads.java:84)*
> * at org.openjena.riot.lang.LangNQuads.parseOne(LangNQuads.java:34)*
> * at org.openjena.riot.lang.LangNTuple.runParser(LangNTuple.java:69)*
> * at org.openjena.riot.lang.LangBase.parse(LangBase.java:43)*
> * at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:134)*
> * at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:121)*
> * at org.openjena.riot.RiotReader.parseQuads(RiotReader.java:107)*
> * at
> com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadQuads$(BulkLoader*
> *.java:160)*
> * at
> com.hp.hpl.jena.tdb.store.bulkloader.BulkLoader.loadDataset(BulkLoade*
> *r.java:121)*
> * at com.hp.hpl.jena.tdb.TDBLoader.loadDataset$(TDBLoader.java:283)*
> * at com.hp.hpl.jena.tdb.TDBLoader.loadDataset(TDBLoader.java:196)*
> * at com.hp.hpl.jena.tdb.TDBLoader.load(TDBLoader.java:75)*
> * at tdb.tdbloader.loadQuads(tdbloader.java:163)*
> * at tdb.tdbloader.exec(tdbloader.java:122)*
> * at arq.cmdline.CmdMain.mainMethod(CmdMain.java:101)*
> * at arq.cmdline.CmdMain.mainRun(CmdMain.java:63)*
> * at arq.cmdline.CmdMain.mainRun(CmdMain.java:50)*
> * at tdb.tdbloader.main(tdbloader.java:53)*
>
>
> I need help in understanding this error and how to solve it. Is there a
> problem with the input file?
>
>
> @prefix ns: <http://rdf.freebase.com/ns/>.
> @prefix key: <http://rdf.freebase.com/key/>.
> @prefix owl: <http://www.w3.org/2002/07/owl#>.
> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
> @prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
>
> ns:m.012rkqx ns:type.object.type ns:common.topic.
> ns:m.012rkqx ns:type.object.name "High Fidelity"@en.
> ns:m.012rkqx ns:type.object.type ns:music.single.
> ns:m.012rkqx ns:type.object.key ns:authority.musicbrainz.name.TRACK3987054.
> ns:m.012rkqx ns:type.object.type ns:music.recording.
> ns:m.012rkqx key:authority.musicbrainz "258c45bd-4437-4580-8988-b3f3be975f9c".
> ns:m.012rkqx key:authority.musicbrainz.name "TRACK3987054".
> ns:m.012rkqx rdf:label "High Fidelity"@en.
> ns:m.012rkqx rdf:type ns:common.topic.
> ns:m.012rkqx rdf:type ns:music.single.
> ns:m.012rkqx rdf:type ns:music.recording.
>