You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Glenn Eriksson <an...@hotmail.com> on 2019/04/08 13:13:02 UTC
Migrate CVS data into RDF
Hi,
I am a Jena newbie and done a couple of example and tutorials. Now that actually have a use-case for Jean I've got stuck converting CVS into RDF. CVS is a common format within transport sector and public transportation. I've noticed that the jena-cvs library has been retired (jena 3.9). I don't want to use third-party tools (cmd) or library and I want control of the process.
Use case: I want to migrate CVS data in GTFS (General Transit Feed Specification) format into RDF
* I have a ontology of GTFS in Turtle format (gtfs.ttl)
* There are 12-13 CVS files with data that I want to merge (one at a time) with the GTFS model
* Store the migrated data in a triple store for further processing
My questions/problem are:
1. Is it possible to use the retired jena-csv module to merge CVS data and GTFS. I don't want to use R2RML if I can avoid it since it's quite verbose?
2. When I use Jean 3.9 in my pom.xml and build with dependencies the JAR file for jena-csv is missing. So I haven't been able test GraphCSV and merge it with the loaded GTFS model. Where can I find the jar file for jena-csv 3.9?
Sorry if this is a rudimentary procedure and question, point me in the right direction if my approach is wrong.
Greetings Glenn
PS. I have read these jena-csv<https://lists.apache.org/thread.html/81dfedb090089a15275d2b83bd676b03eb141f0d92c96d6d6d455c29@%3Cusers.jena.apache.org%3E>, CSV to rdf<https://lists.apache.org/thread.html/ba053bf52d30285bb447337ac4518110bcec3a6bf035b4f708e67037@%3Cusers.jena.apache.org%3E> in the mail archive. I have found the source code for jena-csv but not the the jar.
Sv: Sv: Migrate CVS data into RDF
Posted by Glenn Eriksson <an...@hotmail.com>.
Thanks again Adrian for the input.
It's an exercise to get more familiar with Jena and apply Inference to exploit implicit information from real-world data. That's why I was interested in jena-csv since it was part of the Jena libraries and seamed straight forward to use and be in control of the process. Maybe there was a reason Apache put it to sleep.
Regards Glenn
________________________________
Från: Adrian Gschwend <ml...@netlabs.org>
Skickat: den 9 april 2019 10:22
Till: users@jena.apache.org
Ämne: Re: Sv: Migrate CVS data into RDF
On 09.04.19 09:19, Glenn Eriksson wrote:
> I've been reading about the R2RML Ontop API for Java. But to my
> limited knowledge it seams geared towards queering and transforming
> relational DB (JDBC) to RFD and not trough merging tabular data with
> RDF model.
Most R2RML implementations work fine with CSV as well but you can also
use RML.
We have an XText based DSL for R2RML/RML so writing the mapping files is
far less work. It's available on Github, using Eclipse DSL. But it's not
much documented yet so you might have to play around with it.
https://github.com/mchlrch/experimental-rmdsl
I use this for really large mappings and it works flawlessly.
Other Java based options:
https://tarql.github.io/
https://github.com/carml/carml
carml is RML, but a much faster implementation than the reference
implementation of RML.
another Java R2RML implementation which I did not try myself:
https://github.com/nkons/r2rml-parser
But out of curiosity, you want GTFS in RDF or you want to know what it
means to get there? Because I personally really try to avoid writing
stuff other people already solved unless I have a good reason to do so :)
regards
Adrian
Re: Sv: Migrate CVS data into RDF
Posted by Adrian Gschwend <ml...@netlabs.org>.
On 09.04.19 09:19, Glenn Eriksson wrote:
> I've been reading about the R2RML Ontop API for Java. But to my
> limited knowledge it seams geared towards queering and transforming
> relational DB (JDBC) to RFD and not trough merging tabular data with
> RDF model.
Most R2RML implementations work fine with CSV as well but you can also
use RML.
We have an XText based DSL for R2RML/RML so writing the mapping files is
far less work. It's available on Github, using Eclipse DSL. But it's not
much documented yet so you might have to play around with it.
https://github.com/mchlrch/experimental-rmdsl
I use this for really large mappings and it works flawlessly.
Other Java based options:
https://tarql.github.io/
https://github.com/carml/carml
carml is RML, but a much faster implementation than the reference
implementation of RML.
another Java R2RML implementation which I did not try myself:
https://github.com/nkons/r2rml-parser
But out of curiosity, you want GTFS in RDF or you want to know what it
means to get there? Because I personally really try to avoid writing
stuff other people already solved unless I have a good reason to do so :)
regards
Adrian
Sv: Migrate CVS data into RDF
Posted by Glenn Eriksson <an...@hotmail.com>.
Thanks Andrian for the hint on the Node.JS libraries.
I am sticking to the Java and Kotling environment for various reasons.
I've been reading about the R2RML Ontop API for Java. But to my limited knowledge it seams geared towards queering and transforming relational DB (JDBC) to RFD and not trough merging tabular data with RDF model.
Regards Glenn
________________________________
Från: Adrian Gschwend <ml...@netlabs.org>
Skickat: den 8 april 2019 22:51
Till: users@jena.apache.org
Ämne: Re: Migrate CVS data into RDF
On 08.04.19 15:13, Glenn Eriksson wrote:
Hi Glenn,
> Use case: I want to migrate CVS data in GTFS (General Transit Feed
> Specification) format into RDF
>
> * I have a ontology of GTFS in Turtle format (gtfs.ttl) * There
> are 12-13 CVS files with data that I want to merge (one at a time)
> with the GTFS model * Store the migrated data in a triple store for
> further processing
you might want to checkout
https://github.com/linkedconnections/gtfs2lc
and
https://github.com/OpenTransport/gtfs-csv2rdf
And linkedconnections in general. Pieter does great work with this data.
> 1. Is it possible to use the retired jena-csv module to merge CVS
> data and GTFS. I don't want to use R2RML if I can avoid it since it's
> quite verbose?
In case you still want to do it on your own for whatever reason I would
recommend our CSV on the Web based parser, which is streaming-capable.
GTFS generates a *lot* of data so streaming is a must.
https://github.com/rdf-ext/rdf-parser-csvw
This is for example used here: https://github.com/zazuko/blv-tierseuchen-ld
Node.js based, not JVM.
regards
Adrian
Re: Migrate CVS data into RDF
Posted by Adrian Gschwend <ml...@netlabs.org>.
On 08.04.19 15:13, Glenn Eriksson wrote:
Hi Glenn,
> Use case: I want to migrate CVS data in GTFS (General Transit Feed
> Specification) format into RDF
>
> * I have a ontology of GTFS in Turtle format (gtfs.ttl) * There
> are 12-13 CVS files with data that I want to merge (one at a time)
> with the GTFS model * Store the migrated data in a triple store for
> further processing
you might want to checkout
https://github.com/linkedconnections/gtfs2lc
and
https://github.com/OpenTransport/gtfs-csv2rdf
And linkedconnections in general. Pieter does great work with this data.
> 1. Is it possible to use the retired jena-csv module to merge CVS
> data and GTFS. I don't want to use R2RML if I can avoid it since it's
> quite verbose?
In case you still want to do it on your own for whatever reason I would
recommend our CSV on the Web based parser, which is streaming-capable.
GTFS generates a *lot* of data so streaming is a must.
https://github.com/rdf-ext/rdf-parser-csvw
This is for example used here: https://github.com/zazuko/blv-tierseuchen-ld
Node.js based, not JVM.
regards
Adrian
Re: Migrate CVS data into RDF
Posted by Nicky van Oorschot <ni...@netage.nl>.
Hi Glenn,
You might want to try CARML which is a library for converting: csv, json
and xml to RDF based on a R2RML mapping (https://github.com/carml/carml).
CARML is Java based and you are able to use it as it suits you. Limitation
is the amount of data handled, since CARML does everything in memory,
altough with CSV you could split the file and create chunks.
Met Vriendelijke Groet / With Kind Regards
Nicky van Oorschot
Twitter: @nickyvoorschot
Tel. +31(0)6-20335266
Netage B.V.
http://netage.nl
Esdoornstraat 3
3461ER Linschoten
From: Glenn Eriksson <an...@hotmail.com>
To: "users@jena.apache.org" <us...@jena.apache.org>
Date: 08-04-2019 15:13
Subject: Migrate CVS data into RDF
Hi,
I am a Jena newbie and done a couple of example and tutorials. Now that
actually have a use-case for Jean I've got stuck converting CVS into RDF.
CVS is a common format within transport sector and public transportation.
I've noticed that the jena-cvs library has been retired (jena 3.9). I
don't want to use third-party tools (cmd) or library and I want control of
the process.
Use case: I want to migrate CVS data in GTFS (General Transit Feed
Specification) format into RDF
* I have a ontology of GTFS in Turtle format (gtfs.ttl)
* There are 12-13 CVS files with data that I want to merge (one at a
time) with the GTFS model
* Store the migrated data in a triple store for further processing
My questions/problem are:
1. Is it possible to use the retired jena-csv module to merge CVS data
and GTFS. I don't want to use R2RML if I can avoid it since it's quite
verbose?
2. When I use Jean 3.9 in my pom.xml and build with dependencies the
JAR file for jena-csv is missing. So I haven't been able test GraphCSV and
merge it with the loaded GTFS model. Where can I find the jar file for
jena-csv 3.9?
Sorry if this is a rudimentary procedure and question, point me in the
right direction if my approach is wrong.
Greetings Glenn
PS. I have read these jena-csv<
https://lists.apache.org/thread.html/81dfedb090089a15275d2b83bd676b03eb141f0d92c96d6d6d455c29@%3Cusers.jena.apache.org%3E
>, CSV to rdf<
https://lists.apache.org/thread.html/ba053bf52d30285bb447337ac4518110bcec3a6bf035b4f708e67037@%3Cusers.jena.apache.org%3E
> in the mail archive. I have found the source code for jena-csv but not
the the jar.
Re: Migrate CVS data into RDF
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
And there is also CSV2RDF which uses SPARQL query as a transformation:
https://github.com/AtomGraph/CSV2RDF
On Tue, Apr 9, 2019 at 12:11 PM Andy Seaborne <an...@apache.org> wrote:
>
> Yet another choice is tarql:
>
> https://github.com/tarql/tarql
>
> On 08/04/2019 14:13, Glenn Eriksson wrote:
> > Hi,
> >
> > I am a Jena newbie and done a couple of example and tutorials. Now that actually have a use-case for Jean I've got stuck converting CVS into RDF. CVS is a common format within transport sector and public transportation. I've noticed that the jena-cvs library has been retired (jena 3.9). I don't want to use third-party tools (cmd) or library and I want control of the process.
> >
> > Use case: I want to migrate CVS data in GTFS (General Transit Feed Specification) format into RDF
> >
> > * I have a ontology of GTFS in Turtle format (gtfs.ttl)
> > * There are 12-13 CVS files with data that I want to merge (one at a time) with the GTFS model
> > * Store the migrated data in a triple store for further processing
> >
> > My questions/problem are:
> >
> > 1. Is it possible to use the retired jena-csv module to merge CVS data and GTFS. I don't want to use R2RML if I can avoid it since it's quite verbose?
> > 2. When I use Jean 3.9 in my pom.xml and build with dependencies the JAR file for jena-csv is missing. So I haven't been able test GraphCSV and merge it with the loaded GTFS model. Where can I find the jar file for jena-csv 3.9?
> >
> > Sorry if this is a rudimentary procedure and question, point me in the right direction if my approach is wrong.
> >
> >
> > Greetings Glenn
> >
> >
> > PS. I have read these jena-csv<https://lists.apache.org/thread.html/81dfedb090089a15275d2b83bd676b03eb141f0d92c96d6d6d455c29@%3Cusers.jena.apache.org%3E>, CSV to rdf<https://lists.apache.org/thread.html/ba053bf52d30285bb447337ac4518110bcec3a6bf035b4f708e67037@%3Cusers.jena.apache.org%3E> in the mail archive. I have found the source code for jena-csv but not the the jar.
> >
> >
Re: Migrate CVS data into RDF
Posted by Andy Seaborne <an...@apache.org>.
Yet another choice is tarql:
https://github.com/tarql/tarql
On 08/04/2019 14:13, Glenn Eriksson wrote:
> Hi,
>
> I am a Jena newbie and done a couple of example and tutorials. Now that actually have a use-case for Jean I've got stuck converting CVS into RDF. CVS is a common format within transport sector and public transportation. I've noticed that the jena-cvs library has been retired (jena 3.9). I don't want to use third-party tools (cmd) or library and I want control of the process.
>
> Use case: I want to migrate CVS data in GTFS (General Transit Feed Specification) format into RDF
>
> * I have a ontology of GTFS in Turtle format (gtfs.ttl)
> * There are 12-13 CVS files with data that I want to merge (one at a time) with the GTFS model
> * Store the migrated data in a triple store for further processing
>
> My questions/problem are:
>
> 1. Is it possible to use the retired jena-csv module to merge CVS data and GTFS. I don't want to use R2RML if I can avoid it since it's quite verbose?
> 2. When I use Jean 3.9 in my pom.xml and build with dependencies the JAR file for jena-csv is missing. So I haven't been able test GraphCSV and merge it with the loaded GTFS model. Where can I find the jar file for jena-csv 3.9?
>
> Sorry if this is a rudimentary procedure and question, point me in the right direction if my approach is wrong.
>
>
> Greetings Glenn
>
>
> PS. I have read these jena-csv<https://lists.apache.org/thread.html/81dfedb090089a15275d2b83bd676b03eb141f0d92c96d6d6d455c29@%3Cusers.jena.apache.org%3E>, CSV to rdf<https://lists.apache.org/thread.html/ba053bf52d30285bb447337ac4518110bcec3a6bf035b4f708e67037@%3Cusers.jena.apache.org%3E> in the mail archive. I have found the source code for jena-csv but not the the jar.
>
>