You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Adrian Gschwend <ml...@netlabs.org> on 2015/06/15 14:22:22 UTC
SPARQL FILTER & non standard data types
Hi,
I'm using Pieter Colpaerts script for transforming GTFS public transport
schedule data to RDF, see
https://github.com/OpenTransport/gtfs-csv2rdf/
According to the GTFS specification arrivalTime and departureTime can
have more than 24 hours to represent connections which are technically
on a new day but before the end of the "daily" public transport
schedule. As an example a bus departing at 01:33 would be listed as
25:33:00.
While this makes sense for what GTFS aims to do it's highly inconvenient
for querying the data with SPARQL. When using SPARQL one wants to be
able to use the FILTER keyword to restrict to time ranges for the
particular query. This is in that form not possible as we cannot assign
standard values like xsd:time to it.
As an example I would like to get all departures after 22:00 on that
particular stop/connection.
I wonder what would be needed to implement a new data type for GTFS
datasets with that particular behaviour. In terms of what would be
needed from a Fuseki/SPARQL point of view and is this a good idea or
not? It clearly will break any other SPARQL endpoint which would not
know how to handle this data type but it would not require hacks on
GTFS2RDF side.
I will collect ideas in this issue:
https://github.com/OpenTransport/gtfs-csv2rdf/issues/13
regards
Adrian
Re: SPARQL FILTER & non standard data types
Posted by Andy Seaborne <an...@apache.org>.
On 21/06/15 17:53, Adrian Gschwend wrote:
> On 15.06.15 17:38, Andy Seaborne wrote:
>
> Hi Andy,
>
>> Some thoughts:
>>
>> If you can translate to xsd date/time values, < and > will work.
>
> thanks for the various hints, James Anderson proposed another type to
> use (outside of the list), I listed the response here:
>
> https://github.com/OpenTransport/gtfs-csv2rdf/issues/13
>
> "if one takes your description as it stands, you seek to represent a
> duration either from midnight or from beginning of service, or some
> other point in time which fits your model.
> if you use a duration to model exactly that, xsd:dayTimeDuration should
> have sufficient capacity and the time of day for a given event would be
> the combination of that duration and the start of the respective service
> day."
>
> That pretty much matches the semantics of the GTFS specs so that would
> be the best solution. I tried to figure out what xsd types are supported
> in Jena/Fuseki, is there a reference somewhere?
>
> regards
>
> Adrian
>
(I have not forgotten your question about Fuseki2 configuration)
ARQ supports all the XSD atomic datatypes, including all the
date/time/duration, but not all the XML related ones (so not sequences,
xsd:NOTATION, XML-specific subtypes of xsd:string).
Durations are helpful if you include the start point of the duration.
Just giving a duration doesn't fix it on the timeline, so it might mean
any day or time. Durations are not the most friendly of lexical forms
however.
But it is adding an extra property so a translated
gtfs:arrivalTimeXSD
which is a proper xsd:datetime for calculation (then considering the
gtfs:arrivalTime as some presentation form akin to an rdfs:label) keeps
you inside unrestricted SPARQL. Well, for the limited xsd:dateTime
support required for comparison, not:
xsd:dateTime - xsd:dateTime => xsd:duration
and it strikes me that how long the journey takes is useful information.
That could be added as well (so not need to calculate in SPARQL) but
waiting time between arriving on one bus and leaving on another is a
calculation or N^2 triples.
Andy
Re: SPARQL FILTER & non standard data types
Posted by Adrian Gschwend <ml...@netlabs.org>.
On 15.06.15 17:38, Andy Seaborne wrote:
Hi Andy,
> Some thoughts:
>
> If you can translate to xsd date/time values, < and > will work.
thanks for the various hints, James Anderson proposed another type to
use (outside of the list), I listed the response here:
https://github.com/OpenTransport/gtfs-csv2rdf/issues/13
"if one takes your description as it stands, you seek to represent a
duration either from midnight or from beginning of service, or some
other point in time which fits your model.
if you use a duration to model exactly that, xsd:dayTimeDuration should
have sufficient capacity and the time of day for a given event would be
the combination of that duration and the start of the respective service
day."
That pretty much matches the semantics of the GTFS specs so that would
be the best solution. I tried to figure out what xsd types are supported
in Jena/Fuseki, is there a reference somewhere?
regards
Adrian
Re: SPARQL FILTER & non standard data types
Posted by Andy Seaborne <an...@apache.org>.
Adrian,
Some thoughts:
If you can translate to xsd date/time values, < and > will work.
One approach is to add some custom functions that "casts" the different
way of writing times into something that can be tested.
gtfsFn:timestamp("2015-06-15T25:33:00"^^:customDatatype)
which returns "2015-06-16T01:33:00"^^xsd:dateTime
which is an RDF term that does mao into ARQ's date/time code.
Another approach is like rules: to add properties: this is a post
data-generation expansion (perl or etc will do it).
:bus :departs "25:33:00"^^:customDatatype .
and add
:bus :departsXSD "01:33:00"^^xsd:time .
So "before 09:00am" where 02:00:00 is the end of logical day.
FILTER (?t > "02:00:00"^^xsd:time && ?t < "09:00:00"^^xsd:time)
If "logical day" has overlaps, then it's all going to get a bit weird!
And an off-the-wall solution: fake the timezone. Move the bus times
into +02:00.
Andy
On 15/06/15 13:22, Adrian Gschwend wrote:
> Hi,
>
> I'm using Pieter Colpaerts script for transforming GTFS public transport
> schedule data to RDF, see
>
> https://github.com/OpenTransport/gtfs-csv2rdf/
>
> According to the GTFS specification arrivalTime and departureTime can
> have more than 24 hours to represent connections which are technically
> on a new day but before the end of the "daily" public transport
> schedule. As an example a bus departing at 01:33 would be listed as
> 25:33:00.
>
> While this makes sense for what GTFS aims to do it's highly inconvenient
> for querying the data with SPARQL. When using SPARQL one wants to be
> able to use the FILTER keyword to restrict to time ranges for the
> particular query. This is in that form not possible as we cannot assign
> standard values like xsd:time to it.
>
> As an example I would like to get all departures after 22:00 on that
> particular stop/connection.
>
> I wonder what would be needed to implement a new data type for GTFS
> datasets with that particular behaviour. In terms of what would be
> needed from a Fuseki/SPARQL point of view and is this a good idea or
> not? It clearly will break any other SPARQL endpoint which would not
> know how to handle this data type but it would not require hacks on
> GTFS2RDF side.
>
> I will collect ideas in this issue:
>
> https://github.com/OpenTransport/gtfs-csv2rdf/issues/13
>
> regards
>
> Adrian
>