Spatial query with 'dynamic' dataset

[Lalonde Marc <ma...@crim.ca>]

Hello all,
I'm trying to execute spatial queries with jena in the following 
context: an object's position (lat/long) is updated on a regular basis 
and I run a query ("SELECT ... spatial:withinCircle()...") to verify 
whether the object is close to a particular point of interest. I 
simulate the object's motion by increasing its long coordinate; the 
associated model is removed (i.e. spatialDataset.removeNamedModel() ) 
and recreated with the new position; content of the Lucene indexing 
folder keeps increasing. What happens is that when the object gets 
within the desired radius, the query returns a result (which is good) 
but I keep getting a result even though the object has (eventually) 
moved beyond the point of interest + radius. Could it be because model 
removal causes the spatial indexing to be out of sync? I use 
"RDFDataMgr.read(m, file)" to load an updated model to the dataset.

Thanks,
Marc

Re: Spatial query with 'dynamic' dataset

[Osma Suominen <os...@helsinki.fi>]

Lalonde Marc kirjoitti 25.01.2018 klo 23:25:
> Thanks for your reply. I'm not using fuseki so far, all I did was take 
> JenaSpatialExample1.java and modify a few things. But it definitely has 
> to do with the indexing mechanism: when I systematically destroy the 
> index folder before reloading the model(via  RDFDataMgr.read(...)), the 
> behavior is good. I took a peek at the code under jena-spatial, as far 
> as I can tell there's nothing meant for removing old indexed stuff. Is 
> there a proper way of handling live or dynamic spatial data, apart from 
> reindexing every time a model update needs to be done?

You are right. The deletion of stale index entries from a jena-text 
Lucene index [1] was a feature that was implemented a while ago (Jena 
3.0.0). The same hasn't been implemented for jena-spatial although the 
same approach would probably work there too. So when you update your 
triples, new entries will be added to the jena-spatial index (at least I 
hope so) but old ones will not be deleted.

The only way currently to handle updates is to reindex every time the 
data changes. Alternatively, if it's possible to adapt your SPARQL query 
in such a way that it double-checks the results coming from the spatial 
index (for example checking that the coordinates of objects are actually 
within the area you queried), that might work too in some cases but it's 
obviously not a general solution.

Pull requests welcome ;)

-Osma

[1] 
https://jena.apache.org/documentation/query/text-query.html#uid-field-and-automatic-document-deletion

-- 
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi

Re: Spatial query with 'dynamic' dataset

[Lalonde Marc <ma...@crim.ca>]

Thanks for your reply. I'm not using fuseki so far, all I did was take 
JenaSpatialExample1.java and modify a few things. But it definitely has 
to do with the indexing mechanism: when I systematically destroy the 
index folder before reloading the model(via  RDFDataMgr.read(...)), the 
behavior is good. I took a peek at the code under jena-spatial, as far 
as I can tell there's nothing meant for removing old indexed stuff. Is 
there a proper way of handling live or dynamic spatial data, apart from 
reindexing every time a model update needs to be done?
Thanks,
Marc

Le 2018-01-25 à 02:31, Lorenz Buehmann a écrit :
> The spatial index is based on Lucene and at least for the textindex
> updates to the dataset will be reflected automatically. In your case, as
> you're removing a whole named graph I don't know whether this makes a
> difference, but it might be the case. It would probably be helpful for
> the developers to see your Fuseki config first.
>
> Lorenz
>
>
> On 24.01.2018 18:22, Lalonde Marc wrote:
>> Hello all,
>> I'm trying to execute spatial queries with jena in the following
>> context: an object's position (lat/long) is updated on a regular basis
>> and I run a query ("SELECT ... spatial:withinCircle()...") to verify
>> whether the object is close to a particular point of interest. I
>> simulate the object's motion by increasing its long coordinate; the
>> associated model is removed (i.e. spatialDataset.removeNamedModel() )
>> and recreated with the new position; content of the Lucene indexing
>> folder keeps increasing. What happens is that when the object gets
>> within the desired radius, the query returns a result (which is good)
>> but I keep getting a result even though the object has (eventually)
>> moved beyond the point of interest + radius. Could it be because model
>> removal causes the spatial indexing to be out of sync? I use
>> "RDFDataMgr.read(m, file)" to load an updated model to the dataset.
>>
>> Thanks,
>> Marc
>>
>>
>

-- 
sign _____________________
Marc Lalonde, M.Sc.A.
Conseiller senior / Senior advisor
Équipe Vision et imagerie / Vision and Imaging Team
CRIM (Centre de recherche informatique de Montréal/
Computer Research Institute of Montreal)
1-514 840-1235, ext. 6764
Marc.Lalonde@crim.ca
http://www.crim.ca/fr/equipes/vision-et-imagerie

Re: Spatial query with 'dynamic' dataset

[Lorenz Buehmann <bu...@informatik.uni-leipzig.de>]

The spatial index is based on Lucene and at least for the textindex
updates to the dataset will be reflected automatically. In your case, as
you're removing a whole named graph I don't know whether this makes a
difference, but it might be the case. It would probably be helpful for
the developers to see your Fuseki config first.

Lorenz


On 24.01.2018 18:22, Lalonde Marc wrote:
> Hello all,
> I'm trying to execute spatial queries with jena in the following
> context: an object's position (lat/long) is updated on a regular basis
> and I run a query ("SELECT ... spatial:withinCircle()...") to verify
> whether the object is close to a particular point of interest. I
> simulate the object's motion by increasing its long coordinate; the
> associated model is removed (i.e. spatialDataset.removeNamedModel() )
> and recreated with the new position; content of the Lucene indexing
> folder keeps increasing. What happens is that when the object gets
> within the desired radius, the query returns a result (which is good)
> but I keep getting a result even though the object has (eventually)
> moved beyond the point of interest + radius. Could it be because model
> removal causes the spatial indexing to be out of sync? I use
> "RDFDataMgr.read(m, file)" to load an updated model to the dataset.
>
> Thanks,
> Marc
>
>