You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Leila Deljkovic <le...@koordinates.com> on 2018/01/10 03:15:46 UTC

Spatial search

Hi,

I’m quite new to Solr and am interested in using spatial search for geospatial data (Solr 7.1).

One problem is addressing feature density over a layer and using this to determine if a layer would be a relevant result over a search extent. I’d like to know is it feasible/possible to “split” a data layer into nested documents and index them, then at query time, count the number of nested documents that coincide with the search extent. Or maybe make use of overlapRatio or similar.

Thanks

Re: Spatial search (and nested docs)

Posted by Emir Arnautović <em...@sematext.com>.
Hi Leila,
You should be able to pass score parameter that will tell how to combine children’s scores to parent’s score: https://lucene.apache.org/solr/guide/6_6/other-parsers.html#OtherParsers-Scoring

Maybe you could change approach and index each cell as a separate doc with layer field and use spatial to filter our covered cells and json faceting to sum density over layer field. Something like:

  top_layers:{ 
    type: terms,
    field: layer,
    limit: 10,
    sort: “total_density desc”,
    facet:{
      total_density: “sum(density)”
    }
  }

I didn’t use 

HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 11 Jan 2018, at 00:02, Leila Deljkovic <le...@koordinates.com> wrote:
> 
> Hi Emir,
> 
> Thanks for the reply. My problem has been simplified a bit now. 
> 
> https://lucene.apache.org/solr/guide/7_0/uploading-data-with-index-handlers.html#UploadingDatawithIndexHandlers-NestedChildDocuments <https://lucene.apache.org/solr/guide/7_0/uploading-data-with-index-handlers.html#UploadingDatawithIndexHandlers-NestedChildDocuments>
> 
> I have never used nested documents, but a bit of background is that a spatial data layer consisting of features (points, lines, polygons, or an aerial image) is split up into sections (grid cells) based on the density of these features over the layer; smaller grid cells indicate high density of features in that area. 
> 
> I need to rank results based on density of features and whether dense areas of the layer overlap with the region of space on a map I am searching in. This is important because a layer could cover an entire country, for example if I query for “roads”, the layer would be dense in urban areas as there are more roads there, and less dense in rural areas, and if I am searching for a particular city, this layer would be of interest to me even though it covers the entire country. The idea is for the original layer to be the parent document (which is what should be returned when a query is made), and the child documents are the individual grid cells (which will hold the geometry of the cell and a density field for the features inside the cell). 
> 
> I would like to know if it is possible to rank the parent document based on a function which aggregates fields from the child documents (in this case, the density field). There is not much info on this that I could find online.
> 
> Thanks
> 
>> On 10/01/2018, at 11:58 PM, Emir Arnautović <em...@sematext.com> wrote:
>> 
>> Hi Leila,
>> Maybe I need to refresh my spatial terminology, but I am having troubles following your case. Can you explain a bit more, what is dataset that is indexed and what are query inputs and what should be the result. The one thing that puzzles me the most is “nested documents”.
>> 
>> Thanks,
>> Emir
>> --
>> Monitoring - Log Management - Alerting - Anomaly Detection
>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>> 
>> 
>> 
>>> On 10 Jan 2018, at 04:15, Leila Deljkovic <le...@koordinates.com> wrote:
>>> 
>>> Hi,
>>> 
>>> I’m quite new to Solr and am interested in using spatial search for geospatial data (Solr 7.1).
>>> 
>>> One problem is addressing feature density over a layer and using this to determine if a layer would be a relevant result over a search extent. I’d like to know is it feasible/possible to “split” a data layer into nested documents and index them, then at query time, count the number of nested documents that coincide with the search extent. Or maybe make use of overlapRatio or similar.
>>> 
>>> Thanks
>>> 
>>> 
>> 
>

Re: Spatial search (and nested docs)

Posted by Leila Deljkovic <le...@koordinates.com>.
Hi Emir,

Thanks for the reply. My problem has been simplified a bit now. 

https://lucene.apache.org/solr/guide/7_0/uploading-data-with-index-handlers.html#UploadingDatawithIndexHandlers-NestedChildDocuments <https://lucene.apache.org/solr/guide/7_0/uploading-data-with-index-handlers.html#UploadingDatawithIndexHandlers-NestedChildDocuments>

I have never used nested documents, but a bit of background is that a spatial data layer consisting of features (points, lines, polygons, or an aerial image) is split up into sections (grid cells) based on the density of these features over the layer; smaller grid cells indicate high density of features in that area. 

I need to rank results based on density of features and whether dense areas of the layer overlap with the region of space on a map I am searching in. This is important because a layer could cover an entire country, for example if I query for “roads”, the layer would be dense in urban areas as there are more roads there, and less dense in rural areas, and if I am searching for a particular city, this layer would be of interest to me even though it covers the entire country. The idea is for the original layer to be the parent document (which is what should be returned when a query is made), and the child documents are the individual grid cells (which will hold the geometry of the cell and a density field for the features inside the cell). 

I would like to know if it is possible to rank the parent document based on a function which aggregates fields from the child documents (in this case, the density field). There is not much info on this that I could find online.

Thanks

> On 10/01/2018, at 11:58 PM, Emir Arnautović <em...@sematext.com> wrote:
> 
> Hi Leila,
> Maybe I need to refresh my spatial terminology, but I am having troubles following your case. Can you explain a bit more, what is dataset that is indexed and what are query inputs and what should be the result. The one thing that puzzles me the most is “nested documents”.
> 
> Thanks,
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> 
> 
> 
>> On 10 Jan 2018, at 04:15, Leila Deljkovic <le...@koordinates.com> wrote:
>> 
>> Hi,
>> 
>> I’m quite new to Solr and am interested in using spatial search for geospatial data (Solr 7.1).
>> 
>> One problem is addressing feature density over a layer and using this to determine if a layer would be a relevant result over a search extent. I’d like to know is it feasible/possible to “split” a data layer into nested documents and index them, then at query time, count the number of nested documents that coincide with the search extent. Or maybe make use of overlapRatio or similar.
>> 
>> Thanks
>> 
>> 
>

Re: Spatial search

Posted by Emir Arnautović <em...@sematext.com>.
Hi Leila,
Maybe I need to refresh my spatial terminology, but I am having troubles following your case. Can you explain a bit more, what is dataset that is indexed and what are query inputs and what should be the result. The one thing that puzzles me the most is “nested documents”.

Thanks,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 10 Jan 2018, at 04:15, Leila Deljkovic <le...@koordinates.com> wrote:
> 
> Hi,
> 
> I’m quite new to Solr and am interested in using spatial search for geospatial data (Solr 7.1).
> 
> One problem is addressing feature density over a layer and using this to determine if a layer would be a relevant result over a search extent. I’d like to know is it feasible/possible to “split” a data layer into nested documents and index them, then at query time, count the number of nested documents that coincide with the search extent. Or maybe make use of overlapRatio or similar.
> 
> Thanks
> 
>