You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Dhanesh Radhakrishnan <dh...@hifx.co.in> on 2013/11/15 07:36:10 UTC
Solr spatial search within the polygon
Hi,
I'm experimenting with solr spatial search, with plotting points in the map
(Latitude and longitude) and based on the value I need to get the result.
As the first step I've defined the filed type as
<fieldType name="location_rpt"
class="solr.SpatialRecursivePrefixTreeFieldType"
spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory"
distErrPct="0.025" maxDistErr="0.000009" units="degrees" />
And then added the field *location* as type *location_rpt*
<field name="location" type="location_rpt" indexed="true" stored="true"
multiValued="true" />
Indexed the location filed as $latitude, $longitude So that data in the
*location* will be like
"location":["9.445890,76.540970"]
Next I draw a polygon in google map and collected the lat and lng
coordinates of the polygon.
It will be like
9.472992 76.540817, 9.441328 76.523651 , 9.433708 76.555065 , 9.458092
76.572403, 9.472992 76.540817
Based on this coordinates I performed a query in solr like this
localhost:8983/solr/ha_poc/select?fl=id,name,district,locality&wt=json&
json.nl=map&q=*:*&fq=location:"IsWithin(POLYGON((9.472992 76.540817,
9.441328 76.523651 , 9.433708 76.555065 , 9.458092 76.572403, 9.472992
76.540817))) distErrPct=0"
But I didn't get the result from the solr as I expected.
{
- "responseHeader": {
- "status": 0,
- "QTime": 2
},
- "response": {
- "numFound": 0,
- "start": 0,
- "docs": [ ]
}
}
Is there anything that I missed ???
Can anybody help me in solving this issue with solr spatial search.
I'm using Solr 4.4.0
Added an additional dependency JTS jar file for the support of polygon
/lib/ext/jts-1.13.jar
--
*dhanesh s.r*
Re: Solr spatial search within the polygon
Posted by "Smiley, David W." <ds...@mitre.org>.
Dhanesh,
> I'm pretty sure that the coordinates are in the right position.
> "9.445890,76.540970" is in India, precisely in Kerala state :)
My suspicion was wright; you have all of your latitudes and longitudes in
the wrong position. Your example that I quote you on above is correct
("lat,lon") , but you're not indexing it that way (you're doing "lat
lon"). If you corrected your indexing to use "lat,lon" (as you should
do), you will find that the WKT you generate in your queries is also
reversed. The way you're doing it now prevents you from searching
anywhere out of -90 & +90 degrees longitude. At this point I don't think
I can be any more clear.
Good luck.
~ David
On 11/20/13 12:50 AM, "Dhanesh Radhakrishnan" <dh...@hifx.co.in> wrote:
>Hi David,
>Thank you for your reply
>This is my current schema and field type "location_rpt" is a
>SpatialRecursivePrefixTreeFieldType and
>Field "location" is a type "location_rpt" and its multiValued
>
><types>
> <fieldtype name="string" class="solr.StrField" sortMissingLast="true"
>omitNorms="true"/>
> <fieldType name="long" class="solr.TrieLongField" precisionStep="0"
>positionIncrementGap="0"/>
> <fieldType name="tdouble" class="solr.TrieDoubleField"
>precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
> <fieldType name="location_rpt"
>class="solr.SpatialRecursivePrefixTreeFieldType"
>spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFac
>tory"
>distErrPct="0.025" maxDistErr="0.000009" units="degrees" geo="true" />
></types>
>
>
>
><fields>
> <field name="_version_" type="long" indexed="true" stored="true"
>multiValued="false"/>
> <field name="id" type="string" stored="true" indexed="true"
>multiValued="false" omitNorms="true" omitTermFreqAndPositions="true"
>termVectors="false"/>
> <field name="name" type="string" stored="true" indexed="true" />
> <field name="state" type="string" stored="true" indexed="true"
>required="false" />
> <field name="locality" type="string" stored="true" indexed="true"
>required="false" />
> <field name="address" type="string" stored="true" indexed="true"
>required="false" />
> <field name="location" type="location_rpt" indexed="true"
>stored="true" multiValued="true" />
></fields>
>
>Whenever add a document to solr, I'll collect the current latitude and
>longitude of particular business and index in the field "location"
>It's like
>$doc->setField('location', $business['latitude']."
>".$business['longitude']);
>This should looks like "location":["9.445890 76.540970"] in solr
>
>What I'm doing is that in Map view of search result , there is one
>provision to draw polygon in map and fetch the result based on the
>drawing.
>
>http://localhost:8983/solr/poc/select?fl=id,name,locality&wt=json&json.nl=
>map&q=*:*&fq=state:Kerala&fq=location:"IsWithin(POLYGON((9.471920923238988
>76.5496015548706,9.464174399734185 76.53947353363037,9.457232011740006
>76.55457973480225,9.471920923238988 76.5496015548706)))
>distErrPct=0"&debugQuery=true
>
>I'm pretty sure that the coordinates are in the right position.
>"9.445890,76.540970" is in India, precisely in Kerala state :)
>
>It is highly appreciated that you kindly correct me if I'm in wrong way
>
>
>
>
>
>This is response from solr
>
>"responseHeader": {
> "status": 0,
> "QTime": 5
>},
>"response": {
> "numFound": 3,
> "start": 0,
> "docs": [
> {
> "id": "192",
> "name": "50 cents of ideal plot",
> "locality": "Changanassery"
> },
> {
> "id": "189",
> "name": "new independent house for sale",
> "locality": "Changanassery"
> },
> {
> "id": "188",
> "name": "Renovated Resort style home with 21 cent",
> "locality": "Changanassery"
> }
> ]
>}
>
>
>
>Here is the debug mode output of the query
>
>
>"debug": {
> "rawquerystring": "*:*",
> "querystring": "*:*",
> "parsedquery": "MatchAllDocsQuery(*:*)",
> "parsedquery_toString": "*:*",
> "explain": {
> "188": "\n1.0 = (MATCH) MatchAllDocsQuery, product of:\n 1.0 =
>queryNorm\n",
> "189": "\n1.0 = (MATCH) MatchAllDocsQuery, product of:\n 1.0 =
>queryNorm\n",
> "192": "\n1.0 = (MATCH) MatchAllDocsQuery, product of:\n 1.0 =
>queryNorm\n"
> },
> "QParser": "LuceneQParser",
> "filter_queries": [
> "state:Kerala",
> "location:\"IsWithin(POLYGON((9.471920923238988
>76.5496015548706,9.464174399734185 76.53947353363037,9.457232011740006
>76.55457973480225,9.471920923238988 76.5496015548706))) distErrPct=0\""
> ],
> "parsed_filter_queries": [
> "state:Kerala",
>
>"ConstantScore(org.apache.lucene.spatial.prefix.WithinPrefixTreeFilter@1ed
>6c279
>)"
> ],
> "timing": {
> "time": 5,
> "prepare": {
> "time": 1,
> "query": {
> "time": 1
> },
> "facet": {
> "time": 0
> },
> "mlt": {
> "time": 0
> },
> "highlight": {
> "time": 0
> },
> "stats": {
> "time": 0
> },
> "debug": {
> "time": 0
> }
> },
> "process": {
> "time": 4,
> "query": {
> "time": 3
> },
> "facet": {
> "time": 0
> },
> "mlt": {
> "time": 0
> },
> "highlight": {
> "time": 0
> },
> "stats": {
> "time": 0
> },
> "debug": {
> "time": 1
> }
> }
> }
>}
>
>
>On Tue, Nov 19, 2013 at 8:56 PM, Smiley, David W. <ds...@mitre.org>
>wrote:
>
>>
>>
>> On 11/19/13 4:06 AM, "Dhanesh Radhakrishnan" <dh...@hifx.co.in> wrote:
>>
>> >Hi David,
>> >Thank you so much for the detailed reply. I've checked each and every
>>lat
>> >lng coordinates and its a purely polygon.
>> >After some time I did one change in the lat lng indexing.
>> >Changed the indexing format.
>> >
>> >Initially I indexed the latitude and longitude separated by comma Eg:-
>> >"location":["9.445890,76.540970"]
>> >Instead I indexed with space.
>> > "location":["9.445890 76.540970"]
>>
>> Just to be clear, if you use a space, it's "x y" order. If you use a
>> comma, it's "y, x" order. If you use WKT, it's always a space in X Y
>> order (and of course, the shape name and other stuff). You may have
>> gotten your search to work but I have a hunch you have all your
>>latitudes
>> and longitudes in the wrong position, but I can't possible know for sure
>> because your example datapoint is ambiguous. 76 degrees latitude is
>> pretty far up-there though, hence my hunch you've got it wrong.
>>
>> >
>> >and it worked
>> >
>> >Also from your observation on IsWithIn predicate I tested with
>>Intersects
>> >and I found there is a difference in the QTime.
>> >
>> >For IsWithin
>> > "QTime": 9
>> >ResponseHeader": {
>> >"status": 0,
>> >"QTime": 9
>> >},
>> >
>> >When I used "Intersects"
>> >responseHeader": {
>> >"status": 0,
>> >"QTime": 26
>> >}
>>
>> There's no way the Intersects code is slower than IsWithin; IsWithin
>>needs
>> to visit many more grid tiles -- big ones covering lots of docs.
>>Perhaps
>> you have these times flipped as well ;-) Any way, given you have
>> multi-valued point data, you should choose the spatial predicate that
>> matches what you intend (your requirements). Maybe that's IsWithin,
>>maybe
>> that's Intersects. *if* your field was *not* multi-valued (most people
>> don't have multi-valued spatial data per doc), then these two predicates
>> become semantically equivalent for such data, and so most people should
>> always choose Intersects even if "within" is colloquially how one thinks
>> of it.
>>
>> ~ David
>>
>>
>
>
>--
>*dhanesh s.r*
Re: Solr spatial search within the polygon
Posted by Dhanesh Radhakrishnan <dh...@hifx.co.in>.
Hi David,
Thank you for your reply
This is my current schema and field type "location_rpt" is a
SpatialRecursivePrefixTreeFieldType and
Field "location" is a type "location_rpt" and its multiValued
<types>
<fieldtype name="string" class="solr.StrField" sortMissingLast="true"
omitNorms="true"/>
<fieldType name="long" class="solr.TrieLongField" precisionStep="0"
positionIncrementGap="0"/>
<fieldType name="tdouble" class="solr.TrieDoubleField"
precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
<fieldType name="location_rpt"
class="solr.SpatialRecursivePrefixTreeFieldType"
spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory"
distErrPct="0.025" maxDistErr="0.000009" units="degrees" geo="true" />
</types>
<fields>
<field name="_version_" type="long" indexed="true" stored="true"
multiValued="false"/>
<field name="id" type="string" stored="true" indexed="true"
multiValued="false" omitNorms="true" omitTermFreqAndPositions="true"
termVectors="false"/>
<field name="name" type="string" stored="true" indexed="true" />
<field name="state" type="string" stored="true" indexed="true"
required="false" />
<field name="locality" type="string" stored="true" indexed="true"
required="false" />
<field name="address" type="string" stored="true" indexed="true"
required="false" />
<field name="location" type="location_rpt" indexed="true"
stored="true" multiValued="true" />
</fields>
Whenever add a document to solr, I'll collect the current latitude and
longitude of particular business and index in the field "location"
It's like
$doc->setField('location', $business['latitude']."
".$business['longitude']);
This should looks like "location":["9.445890 76.540970"] in solr
What I'm doing is that in Map view of search result , there is one
provision to draw polygon in map and fetch the result based on the drawing.
http://localhost:8983/solr/poc/select?fl=id,name,locality&wt=json&json.nl=map&q=*:*&fq=state:Kerala&fq=location:"IsWithin(POLYGON((9.471920923238988
76.5496015548706,9.464174399734185 76.53947353363037,9.457232011740006
76.55457973480225,9.471920923238988 76.5496015548706)))
distErrPct=0"&debugQuery=true
I'm pretty sure that the coordinates are in the right position.
"9.445890,76.540970" is in India, precisely in Kerala state :)
It is highly appreciated that you kindly correct me if I'm in wrong way
This is response from solr
"responseHeader": {
"status": 0,
"QTime": 5
},
"response": {
"numFound": 3,
"start": 0,
"docs": [
{
"id": "192",
"name": "50 cents of ideal plot",
"locality": "Changanassery"
},
{
"id": "189",
"name": "new independent house for sale",
"locality": "Changanassery"
},
{
"id": "188",
"name": "Renovated Resort style home with 21 cent",
"locality": "Changanassery"
}
]
}
Here is the debug mode output of the query
"debug": {
"rawquerystring": "*:*",
"querystring": "*:*",
"parsedquery": "MatchAllDocsQuery(*:*)",
"parsedquery_toString": "*:*",
"explain": {
"188": "\n1.0 = (MATCH) MatchAllDocsQuery, product of:\n 1.0 =
queryNorm\n",
"189": "\n1.0 = (MATCH) MatchAllDocsQuery, product of:\n 1.0 =
queryNorm\n",
"192": "\n1.0 = (MATCH) MatchAllDocsQuery, product of:\n 1.0 =
queryNorm\n"
},
"QParser": "LuceneQParser",
"filter_queries": [
"state:Kerala",
"location:\"IsWithin(POLYGON((9.471920923238988
76.5496015548706,9.464174399734185 76.53947353363037,9.457232011740006
76.55457973480225,9.471920923238988 76.5496015548706))) distErrPct=0\""
],
"parsed_filter_queries": [
"state:Kerala",
"ConstantScore(org.apache.lucene.spatial.prefix.WithinPrefixTreeFilter@1ed6c279
)"
],
"timing": {
"time": 5,
"prepare": {
"time": 1,
"query": {
"time": 1
},
"facet": {
"time": 0
},
"mlt": {
"time": 0
},
"highlight": {
"time": 0
},
"stats": {
"time": 0
},
"debug": {
"time": 0
}
},
"process": {
"time": 4,
"query": {
"time": 3
},
"facet": {
"time": 0
},
"mlt": {
"time": 0
},
"highlight": {
"time": 0
},
"stats": {
"time": 0
},
"debug": {
"time": 1
}
}
}
}
On Tue, Nov 19, 2013 at 8:56 PM, Smiley, David W. <ds...@mitre.org> wrote:
>
>
> On 11/19/13 4:06 AM, "Dhanesh Radhakrishnan" <dh...@hifx.co.in> wrote:
>
> >Hi David,
> >Thank you so much for the detailed reply. I've checked each and every lat
> >lng coordinates and its a purely polygon.
> >After some time I did one change in the lat lng indexing.
> >Changed the indexing format.
> >
> >Initially I indexed the latitude and longitude separated by comma Eg:-
> >"location":["9.445890,76.540970"]
> >Instead I indexed with space.
> > "location":["9.445890 76.540970"]
>
> Just to be clear, if you use a space, it's "x y" order. If you use a
> comma, it's "y, x" order. If you use WKT, it's always a space in X Y
> order (and of course, the shape name and other stuff). You may have
> gotten your search to work but I have a hunch you have all your latitudes
> and longitudes in the wrong position, but I can't possible know for sure
> because your example datapoint is ambiguous. 76 degrees latitude is
> pretty far up-there though, hence my hunch you've got it wrong.
>
> >
> >and it worked
> >
> >Also from your observation on IsWithIn predicate I tested with Intersects
> >and I found there is a difference in the QTime.
> >
> >For IsWithin
> > "QTime": 9
> >ResponseHeader": {
> >"status": 0,
> >"QTime": 9
> >},
> >
> >When I used "Intersects"
> >responseHeader": {
> >"status": 0,
> >"QTime": 26
> >}
>
> There's no way the Intersects code is slower than IsWithin; IsWithin needs
> to visit many more grid tiles -- big ones covering lots of docs. Perhaps
> you have these times flipped as well ;-) Any way, given you have
> multi-valued point data, you should choose the spatial predicate that
> matches what you intend (your requirements). Maybe that's IsWithin, maybe
> that's Intersects. *if* your field was *not* multi-valued (most people
> don't have multi-valued spatial data per doc), then these two predicates
> become semantically equivalent for such data, and so most people should
> always choose Intersects even if "within" is colloquially how one thinks
> of it.
>
> ~ David
>
>
--
*dhanesh s.r*
Re: Solr spatial search within the polygon
Posted by "Smiley, David W." <ds...@mitre.org>.
On 11/19/13 4:06 AM, "Dhanesh Radhakrishnan" <dh...@hifx.co.in> wrote:
>Hi David,
>Thank you so much for the detailed reply. I've checked each and every lat
>lng coordinates and its a purely polygon.
>After some time I did one change in the lat lng indexing.
>Changed the indexing format.
>
>Initially I indexed the latitude and longitude separated by comma Eg:-
>"location":["9.445890,76.540970"]
>Instead I indexed with space.
> "location":["9.445890 76.540970"]
Just to be clear, if you use a space, it's "x y" order. If you use a
comma, it's "y, x" order. If you use WKT, it's always a space in X Y
order (and of course, the shape name and other stuff). You may have
gotten your search to work but I have a hunch you have all your latitudes
and longitudes in the wrong position, but I can't possible know for sure
because your example datapoint is ambiguous. 76 degrees latitude is
pretty far up-there though, hence my hunch you've got it wrong.
>
>and it worked
>
>Also from your observation on IsWithIn predicate I tested with Intersects
>and I found there is a difference in the QTime.
>
>For IsWithin
> "QTime": 9
>ResponseHeader": {
>"status": 0,
>"QTime": 9
>},
>
>When I used "Intersects"
>responseHeader": {
>"status": 0,
>"QTime": 26
>}
There's no way the Intersects code is slower than IsWithin; IsWithin needs
to visit many more grid tiles -- big ones covering lots of docs. Perhaps
you have these times flipped as well ;-) Any way, given you have
multi-valued point data, you should choose the spatial predicate that
matches what you intend (your requirements). Maybe that's IsWithin, maybe
that's Intersects. *if* your field was *not* multi-valued (most people
don't have multi-valued spatial data per doc), then these two predicates
become semantically equivalent for such data, and so most people should
always choose Intersects even if "within" is colloquially how one thinks
of it.
~ David
Re: Solr spatial search within the polygon
Posted by Dhanesh Radhakrishnan <dh...@hifx.co.in>.
Hi David,
Thank you so much for the detailed reply. I've checked each and every lat
lng coordinates and its a purely polygon.
After some time I did one change in the lat lng indexing.
Changed the indexing format.
Initially I indexed the latitude and longitude separated by comma Eg:-
"location":["9.445890,76.540970"]
Instead I indexed with space.
"location":["9.445890 76.540970"]
and it worked
Also from your observation on IsWithIn predicate I tested with Intersects
and I found there is a difference in the QTime.
For IsWithin
"QTime": 9
ResponseHeader": {
"status": 0,
"QTime": 9
},
When I used "Intersects"
responseHeader": {
"status": 0,
"QTime": 26
}
Thank you so much
Regards
dhanesh s.r
Re: Solr spatial search within the polygon
Posted by javinsnc <ja...@mitula.com>.
Ok David. I give it a shot.
Thanks again!
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-spatial-search-within-the-polygon-tp4101147p4122647.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr spatial search within the polygon
Posted by "David Smiley (@MITRE.org)" <DS...@mitre.org>.
Lucene has multiple modules, one of which is "spatial". You'll see it in the
source tree checkout underneath the lucene directory.
Javadocs: http://lucene.apache.org/core/4_7_0/spatial/index.html
SpatialExample.java:
https://github.com/apache/lucene-solr/blob/trunk/lucene/spatial/src/test/org/apache/lucene/spatial/SpatialExample.java
Note: I simply went to the project on GitHub and typed the file name into
the search box and it came right up.
-----
Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-spatial-search-within-the-polygon-tp4101147p4122645.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr spatial search within the polygon
Posted by javinsnc <ja...@mitula.com>.
Could you please send me where I can find this .java?
What do you refer by "Lucene-spatial module"?
Thanks for your time David!
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-spatial-search-within-the-polygon-tp4101147p4122642.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr spatial search within the polygon
Posted by "David Smiley (@MITRE.org)" <DS...@mitre.org>.
You're going to have to use the Lucene-spatial module directly then. There's
SpatialExample.java to get you started.
javinsnc wrote
>
> David Smiley (@MITRE.org) wrote
>> On 3/10/14, 12:56 PM, "javinsnc" <
>> javiersangrador@
>> > wrote:
>> This is indeed the source of the problem.
>>
>> Why do you index with Lucene’s API and not Solr’s? Solr not only has a
>> web-service API but it also has the SolrJ API that can embed Solr —
>> EmbeddedSolrServer. I only recommend embedding Solr in limited
>> circumstances as it’s more flexible and usually plenty fast to
>> communicate
>> with Solr normally, or to easily customize Solr to load data from a
>> custom
>> file and so the indexing is all in-process. You would do the latter with
>> either a custom DataImportHandler piece or a “ContentStreamLoader
>> subclass.
> Because I started indexing by lucene and for now, it's impossible to
> change it to Solr (although I know the benefits). Maybe in the future.
>
> So do you know how I should index the field in Lucene? I need to know the
> exact type for this field. I think Lucene index fields as String by
> default, right?
>
> Thanks in advance!
-----
Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-spatial-search-within-the-polygon-tp4101147p4122641.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr spatial search within the polygon
Posted by javinsnc <ja...@mitula.com>.
David Smiley (@MITRE.org) wrote
> On 3/10/14, 12:56 PM, "javinsnc" <
> javiersangrador@
> > wrote:
> This is indeed the source of the problem.
>
> Why do you index with Lucene’s API and not Solr’s? Solr not only has a
> web-service API but it also has the SolrJ API that can embed Solr —
> EmbeddedSolrServer. I only recommend embedding Solr in limited
> circumstances as it’s more flexible and usually plenty fast to communicate
> with Solr normally, or to easily customize Solr to load data from a custom
> file and so the indexing is all in-process. You would do the latter with
> either a custom DataImportHandler piece or a “ContentStreamLoader
> subclass.
Because I started indexing by lucene and for now, it's impossible to change
it to Solr (although I know the benefits). Maybe in the future.
So do you know how I should index the field in Lucene? I need to know the
exact type for this field. I think Lucene index fields as String by default,
right?
Thanks in advance!
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-spatial-search-within-the-polygon-tp4101147p4122640.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr spatial search within the polygon
Posted by "Smiley, David W." <ds...@mitre.org>.
On 3/10/14, 12:56 PM, "javinsnc" <ja...@mitula.com> wrote:
>>>
>>>/*********************/
>>>/* Document contents */
>>>/*********************/
>>>I have tried with 3 different content for my documents (lat-lon refers
>>>to
>>>Madrid, Spain):
>>
>> Um…. Just to be absolutely sure, are you adding the data in Solr’s XML
>> format, which is this?:
>>
>> XML Formatted Index Updates
>>
>><https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+
>>Index
>> +Handlers#UploadingDatawithIndexHandlers-XMLFormattedIndexUpdates>
>>
>> I am not sure If I understand you properly. I index the documents with
>> Lucene, not with not with update handler of solr. Maybe here is the
>> problem. Can you set the type of the field (apart from
>> <
>> fieldType defined in schema.xml) when indexing in Lucene? Months ago, I
>> needed to index a LONG field, I do the trick with this aproach and it
>> works.
>>
>> One thing is how Solr retrieve he data (defined in schema.xml with its
>> <fieldType
>> >
>> ) and other thing is how lucene index the field, right?
This is indeed the source of the problem.
Why do you index with Lucene’s API and not Solr’s? Solr not only has a
web-service API but it also has the SolrJ API that can embed Solr —
EmbeddedSolrServer. I only recommend embedding Solr in limited
circumstances as it’s more flexible and usually plenty fast to communicate
with Solr normally, or to easily customize Solr to load data from a custom
file and so the indexing is all in-process. You would do the latter with
either a custom DataImportHandler piece or a “ContentStreamLoader subclass.
>>
>>
>> The examples you give below are the *output* XML format which is not the
>> same as the input format. In particular you don’t give arrays of values
>> to Solr; you simply give more than one field element that has the same
>> name.
>>>
>>>
>>>a) As it is WKT format, I tried "longitude latitude" (x y)
>>>
>> <doc>
>>>
>> <arr name="LOCATION">
>>>
>> <str>
>> -3.69278 40.442179
>> </str>
>>>
>> </arr>
>>>
>> </doc>
>> That should work but I don’t recommend that, as a matter of taste, if
>>all
>> your data is in latitude & longitude, as opposed to projected data or
>>any
>> other spatial data.
>>
>> What do you recommend?
>>
>> With "all your data is in latitude & longitude", do you refer that every
>> doc in the index has only the field LOCATION? If the answer is yes, then
>> no, there is more fields in all the documents.
I’m only talking about the spatial field. I mean if your *spatial data*
is entirely data points where the two dimensions are latitude and
longitude on the surface of the earth (or hypothetically some other
spherical place).
~ David
Re: Solr spatial search within the polygon
Posted by javinsnc <ja...@mitula.com>.
David Smiley (@MITRE.org) wrote
> On 3/10/14, 6:45 AM, "Javi" <
> javiersangrador@
> > wrote:
>
>>/**************/
>>/* 1. library */
>>/**************/
>>
>>(1) I use "jts-1.13.jar" and "spatial4j-0.4.1.jar"
>>(I think they are the latest version)
>
> You should only need to add JTS; spatial4j is included in Solr. Where
> exactly did you put it?
>
> \solr-4.6\solr-webapp\webapp\WEB-INF\lib
> As the other lucene librarires. If I delete from this path, solr returns
> me an error, so I think it's in the proper place.
>
>>
>>/*****************/
>>/* 2. schema.xml */
>>/*****************/
>>
> <fieldType name="locationSolr"
>>
> class="solr.SpatialRecursivePrefixTreeFieldType"
>>spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFac
>>to
>>ry" distErrPct="0.025" maxDistErr="0.000009" units="degrees" />
>>
>>(I omit geo="true" because it is the default)
>>
>>...
>>
>>
> <field name="LOCATION" type="locationSolr" indexed="true" stored="true"/>
>>
>>(Here I dont know what means if I add multiValued="true")
>
> How many points might there be in this field for a given document? 0 or
> 1? Don’t set multiValued=true but if you expect possibly more than 1 then
> set it to true.
>
> Just one point, so I omit multiValued. Thanks.
>
>>
>>/*********************/
>>/* Document contents */
>>/*********************/
>>I have tried with 3 different content for my documents (lat-lon refers to
>>Madrid, Spain):
>
> Um…. Just to be absolutely sure, are you adding the data in Solr’s XML
> format, which is this?:
>
> XML Formatted Index Updates
> <https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index
> +Handlers#UploadingDatawithIndexHandlers-XMLFormattedIndexUpdates>
>
> I am not sure If I understand you properly. I index the documents with
> Lucene, not with not with update handler of solr. Maybe here is the
> problem. Can you set the type of the field (apart from
> <
> fieldType defined in schema.xml) when indexing in Lucene? Months ago, I
> needed to index a LONG field, I do the trick with this aproach and it
> works.
>
> One thing is how Solr retrieve he data (defined in schema.xml with its
> <fieldType
> >
> ) and other thing is how lucene index the field, right?
>
>
> The examples you give below are the *output* XML format which is not the
> same as the input format. In particular you don’t give arrays of values
> to Solr; you simply give more than one field element that has the same
> name.
>
>>
>>
>>a) As it is WKT format, I tried "longitude latitude" (x y)
>>
> <doc>
>>
> <arr name="LOCATION">
>>
> <str>
> -3.69278 40.442179
> </str>
>>
> </arr>
>>
> </doc>
> That should work but I don’t recommend that, as a matter of taste, if all
> your data is in latitude & longitude, as opposed to projected data or any
> other spatial data.
>
> What do you recommend?
>
> With "all your data is in latitude & longitude", do you refer that every
> doc in the index has only the field LOCATION? If the answer is yes, then
> no, there is more fields in all the documents.
>
>>
>>b) As it is WKT format, I tried "POINT(longitude latitude) (x y)
>>
>>
> <doc>
>>
> <arr name="LOCATION">
>>
> <str>
> POINT(-3.69278 40.442179)
> </str>
>>
> </arr>
>>
> </doc>
>>
>>And
>
> Again, that should work but see my comment above.
>
>>
>>c) I tried no WKT format by adding a comma and using "longitude,latitude"
>>
>>
> <doc>
>>
> <arr name="LOCATION">
>>
> <str>
> 40.442179,-3.69278
> </str>
>>
> </arr>
>>
> </doc>
> That is *wrong*. Remove the comma and it will then be okay. But again,
> see my earlier advise on lat & lon data.
>
>>
>>d) I tried no WKT format by adding a comma and using "latitude,longitude"
>>
>>
> <doc>
>>
> <arr name="LOCATION">
>>
> <str>
> -3.69278,40.442179
> </str>
>>
> </arr>
>>
> </doc>
> But that isn’t latitude then longitude of Madrid; you have it reversed.
> “latitude,longitude” of Madrid is “40.442719,-3.69278”.
>
>>
>>/*****************/
>>/* My solr query */
>>/*****************/
>>
>>a)
>>_Description: This POLYGON (in WKT format, so "longitude latitude") is a
>>triangle that cover Madrid at all, so my point would be inside them.
>>_Result: Query return 0 documents (which is wrong).
>>
>>http://localhost:8983/solr/pisos22/select?q=*%3A*&
>>fl=LOCATION&
>>wt=xml&
>>indent=true&
>>fq=LOCATION:"IsWithin(POLYGON((
>>-3.732605 40.531415,
>>-3.856201 40.336993,
>>-3.493652 40.332806,
>>-3.732605 40.531415
>>))) distErrPct=0"
>>
>>b)
>>_Descripcion: This POLYGON (in WKT format, so "longitude latitude") is a
>>rectangle out of Madrid, so my point would not be inside them.
>>_Result: Query return 0 documents (which is correct).
>>
>>http://localhost:8983/solr/pisos22/select?q=*%3A*&
>>fl=LOCATION&
>>wt=xml&
>>indent=true&
>>fq=LOCATION:"IsWithin(POLYGON((
>>-4.0594 40.8708,
>>-4.0621 40.7211 ,
>>-3.8095 40.7127,
>>-3.8232 40.8687,
>>-4.0594 40.8708
>>))) distErrPct=0"
>>
>>***I also tried modifying the order of lat/lon but I am not able to find
>>out
>>the solution to make it work.
>
> The “x y” order looks good. “IsWithin” should work but if all your
> indexed data is points then use “Intersects” which is much faster.
>
> As a sanity check can you simply do a {!geofilt} query with the “pt” set
> to madrid and a hundred kilometers or whatever?
>
> I have tried {!geofilt pt=40.442179,-3.69278 sfield=LOCATION d=100} in FQ
> and returned 0 docs :(((
>
> ~ David
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-spatial-search-within-the-polygon-tp4101147p4122623.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr spatial search within the polygon
Posted by "Smiley, David W." <ds...@mitre.org>.
On 3/10/14, 12:12 PM, "Smiley, David W." <ds...@mitre.org> wrote:
>>
>>
>>
>>c) I tried no WKT format by adding a comma and using "longitude,latitude"
>>
>><doc>
>> <arr name="LOCATION">
>> <str>40.442179,-3.69278</str>
>> </arr>
>></doc>
>
>That is *wrong*. Remove the comma and it will then be okay. But again,
>see my earlier advise on lat & lon data.
Whoops; I mean… “-3.69 40.44” would be a valid way — X Y order.
Re: Solr spatial search within the polygon
Posted by "Smiley, David W." <ds...@mitre.org>.
On 3/10/14, 6:45 AM, "Javi" <ja...@mitula.com> wrote:
>Hi all.
>
>I need your help! I have read every post about Spatial in Solr because I
>need to check if a point (latitude,longitude) is inside a Polygon.
>
>/**************/
>/* 1. library */
>/**************/
>
>(1) I use "jts-1.13.jar" and "spatial4j-0.4.1.jar"
>(I think they are the latest version)
You should only need to add JTS; spatial4j is included in Solr. Where
exactly did you put it?
>
>/*****************/
>/* 2. schema.xml */
>/*****************/
><fieldType name="locationSolr"
>class="solr.SpatialRecursivePrefixTreeFieldType"
>spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFac
>to
>ry" distErrPct="0.025" maxDistErr="0.000009" units="degrees" />
>
>(I omit geo="true" because it is the default)
>
>...
>
><field name="LOCATION" type="locationSolr" indexed="true" stored="true"/>
>
>(Here I dont know what means if I add multiValued="true")
How many points might there be in this field for a given document? 0 or
1? Don’t set multiValued=true but if you expect possibly more than 1 then
set it to true.
>
>/*********************/
>/* Document contents */
>/*********************/
>I have tried with 3 different content for my documents (lat-lon refers to
>Madrid, Spain):
Um…. Just to be absolutely sure, are you adding the data in Solr’s XML
format, which is this?:
XML Formatted Index Updates
<https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index
+Handlers#UploadingDatawithIndexHandlers-XMLFormattedIndexUpdates>
The examples you give below are the *output* XML format which is not the
same as the input format. In particular you don’t give arrays of values
to Solr; you simply give more than one field element that has the same
name.
>
>
>a) As it is WKT format, I tried "longitude latitude" (x y)
><doc>
> <arr name="LOCATION">
> <str>-3.69278 40.442179</str>
> </arr>
></doc>
That should work but I don’t recommend that, as a matter of taste, if all
your data is in latitude & longitude, as opposed to projected data or any
other spatial data.
>
>b) As it is WKT format, I tried "POINT(longitude latitude) (x y)
>
><doc>
> <arr name="LOCATION">
> <str>POINT(-3.69278 40.442179)</str>
> </arr>
></doc>
>
>And
Again, that should work but see my comment above.
>
>c) I tried no WKT format by adding a comma and using "longitude,latitude"
>
><doc>
> <arr name="LOCATION">
> <str>40.442179,-3.69278</str>
> </arr>
></doc>
That is *wrong*. Remove the comma and it will then be okay. But again,
see my earlier advise on lat & lon data.
>
>d) I tried no WKT format by adding a comma and using "latitude,longitude"
>
><doc>
> <arr name="LOCATION">
> <str>-3.69278,40.442179</str>
> </arr>
></doc>
But that isn’t latitude then longitude of Madrid; you have it reversed.
“latitude,longitude” of Madrid is “40.442719,-3.69278”.
>
>/*****************/
>/* My solr query */
>/*****************/
>
>a)
>_Description: This POLYGON (in WKT format, so "longitude latitude") is a
>triangle that cover Madrid at all, so my point would be inside them.
>_Result: Query return 0 documents (which is wrong).
>
>http://localhost:8983/solr/pisos22/select?q=*%3A*&
>fl=LOCATION&
>wt=xml&
>indent=true&
>fq=LOCATION:"IsWithin(POLYGON((
>-3.732605 40.531415,
>-3.856201 40.336993,
>-3.493652 40.332806,
>-3.732605 40.531415
>))) distErrPct=0"
>
>b)
>_Descripcion: This POLYGON (in WKT format, so "longitude latitude") is a
>rectangle out of Madrid, so my point would not be inside them.
>_Result: Query return 0 documents (which is correct).
>
>http://localhost:8983/solr/pisos22/select?q=*%3A*&
>fl=LOCATION&
>wt=xml&
>indent=true&
>fq=LOCATION:"IsWithin(POLYGON((
>-4.0594 40.8708,
>-4.0621 40.7211 ,
>-3.8095 40.7127,
>-3.8232 40.8687,
>-4.0594 40.8708
>))) distErrPct=0"
>
>***I also tried modifying the order of lat/lon but I am not able to find
>out
>the solution to make it work.
The “x y” order looks good. “IsWithin” should work but if all your
indexed data is points then use “Intersects” which is much faster.
As a sanity check can you simply do a {!geofilt} query with the “pt” set
to madrid and a hundred kilometers or whatever?
~ David
Re: Solr spatial search within the polygon
Posted by Javi <ja...@mitula.com>.
Hi all.
I need your help! I have read every post about Spatial in Solr because I
need to check if a point (latitude,longitude) is inside a Polygon.
/**************/
/* 1. library */
/**************/
(1) I use "jts-1.13.jar" and "spatial4j-0.4.1.jar"
(I think they are the latest version)
/*****************/
/* 2. schema.xml */
/*****************/
<fieldType name="locationSolr"
class="solr.SpatialRecursivePrefixTreeFieldType"
spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFacto
ry" distErrPct="0.025" maxDistErr="0.000009" units="degrees" />
(I omit geo="true" because it is the default)
...
<field name="LOCATION" type="locationSolr" indexed="true" stored="true"/>
(Here I dont know what means if I add multiValued="true")
/*********************/
/* Document contents */
/*********************/
I have tried with 3 different content for my documents (lat-lon refers to
Madrid, Spain):
a) As it is WKT format, I tried "longitude latitude" (x y)
<doc>
<arr name="LOCATION">
<str>-3.69278 40.442179</str>
</arr>
</doc>
b) As it is WKT format, I tried "POINT(longitude latitude) (x y)
<doc>
<arr name="LOCATION">
<str>POINT(-3.69278 40.442179)</str>
</arr>
</doc>
and
c) I tried no WKT format by adding a comma and using "longitude,latitude"
<doc>
<arr name="LOCATION">
<str>40.442179,-3.69278</str>
</arr>
</doc>
d) I tried no WKT format by adding a comma and using "latitude,longitude"
<doc>
<arr name="LOCATION">
<str>-3.69278,40.442179</str>
</arr>
</doc>
/*****************/
/* My solr query */
/*****************/
a)
_Description: This POLYGON (in WKT format, so "longitude latitude") is a
triangle that cover Madrid at all, so my point would be inside them.
_Result: Query return 0 documents (which is wrong).
http://localhost:8983/solr/pisos22/select?q=*%3A*&
fl=LOCATION&
wt=xml&
indent=true&
fq=LOCATION:"IsWithin(POLYGON((
-3.732605 40.531415,
-3.856201 40.336993,
-3.493652 40.332806,
-3.732605 40.531415
))) distErrPct=0"
b)
_Descripcion: This POLYGON (in WKT format, so "longitude latitude") is a
rectangle out of Madrid, so my point would not be inside them.
_Result: Query return 0 documents (which is correct).
http://localhost:8983/solr/pisos22/select?q=*%3A*&
fl=LOCATION&
wt=xml&
indent=true&
fq=LOCATION:"IsWithin(POLYGON((
-4.0594 40.8708,
-4.0621 40.7211 ,
-3.8095 40.7127,
-3.8232 40.8687,
-4.0594 40.8708
))) distErrPct=0"
***I also tried modifying the order of lat/lon but I am not able to find out
the solution to make it work.
Re: Solr spatial search within the polygon
Posted by "Smiley, David W." <ds...@mitre.org>.
Hi.
It's clear there is an ordering problem in your latitudes and longitudes.
If indeed you intend to index latitude 9.44Š and longitude 76.45Š as you
said, then you are indexing it correctly. You may also choose to index in
WKT format, which would be POINT(76.45 9.44) but either is fine. However
your query shape shows that you have latitudes and longitudes in the wrong
order. Your query shape is in WKT format, and WKT is in "x y" order. Yet
you have given it in latitude longitude order which is reversed.
Another observation is that you are using the "IsWithin" predicate. There
isn't anything wrong with that, but if all of your indexed data is points,
and you only need at least some of a document's multi-valued point data to
be in the shape for a match (I.e. You don't care if one point is in but
another is out for the same document), then "Intersects" is much faster.
IsWithin must look in all places outside your shape (in addition to
inside) to attempt to ensure it doesn't include external data.
~ David
On 11/15/13 1:36 AM, "Dhanesh Radhakrishnan" <dh...@hifx.co.in> wrote:
>Hi,
>I'm experimenting with solr spatial search, with plotting points in the
>map
>(Latitude and longitude) and based on the value I need to get the result.
>
>As the first step I've defined the filed type as
><fieldType name="location_rpt"
>class="solr.SpatialRecursivePrefixTreeFieldType"
>spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFac
>tory"
>distErrPct="0.025" maxDistErr="0.000009" units="degrees" />
>
>And then added the field *location* as type *location_rpt*
> <field name="location" type="location_rpt" indexed="true" stored="true"
>multiValued="true" />
>
>Indexed the location filed as $latitude, $longitude So that data in the
>*location* will be like
>
>"location":["9.445890,76.540970"]
>
>Next I draw a polygon in google map and collected the lat and lng
>coordinates of the polygon.
>It will be like
>9.472992 76.540817, 9.441328 76.523651 , 9.433708 76.555065 , 9.458092
>76.572403, 9.472992 76.540817
>
>Based on this coordinates I performed a query in solr like this
>localhost:8983/solr/ha_poc/select?fl=id,name,district,locality&wt=json&
>json.nl=map&q=*:*&fq=location:"IsWithin(POLYGON((9.472992 76.540817,
>9.441328 76.523651 , 9.433708 76.555065 , 9.458092 76.572403, 9.472992
>76.540817))) distErrPct=0"
>
>
>But I didn't get the result from the solr as I expected.
>
>{
>
> - "responseHeader": {
> - "status": 0,
> - "QTime": 2
> },
> - "response": {
> - "numFound": 0,
> - "start": 0,
> - "docs": [ ]
> }
>
>}
>
>
>Is there anything that I missed ???
>Can anybody help me in solving this issue with solr spatial search.
>I'm using Solr 4.4.0
>Added an additional dependency JTS jar file for the support of polygon
>/lib/ext/jts-1.13.jar
>
>--
>*dhanesh s.r*