You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Adam Estrada <es...@gmail.com> on 2011/02/09 05:18:15 UTC
[WKT] Spatial Searching
I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
Thanks,
Adam
Re: [WKT] Spatial Searching
Posted by Ryan McKinley <ry...@gmail.com>.
> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
check:
http://code.google.com/p/lucene-spatial-playground/
This is my sketch / soon-to-be-proposal for what I think lucene
spatial should look like. It includes a WKTField that can do complex
geometry queries:
https://lucene-spatial-playground.googlecode.com/svn/trunk/spatial-lucene/src/main/java/org/apache/lucene/spatial/search/jts/
ryan
Re: [WKT] Spatial Searching
Posted by Estrada Groups <es...@gmail.com>.
Thought I would share this on web mapping...it's a great write up and something to consider when talking about working with spatial data.
http://www.tokumine.com/2010/09/20/gis-data-payload-sizes/
Adam
On Feb 9, 2011, at 7:03 AM, Grant Ingersoll <gs...@apache.org> wrote:
> The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out.
>
> On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
>
>> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>>
>> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>>
>> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>>
>> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
>> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
>> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
>> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
>> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
>> Thanks,
>> Adam
>>
>>
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
Re: [WKT] Spatial Searching
Posted by Estrada Groups <es...@gmail.com>.
How could i stub this out not being a java guy? What is needed in order to do this?
Licensing is always going to be an issue with JTS which is why I am interested in the project SIS sitting in incubation right now.
I willing to put forth the effort if I had a little direction from the peanut gallery ;-)
Adam
On Feb 9, 2011, at 7:03 AM, Grant Ingersoll <gs...@apache.org> wrote:
> The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out.
>
> On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
>
>> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>>
>> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>>
>> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>>
>> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
>> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
>> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
>> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
>> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
>> Thanks,
>> Adam
>>
>>
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
Re: [WKT] Spatial Searching
Posted by Adam Estrada <es...@gmail.com>.
Thought I would share this on web mapping...it's a great write up and something to consider when talking about working with spatial data.
http://www.tokumine.com/2010/09/20/gis-data-payload-sizes/
Adam
On Feb 9, 2011, at 7:03 AM, Grant Ingersoll wrote:
> The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out.
>
> On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
>
>> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>>
>> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>>
>> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>>
>> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
>> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
>> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
>> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
>> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
>> Thanks,
>> Adam
>>
>>
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
Re: [WKT] Spatial Searching
Posted by Adam Estrada <es...@gmail.com>.
Grant,
How could i stub this out not being a java guy? What is needed in order to do this?
Licensing is always going to be an issue with JTS which is why I am interested in the project SIS sitting in incubation right now.
I'm willing to put forth the effort if I had a little direction on how to implement it from the peanut gallery ;-)
Adam
On Feb 9, 2011, at 7:03 AM, Grant Ingersoll wrote:
> The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out.
>
> On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
>
>> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>>
>> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>>
>> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>>
>> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
>> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
>> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
>> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
>> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
>> Thanks,
>> Adam
>>
>>
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
Re: [WKT] Spatial Searching
Posted by Grant Ingersoll <gs...@apache.org>.
The show stopper for JTS is it's license, unfortunately. Otherwise, I think it would be done already! We could, since it's LGPL, make it an optional dependency, assuming someone can stub it out.
On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>
> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>
> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>
> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
> Thanks,
> Adam
>
>
>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Re: [WKT] Spatial Searching
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
+1 to David's patch from SOLR-2155.
It would be great to implement. Great job using GDAL on converting the WKT Adam!
Cheers,
Chris
On Feb 8, 2011, at 8:18 PM, Adam Estrada wrote:
> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>
> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>
> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>
> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
> Thanks,
> Adam
>
>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [WKT] Spatial Searching
Posted by Estrada Groups <es...@gmail.com>.
Outstanding! Thanks David...I can't wait to take a look at it.
Adam
Sent from my iPhone
On Mar 28, 2011, at 2:16 PM, "Smiley, David W." <ds...@mitre.org> wrote:
> (This is one of those messages that I would have responded to at the time if I only noticed it.)
>
> There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient.
>
> I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS.
>
> ~ David Smiley
> Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
>
> On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
>
>> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>>
>> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>>
>> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>>
>> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
>> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
>> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
>> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
>> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
>> Thanks,
>> Adam
Re: [WKT] Spatial Searching
Posted by Walter Underwood <wu...@wunderwood.org>.
On Mar 29, 2011, at 8:12 AM, Mattmann, Chris A (388J) wrote:
>>
>> RE SIS... I wonder how the expertise on that project compares to that of JTS's Martin Davis -- an expert, and the library has been in use for 10 years.
>
> Time will tell. I'd favor the Apache model where instead of name dropping we rely on the collective expertise p a group of like minded individuals
Lucene was originally the work of one expert. That's not a bad way to start.
wunder
--
Walter Underwood
Re: [WKT] Spatial Searching
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
>
> It's not saying specifically we can't compile against LGPL, it's ambiguously saying "include". I take that to mean the result of the build -- e.g. class and jar files, which may not include LGPL.
This uncertainty was was enough to make it a non starter for us.
> RE SIS... I wonder how the expertise on that project compares to that of JTS's Martin Davis -- an expert, and the library has been in use for 10 years.
Time will tell. I'd favor the Apache model where instead of name dropping we rely on the collective expertise p a group of like minded individuals.
Cheers,
Chris
>
> ~ David
> ________________________________________
> From: Mattmann, Chris A (388J) [chris.a.mattmann@jpl.nasa.gov]
> Sent: Tuesday, March 29, 2011 1:00 AM
> To: solr-user@lucene.apache.org
> Cc: Adam Estrada
> Subject: Re: [WKT] Spatial Searching
>
> LGPL licenses and Apache aren't exactly compatible, see:
>
> http://www.apache.org/legal/3party.html#transition-examples-lgpl
> http://www.apache.org/legal/resolved.html#category-x
>
> In practice, this was the reason we started the SIS project.
>
> Cheers,
> Chris
>
> On Mar 28, 2011, at 11:16 AM, Smiley, David W. wrote:
>
>> (This is one of those messages that I would have responded to at the time if I only noticed it.)
>>
>> There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient.
>>
>> I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS.
>>
>> ~ David Smiley
>> Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
>>
>> On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
>>
>>> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>>>
>>> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>>>
>>> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>>>
>>> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
>>> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
>>> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
>>> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
>>> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
>>> Thanks,
>>> Adam
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
RE: [WKT] Spatial Searching
Posted by "Smiley, David W." <ds...@mitre.org>.
Thanks for the links Chris. I think my approach in SOLR-2155 complies with those rules. The only part that concerns me wether this is true is the rule regarding the default action of a build script:
YOU MUST NOT distribute build scripts or documentation within an Apache product with the purpose of causing the default/standard build of an Apache product to include any part of a prohibited work.
It's not saying specifically we can't compile against LGPL, it's ambiguously saying "include". I take that to mean the result of the build -- e.g. class and jar files, which may not include LGPL.
RE SIS... I wonder how the expertise on that project compares to that of JTS's Martin Davis -- an expert, and the library has been in use for 10 years.
~ David
________________________________________
From: Mattmann, Chris A (388J) [chris.a.mattmann@jpl.nasa.gov]
Sent: Tuesday, March 29, 2011 1:00 AM
To: solr-user@lucene.apache.org
Cc: Adam Estrada
Subject: Re: [WKT] Spatial Searching
LGPL licenses and Apache aren't exactly compatible, see:
http://www.apache.org/legal/3party.html#transition-examples-lgpl
http://www.apache.org/legal/resolved.html#category-x
In practice, this was the reason we started the SIS project.
Cheers,
Chris
On Mar 28, 2011, at 11:16 AM, Smiley, David W. wrote:
> (This is one of those messages that I would have responded to at the time if I only noticed it.)
>
> There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient.
>
> I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS.
>
> ~ David Smiley
> Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
>
> On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
>
>> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>>
>> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>>
>> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>>
>> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
>> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
>> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
>> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
>> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
>> Thanks,
>> Adam
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [WKT] Spatial Searching
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
LGPL licenses and Apache aren't exactly compatible, see:
http://www.apache.org/legal/3party.html#transition-examples-lgpl
http://www.apache.org/legal/resolved.html#category-x
In practice, this was the reason we started the SIS project.
Cheers,
Chris
On Mar 28, 2011, at 11:16 AM, Smiley, David W. wrote:
> (This is one of those messages that I would have responded to at the time if I only noticed it.)
>
> There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient.
>
> I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS.
>
> ~ David Smiley
> Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
>
> On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
>
>> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>>
>> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>>
>> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>>
>> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
>> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
>> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
>> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
>> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
>> Thanks,
>> Adam
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [WKT] Spatial Searching
Posted by "Smiley, David W." <ds...@mitre.org>.
(This is one of those messages that I would have responded to at the time if I only noticed it.)
There is not yet indexing of arbitrary shapes (i.e. your data can only be points), but with SOLR-2155 you can query via WKT thanks to JTS. If you want to index shapes then you'll have to wait a month or two for work that is underway right now. It's coming; be patient.
I don't see the LGPL licensing as a problem; it's *L*GPL, not GPL, after all. In SOLR-2155 the patch I take measures to download this library dynamically at build time and compile against it. JTS need not ship with Solr; the user can get it themselves if they want this capability. Non-JTS query shapes should work without the presence of JTS.
~ David Smiley
Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
On Feb 8, 2011, at 11:18 PM, Adam Estrada wrote:
> I just came across a ~nudge post over in the SIS list on what the status is for that project. This got me looking more in to spatial mods with Solr4.0. I found this enhancement in Jira. https://issues.apache.org/jira/browse/SOLR-2155. In this issue, David mentions that he's already integrated JTS in to Solr4.0 for querying on polygons stored as WKT.
>
> It's relatively easy to get WKT strings in to Solr but does the Field type exist yet? Is there a patch or something that I can test out?
>
> Here's how I would do it using GDAL/OGR and the already existing csv update handler. http://www.gdal.org/ogr/drv_csv.html
>
> ogr2ogr -f CSV output.csv input.shp -lco GEOMETRY=AS_WKT
> This converts a shapefile to a csv with the geometries in tact in the form of WKT. You can then get the data in to Solr by running the following command.
> curl "http://localhost:8983/solr/update/csv?commit=true&separator=%2C&fieldnames=id,attr1,attr2,attr3,geom&stream.file=C:\tmp\output.csv&overwrite=true&stream.contentType=text/plain;charset=utf-8"
> There are lots of flavors of geometries so I suspect that this will be a daunting task but because JTS recognizes each geometry type it should be possible to work with them.
> Does anyone know of a patch or even when this functionality might be included in to Solr4.0? I need to query for polygons ;-)
> Thanks,
> Adam