You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Sean Bigdatafun <se...@gmail.com> on 2011/04/07 04:31:15 UTC
difference between geospatial search from database angle and from
solr angle
I understand Solr can do pretty powerful geospatial search
http://www.ibm.com/developerworks/java/library/j-spatial/
<http://www.ibm.com/developerworks/java/library/j-spatial/>But I also
understand lots of DB researchers have done lots of geospatial related work,
can someone give an overview of the difference from the different angel?
Thanks,
--
--Sean
Re: difference between geospatial search from database angle and from
solr angle
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Sun, Apr 10, 2011 at 5:24 PM, Lance Norskog <go...@gmail.com> wrote:
> Wait! How can you do distance calculations across different shards efficiently?
Basic spatial search (bounding box filter, radius filter, sort by
distance) has no cross-document component, so "it just works" with
distributed search.
-Yonik
http://www.lucenerevolution.org -- Lucene/Solr User Conference, May
25-26, San Francisco
Re: difference between geospatial search from database angle and from
solr angle
Posted by Lance Norskog <go...@gmail.com>.
Wait! How can you do distance calculations across different shards efficiently?
On Thu, Apr 7, 2011 at 7:19 AM, Smiley, David W. <ds...@mitre.org> wrote:
> I haven't used PostGIS so I can't offer a real comparison. I think if you were to try out both, you'd be impressed with Solr's performance/scalability thanks in large part to its sharding. But for "functionality richness" in so far as geospatial is concerned, that's where Solr currently comes short. It just has the basic stuff 80% of people want.
>
> ~ David Smiley
> Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
>
> On Apr 7, 2011, at 2:24 AM, Sean Bigdatafun wrote:
>
>> Thanks, David.
>>
>> I am thinking of a scenario that billions of objects, whose indices are too
>> big for a single machine to serve the indexing, to serve the querying. Is
>> there any sharding mechanism?
>>
>>
>> Can you give a comparison between solr-based geospatial search and PostGIS
>> based geospatial search?
>> * scalability
>> * functionality richness
>> * incremental indexing (re-indexing) cost
>> * query cost
>> * sharding scheme support
>
>
>
>
>
>
>
--
Lance Norskog
goksron@gmail.com
Re: difference between geospatial search from database angle and
from solr angle
Posted by "Smiley, David W." <ds...@mitre.org>.
I haven't used PostGIS so I can't offer a real comparison. I think if you were to try out both, you'd be impressed with Solr's performance/scalability thanks in large part to its sharding. But for "functionality richness" in so far as geospatial is concerned, that's where Solr currently comes short. It just has the basic stuff 80% of people want.
~ David Smiley
Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
On Apr 7, 2011, at 2:24 AM, Sean Bigdatafun wrote:
> Thanks, David.
>
> I am thinking of a scenario that billions of objects, whose indices are too
> big for a single machine to serve the indexing, to serve the querying. Is
> there any sharding mechanism?
>
>
> Can you give a comparison between solr-based geospatial search and PostGIS
> based geospatial search?
> * scalability
> * functionality richness
> * incremental indexing (re-indexing) cost
> * query cost
> * sharding scheme support
Re: difference between geospatial search from database angle and from
solr angle
Posted by Erick Erickson <er...@gmail.com>.
Have you looked at solr sharding?
Best
Erick
On Thu, Apr 7, 2011 at 2:24 AM, Sean Bigdatafun
<se...@gmail.com>wrote:
> Thanks, David.
>
> I am thinking of a scenario that billions of objects, whose indices are too
> big for a single machine to serve the indexing, to serve the querying. Is
> there any sharding mechanism?
>
>
> Can you give a comparison between solr-based geospatial search and PostGIS
> based geospatial search?
> * scalability
> * functionality richness
> * incremental indexing (re-indexing) cost
> * query cost
> * sharding scheme support
>
>
>
> On Wed, Apr 6, 2011 at 9:42 PM, David Smiley (@MITRE.org) <
> DSMILEY@mitre.org
> > wrote:
>
> > Sean,
> > Geospatial search in Lucene/Solr is of course implemented based on
> > Lucene's underlying index technology. That technology was originally just
> > for text but it's been adapted very successfully for numerics and
> querying
> > ranges too. The only mature geospatial field type in Solr 3.1 is
> LatLonType
> > which under the hood is simply a pair of latitude & longitude numeric
> > fields. There really isn't anything sophisticated (geospatially
> speaking)
> > in Solr 3.1. I'm not sure what sort of geospatial DB research you have in
> > mind but I would expect other systems would be free to use an indexing
> > strategy designed for spatial such as "R-Trees". Nevertheless, I think
> > Lucene offers the underlying primitives to compete with systems using
> other
> > technologies. Case in point is my patch SOLR-2155 which indexes a single
> > point in the form of a "geohash" at multiple resolutions (geohash lengths
> > AKA spatial prefixes / grids) and uses a recursive algorithm to
> efficiently
> > query an arbitrary shape. It's quite fast and bests LatLonType already;
> > and
> > there's a lot more I can do to make it faster.
> > This is definitely a field of interest and a growing one in the
> > Lucene/Solr community. There are even some external spatial providers
> > (JTeam, MetaCarta) and I'm partnering with other individuals to create a
> > new
> > one. Expect to see more in the coming months. If you're looking for
> some
> > specific geospatial capabilities then let us know.
> >
> > ~ David Smiley
> > Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
> >
> > -----
> > Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/difference-between-geospatial-search-from-database-angle-and-from-solr-angle-tp2788442p2788972.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>
>
>
> --
> --Sean
>
Re: difference between geospatial search from database angle and from
solr angle
Posted by Sean Bigdatafun <se...@gmail.com>.
Thanks, David.
I am thinking of a scenario that billions of objects, whose indices are too
big for a single machine to serve the indexing, to serve the querying. Is
there any sharding mechanism?
Can you give a comparison between solr-based geospatial search and PostGIS
based geospatial search?
* scalability
* functionality richness
* incremental indexing (re-indexing) cost
* query cost
* sharding scheme support
On Wed, Apr 6, 2011 at 9:42 PM, David Smiley (@MITRE.org) <DSMILEY@mitre.org
> wrote:
> Sean,
> Geospatial search in Lucene/Solr is of course implemented based on
> Lucene's underlying index technology. That technology was originally just
> for text but it's been adapted very successfully for numerics and querying
> ranges too. The only mature geospatial field type in Solr 3.1 is LatLonType
> which under the hood is simply a pair of latitude & longitude numeric
> fields. There really isn't anything sophisticated (geospatially speaking)
> in Solr 3.1. I'm not sure what sort of geospatial DB research you have in
> mind but I would expect other systems would be free to use an indexing
> strategy designed for spatial such as "R-Trees". Nevertheless, I think
> Lucene offers the underlying primitives to compete with systems using other
> technologies. Case in point is my patch SOLR-2155 which indexes a single
> point in the form of a "geohash" at multiple resolutions (geohash lengths
> AKA spatial prefixes / grids) and uses a recursive algorithm to efficiently
> query an arbitrary shape. It's quite fast and bests LatLonType already;
> and
> there's a lot more I can do to make it faster.
> This is definitely a field of interest and a growing one in the
> Lucene/Solr community. There are even some external spatial providers
> (JTeam, MetaCarta) and I'm partnering with other individuals to create a
> new
> one. Expect to see more in the coming months. If you're looking for some
> specific geospatial capabilities then let us know.
>
> ~ David Smiley
> Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
>
> -----
> Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/difference-between-geospatial-search-from-database-angle-and-from-solr-angle-tp2788442p2788972.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
--Sean
Re: difference between geospatial search from database angle and
from solr angle
Posted by "David Smiley (@MITRE.org)" <DS...@mitre.org>.
Sean,
Geospatial search in Lucene/Solr is of course implemented based on
Lucene's underlying index technology. That technology was originally just
for text but it's been adapted very successfully for numerics and querying
ranges too. The only mature geospatial field type in Solr 3.1 is LatLonType
which under the hood is simply a pair of latitude & longitude numeric
fields. There really isn't anything sophisticated (geospatially speaking)
in Solr 3.1. I'm not sure what sort of geospatial DB research you have in
mind but I would expect other systems would be free to use an indexing
strategy designed for spatial such as "R-Trees". Nevertheless, I think
Lucene offers the underlying primitives to compete with systems using other
technologies. Case in point is my patch SOLR-2155 which indexes a single
point in the form of a "geohash" at multiple resolutions (geohash lengths
AKA spatial prefixes / grids) and uses a recursive algorithm to efficiently
query an arbitrary shape. It's quite fast and bests LatLonType already; and
there's a lot more I can do to make it faster.
This is definitely a field of interest and a growing one in the
Lucene/Solr community. There are even some external spatial providers
(JTeam, MetaCarta) and I'm partnering with other individuals to create a new
one. Expect to see more in the coming months. If you're looking for some
specific geospatial capabilities then let us know.
~ David Smiley
Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
-----
Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book
--
View this message in context: http://lucene.472066.n3.nabble.com/difference-between-geospatial-search-from-database-angle-and-from-solr-angle-tp2788442p2788972.html
Sent from the Solr - User mailing list archive at Nabble.com.