You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Doug Daniels <dd...@rooftophq.com> on 2007/10/18 17:34:42 UTC
Re: Geographical distance searching
Hi Patrick,
Are the solr components of that demo in the repository as well? I couldn't
find them there.
Best,
Doug
patrick o'leary wrote:
>
> As far as I'm concerned nothings going to beat PG's GIS calculations,
> but it's tsearch was
> a lot slower than myisam.
>
> My goal was a single solution to reduce our complexity, but am
> interested to know if combining
> both an rdbms & lucene works for you. Definitely let me know how it goes !
>
> P
>
> Guillaume Smet wrote:
>> Hi Patrick,
>>
>> On 9/27/07, patrick o'leary <po...@aol.com> wrote:
>>
>>> p.s after a little tidy up I'll be adding this to both lucene and
>>> solr's repositories if folks feel that it's a useful addition.
>>>
>>
>> It's definitely very interesting. Did you compare performances of
>> Lucene with a database allowing you to perform real GIS queries?
>> I'm more a PostgreSQL guy and I must admit we usually use cube contrib
>> or PostGIS for this sort of thing and with both, we are capable to use
>> indexes for proximity queries and they can be pretty fast. Using the
>> method you used with MySQL is definitely too slow and not used as soon
>> as you have a certain amount of data in your table.
>>
>> Regards,
>>
>>
>
> --
>
> Patrick O'Leary
>
>
> You see, wire telegraph is a kind of a very, very long cat. You pull his
> tail in New York and his head is meowing in Los Angeles.
> Do you understand this?
> And radio operates exactly the same way: you send signals here, they
> receive them there. The only difference is that there is no cat.
> - Albert Einstein
>
> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
> <http://www.linkedin.com/in/pjaol>
>
>
--
View this message in context: http://www.nabble.com/Geographical-distance-searching-tf4524338.html#a13277230
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Geographical distance searching
Posted by Chris Hostetter <ho...@fucit.org>.
: aaah -- Just to be clear, if you could augment the doc list with a calculated
: field ('distance') you would not need to extend XMLWritter - correct?
:
: And then it would also work with the JSON/pyton/php etc writers.
i don't think i ever looked at the patch in question (is it in Jira?) but
i would definitely recommend going a route like this rather then adding a
new "primitive" type that all the ResponseWriters would need to know
about.
Personally: i don't even think "augmenting DocList" should be done ...
once the distance score calculations are done, and the documents are
ordered, returning the distance as numeric value for the clients is really
no different then returning highlighting info or score explanations ... it
should be a seperate piece of info in the response.
(i think if we had it to do all over again, i would suggest the same
approach for returning the "scores" of docs ... at the moment there is
ambiguity between the relevancy score and the possibility of a field named
"score")
-Hoss
Re: Geographical distance searching
Posted by Ryan McKinley <ry...@gmail.com>.
patrick o'leary wrote:
> Actually misspoke it's the XMLWritter that's final that was a little
> annoying rather than a handler.
> Would have been nice to just extend that, and cut down on the code.
>
aaah -- Just to be clear, if you could augment the doc list with a
calculated field ('distance') you would not need to extend XMLWritter -
correct?
And then it would also work with the JSON/pyton/php etc writers.
ryan
Re: Geographical distance searching
Posted by patrick o'leary <po...@aol.com>.
Actually misspoke it's the XMLWritter that's final that was a little
annoying rather than a handler.
Would have been nice to just extend that, and cut down on the code.
P
Ryan McKinley wrote:
>>
>> SOLR-281 looks like it will solve one of my frustrations, another
>> being that the handlers were final ;-)
>
> What handlers are final that you found annoying?
>
>> Is it close to being committed to the trunk?
>>
>
> I hope so ;) Since this patch reworks the *core* query handlers
> (dismax/standard) I really want someone else to look it over before
> committing...
>
> ryan
>
>
--
Patrick O'Leary
AOL Local Search Technologies
Phone: + 1 703 265 8763
You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
Do you understand this?
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
- Albert Einstein
View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
<http://www.linkedin.com/in/pjaol>
Re: Geographical distance searching
Posted by Ryan McKinley <ry...@gmail.com>.
>
> SOLR-281 looks like it will solve one of my frustrations, another being
> that the handlers were final ;-)
What handlers are final that you found annoying?
> Is it close to being committed to the trunk?
>
I hope so ;) Since this patch reworks the *core* query handlers
(dismax/standard) I really want someone else to look it over before
committing...
ryan
Re: Geographical distance searching
Posted by patrick o'leary <po...@aol.com>.
Hi Ryan
Thanks for looking at it, yes definitely I'd like to contribute back.
I'm currently working out some of the logistics to make it easier to
maintain.
I think I'd need a centralized project to do the releases from, it will
have 2 sets
of patches in 2 different trunks (lucene & solr), until it becomes core
in lucene.
So a batteries included release will make that easier for users, and
developers alike.
Once it's accepted I'll move development to ASF.
SOLR-281 looks like it will solve one of my frustrations, another being
that the handlers were final ;-)
Is it close to being committed to the trunk?
Thanks
P
Ryan McKinley wrote:
> This looks good!
>
> Are you interested in contributing it to solr core?
>
> One major thing in the solr pipeline you may want to be aware of is
> the search component interface (SOLR-281).
>
> This would let you make simple component that adds the:
> DistanceQuery dq = new DistanceQuery(dlat,dlng,dradius);
> dsort = new DistanceSortSource(filter);
>
> and later adds the 'distance' to each result
>
> This way you could reuse the other standard search stuff (faceting,
> debugging, etc) and would not need to make your own custom
> LocalResponseWritter.
>
> ryan
>
>
> Doug Daniels wrote:
>> Hi Patrick,
>>
>> Was mainly interested in seeing how you did the RequestHandler.
>> Thanks for
>> sending the link!
>>
>> Best,
>> Doug
>>
>>
>> patrick o'leary wrote:
>>> Hi Doug
>>>
>>> What exactly are you looking for?
>>> The code for localsolr is still in dev state, but I've left my work
>>> open
>>> and available for download
>>> at http://www.nsshutdown.com/viewcvs/viewcvs.cgi/localsolr/
>>>
>>> Once I'm happy with it, I'll donate it back in the form of patches
>>> until
>>> / unless it's accepted
>>> as a contribution, depending on how folks feel.
>>>
>>> If your talking about the demo ui, it's a little piece of html & JS,
>>> you
>>> can pull directly from the jar.
>>> I've not included that in the repository.
>>>
>>> HTH
>>> P
>>>
>>> Doug Daniels wrote:
>>>> Hi Patrick,
>>>>
>>>> Are the solr components of that demo in the repository as well? I
>>>> couldn't
>>>> find them there.
>>>>
>>>> Best,
>>>> Doug
>>>>
>>>>
>>>> patrick o'leary wrote:
>>>>
>>>>> As far as I'm concerned nothings going to beat PG's GIS calculations,
>>>>> but it's tsearch was
>>>>> a lot slower than myisam.
>>>>>
>>>>> My goal was a single solution to reduce our complexity, but am
>>>>> interested to know if combining
>>>>> both an rdbms & lucene works for you. Definitely let me know how
>>>>> it goes
>>>>> !
>>>>>
>>>>> P
>>>>>
>>>>> Guillaume Smet wrote:
>>>>>
>>>>>> Hi Patrick,
>>>>>>
>>>>>> On 9/27/07, patrick o'leary <po...@aol.com> wrote:
>>>>>>
>>>>>>> p.s after a little tidy up I'll be adding this to both lucene and
>>>>>>> solr's repositories if folks feel that it's a useful addition.
>>>>>>>
>>>>>> It's definitely very interesting. Did you compare performances of
>>>>>> Lucene with a database allowing you to perform real GIS queries?
>>>>>> I'm more a PostgreSQL guy and I must admit we usually use cube
>>>>>> contrib
>>>>>> or PostGIS for this sort of thing and with both, we are capable
>>>>>> to use
>>>>>> indexes for proximity queries and they can be pretty fast. Using the
>>>>>> method you used with MySQL is definitely too slow and not used as
>>>>>> soon
>>>>>> as you have a certain amount of data in your table.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>>
>>>>> --
>>>>>
>>>>> Patrick O'Leary
>>>>>
>>>>>
>>>>> You see, wire telegraph is a kind of a very, very long cat. You
>>>>> pull his
>>>>> tail in New York and his head is meowing in Los Angeles.
>>>>> Do you understand this? And radio operates exactly the same way:
>>>>> you send signals here, they
>>>>> receive them there. The only difference is that there is no cat.
>>>>> - Albert Einstein
>>>>>
>>>>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>>>>> <http://www.linkedin.com/in/pjaol>
>>>>>
>>>>>
>>>>>
>>>>
>>> --
>>>
>>> Patrick O'Leary
>>>
>>>
>>> You see, wire telegraph is a kind of a very, very long cat. You pull
>>> his
>>> tail in New York and his head is meowing in Los Angeles.
>>> Do you understand this? And radio operates exactly the same way:
>>> you send signals here, they
>>> receive them there. The only difference is that there is no cat.
>>> - Albert Einstein
>>>
>>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>>> <http://www.linkedin.com/in/pjaol>
>>>
>>>
>>
>
>
--
Patrick O'Leary
You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
Do you understand this?
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
- Albert Einstein
View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
<http://www.linkedin.com/in/pjaol>
Re: Geographical distance searching
Posted by Ryan McKinley <ry...@gmail.com>.
This looks good!
Are you interested in contributing it to solr core?
One major thing in the solr pipeline you may want to be aware of is the
search component interface (SOLR-281).
This would let you make simple component that adds the:
DistanceQuery dq = new DistanceQuery(dlat,dlng,dradius);
dsort = new DistanceSortSource(filter);
and later adds the 'distance' to each result
This way you could reuse the other standard search stuff (faceting,
debugging, etc) and would not need to make your own custom
LocalResponseWritter.
ryan
Doug Daniels wrote:
> Hi Patrick,
>
> Was mainly interested in seeing how you did the RequestHandler. Thanks for
> sending the link!
>
> Best,
> Doug
>
>
> patrick o'leary wrote:
>> Hi Doug
>>
>> What exactly are you looking for?
>> The code for localsolr is still in dev state, but I've left my work open
>> and available for download
>> at http://www.nsshutdown.com/viewcvs/viewcvs.cgi/localsolr/
>>
>> Once I'm happy with it, I'll donate it back in the form of patches until
>> / unless it's accepted
>> as a contribution, depending on how folks feel.
>>
>> If your talking about the demo ui, it's a little piece of html & JS, you
>> can pull directly from the jar.
>> I've not included that in the repository.
>>
>> HTH
>> P
>>
>> Doug Daniels wrote:
>>> Hi Patrick,
>>>
>>> Are the solr components of that demo in the repository as well? I
>>> couldn't
>>> find them there.
>>>
>>> Best,
>>> Doug
>>>
>>>
>>> patrick o'leary wrote:
>>>
>>>> As far as I'm concerned nothings going to beat PG's GIS calculations,
>>>> but it's tsearch was
>>>> a lot slower than myisam.
>>>>
>>>> My goal was a single solution to reduce our complexity, but am
>>>> interested to know if combining
>>>> both an rdbms & lucene works for you. Definitely let me know how it goes
>>>> !
>>>>
>>>> P
>>>>
>>>> Guillaume Smet wrote:
>>>>
>>>>> Hi Patrick,
>>>>>
>>>>> On 9/27/07, patrick o'leary <po...@aol.com> wrote:
>>>>>
>>>>>
>>>>>> p.s after a little tidy up I'll be adding this to both lucene and
>>>>>> solr's repositories if folks feel that it's a useful addition.
>>>>>>
>>>>>>
>>>>> It's definitely very interesting. Did you compare performances of
>>>>> Lucene with a database allowing you to perform real GIS queries?
>>>>> I'm more a PostgreSQL guy and I must admit we usually use cube contrib
>>>>> or PostGIS for this sort of thing and with both, we are capable to use
>>>>> indexes for proximity queries and they can be pretty fast. Using the
>>>>> method you used with MySQL is definitely too slow and not used as soon
>>>>> as you have a certain amount of data in your table.
>>>>>
>>>>> Regards,
>>>>>
>>>>>
>>>>>
>>>> --
>>>>
>>>> Patrick O'Leary
>>>>
>>>>
>>>> You see, wire telegraph is a kind of a very, very long cat. You pull his
>>>> tail in New York and his head is meowing in Los Angeles.
>>>> Do you understand this?
>>>> And radio operates exactly the same way: you send signals here, they
>>>> receive them there. The only difference is that there is no cat.
>>>> - Albert Einstein
>>>>
>>>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>>>> <http://www.linkedin.com/in/pjaol>
>>>>
>>>>
>>>>
>>>
>> --
>>
>> Patrick O'Leary
>>
>>
>> You see, wire telegraph is a kind of a very, very long cat. You pull his
>> tail in New York and his head is meowing in Los Angeles.
>> Do you understand this?
>> And radio operates exactly the same way: you send signals here, they
>> receive them there. The only difference is that there is no cat.
>> - Albert Einstein
>>
>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>> <http://www.linkedin.com/in/pjaol>
>>
>>
>
Re: Geographical distance searching
Posted by Doug Daniels <dd...@rooftophq.com>.
Hi Patrick,
Was mainly interested in seeing how you did the RequestHandler. Thanks for
sending the link!
Best,
Doug
patrick o'leary wrote:
>
> Hi Doug
>
> What exactly are you looking for?
> The code for localsolr is still in dev state, but I've left my work open
> and available for download
> at http://www.nsshutdown.com/viewcvs/viewcvs.cgi/localsolr/
>
> Once I'm happy with it, I'll donate it back in the form of patches until
> / unless it's accepted
> as a contribution, depending on how folks feel.
>
> If your talking about the demo ui, it's a little piece of html & JS, you
> can pull directly from the jar.
> I've not included that in the repository.
>
> HTH
> P
>
> Doug Daniels wrote:
>> Hi Patrick,
>>
>> Are the solr components of that demo in the repository as well? I
>> couldn't
>> find them there.
>>
>> Best,
>> Doug
>>
>>
>> patrick o'leary wrote:
>>
>>> As far as I'm concerned nothings going to beat PG's GIS calculations,
>>> but it's tsearch was
>>> a lot slower than myisam.
>>>
>>> My goal was a single solution to reduce our complexity, but am
>>> interested to know if combining
>>> both an rdbms & lucene works for you. Definitely let me know how it goes
>>> !
>>>
>>> P
>>>
>>> Guillaume Smet wrote:
>>>
>>>> Hi Patrick,
>>>>
>>>> On 9/27/07, patrick o'leary <po...@aol.com> wrote:
>>>>
>>>>
>>>>> p.s after a little tidy up I'll be adding this to both lucene and
>>>>> solr's repositories if folks feel that it's a useful addition.
>>>>>
>>>>>
>>>> It's definitely very interesting. Did you compare performances of
>>>> Lucene with a database allowing you to perform real GIS queries?
>>>> I'm more a PostgreSQL guy and I must admit we usually use cube contrib
>>>> or PostGIS for this sort of thing and with both, we are capable to use
>>>> indexes for proximity queries and they can be pretty fast. Using the
>>>> method you used with MySQL is definitely too slow and not used as soon
>>>> as you have a certain amount of data in your table.
>>>>
>>>> Regards,
>>>>
>>>>
>>>>
>>> --
>>>
>>> Patrick O'Leary
>>>
>>>
>>> You see, wire telegraph is a kind of a very, very long cat. You pull his
>>> tail in New York and his head is meowing in Los Angeles.
>>> Do you understand this?
>>> And radio operates exactly the same way: you send signals here, they
>>> receive them there. The only difference is that there is no cat.
>>> - Albert Einstein
>>>
>>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>>> <http://www.linkedin.com/in/pjaol>
>>>
>>>
>>>
>>
>>
>
> --
>
> Patrick O'Leary
>
>
> You see, wire telegraph is a kind of a very, very long cat. You pull his
> tail in New York and his head is meowing in Los Angeles.
> Do you understand this?
> And radio operates exactly the same way: you send signals here, they
> receive them there. The only difference is that there is no cat.
> - Albert Einstein
>
> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
> <http://www.linkedin.com/in/pjaol>
>
>
--
View this message in context: http://www.nabble.com/Geographical-distance-searching-tf4524338.html#a13296862
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Geographical distance searching
Posted by patrick o'leary <po...@aol.com>.
Hi Doug
What exactly are you looking for?
The code for localsolr is still in dev state, but I've left my work open
and available for download
at http://www.nsshutdown.com/viewcvs/viewcvs.cgi/localsolr/
Once I'm happy with it, I'll donate it back in the form of patches until
/ unless it's accepted
as a contribution, depending on how folks feel.
If your talking about the demo ui, it's a little piece of html & JS, you
can pull directly from the jar.
I've not included that in the repository.
HTH
P
Doug Daniels wrote:
> Hi Patrick,
>
> Are the solr components of that demo in the repository as well? I couldn't
> find them there.
>
> Best,
> Doug
>
>
> patrick o'leary wrote:
>
>> As far as I'm concerned nothings going to beat PG's GIS calculations,
>> but it's tsearch was
>> a lot slower than myisam.
>>
>> My goal was a single solution to reduce our complexity, but am
>> interested to know if combining
>> both an rdbms & lucene works for you. Definitely let me know how it goes !
>>
>> P
>>
>> Guillaume Smet wrote:
>>
>>> Hi Patrick,
>>>
>>> On 9/27/07, patrick o'leary <po...@aol.com> wrote:
>>>
>>>
>>>> p.s after a little tidy up I'll be adding this to both lucene and
>>>> solr's repositories if folks feel that it's a useful addition.
>>>>
>>>>
>>> It's definitely very interesting. Did you compare performances of
>>> Lucene with a database allowing you to perform real GIS queries?
>>> I'm more a PostgreSQL guy and I must admit we usually use cube contrib
>>> or PostGIS for this sort of thing and with both, we are capable to use
>>> indexes for proximity queries and they can be pretty fast. Using the
>>> method you used with MySQL is definitely too slow and not used as soon
>>> as you have a certain amount of data in your table.
>>>
>>> Regards,
>>>
>>>
>>>
>> --
>>
>> Patrick O'Leary
>>
>>
>> You see, wire telegraph is a kind of a very, very long cat. You pull his
>> tail in New York and his head is meowing in Los Angeles.
>> Do you understand this?
>> And radio operates exactly the same way: you send signals here, they
>> receive them there. The only difference is that there is no cat.
>> - Albert Einstein
>>
>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>> <http://www.linkedin.com/in/pjaol>
>>
>>
>>
>
>
--
Patrick O'Leary
You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
Do you understand this?
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
- Albert Einstein
View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
<http://www.linkedin.com/in/pjaol>