You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by mm...@preselect-media.com on 2014/04/03 11:42:42 UTC
Solr join and lucene scoring
Hello,
referencing to this issue:
https://issues.apache.org/jira/browse/SOLR-4307
Is it still not possible with the solr query time join to use scoring?
Do I still have to write my own plugin or is there a plugin somewhere
I could use?
I never wrote a plugin for solr before, so I would prefer if I don't
have to start from scratch.
THX,
Moritz
Re: Solr join and lucene scoring
Posted by Alvaro Cabrerizo <to...@gmail.com>.
Right,
The examples posted before had a trick (both weren't pure join queries
thats why the results had score).
Having a pure join query (q={!join from=id to=id}printer) gives all the
scores as 1 as Moritz said).
Regards.
On Wed, Apr 9, 2014 at 3:13 PM, Mikhail Khludnev <mkhludnev@griddynamics.com
> wrote:
> On Wed, Apr 9, 2014 at 1:33 PM, <mm...@preselect-media.com> wrote:
>
> > Hello Mikhail,
> >
> > thx for the clarification. I'm a little bit confused by the answer of
> > Alvaro, but my own tests didn't result in a proper score, so I think
> you're
> > right and it's still not implemented.
> >
> > What do you mean with the "impedance between Lucene and Solr"?
> >
> It's an old story, and unfortunately obvious. Using Lucene's code in Solr
> might not be straightforward. I haven't looked at this problem
> particularly, it's just a caveat.
>
> >
> > Why isn't the possibility of scoring in joins not implemented in Solr
> > anyways when Lucene offers a solution for that?
> >
> As you can see these are two separate implementation. It seems like Solr
> guys just didn't care about scoring (and here I share their point). It's
> just an exercise for someone, who needs it.
>
>
> >
> > Best regards,
> > Moritz
> >
> > Zitat von Mikhail Khludnev <mk...@griddynamics.com>:
> >
> > On Thu, Apr 3, 2014 at 1:42 PM, <mm...@preselect-media.com> wrote:
> >>
> >> Hello,
> >>>
> >>> referencing to this issue:
> >>> https://issues.apache.org/jira/browse/SOLR-4307
> >>>
> >>> Is it still not possible with the solr query time join to use scoring?
> >>>
> >>> It's not implemented still.
> >> https://github.com/apache/lucene-solr/blob/trunk/solr/
> >> core/src/java/org/apache/solr/search/JoinQParserPlugin.java#L549
> >>
> >>
> >> Do I still have to write my own plugin or is there a plugin somewhere I
> >>> could use?
> >>>
> >>> I never wrote a plugin for solr before, so I would prefer if I don't
> have
> >>> to start from scratch.
> >>>
> >>> The right approach from my POV is to use Lucene's join
> >> https://github.com/apache/lucene-solr/blob/trunk/lucene/
> >> join/src/java/org/apache/lucene/search/join/JoinUtil.javain
> >> new QParser, but solving the impedance between Lucene and Solr, might
> >> be
> >> tricky.
> >>
> >>
> >>
> >>
> >>> THX,
> >>> Moritz
> >>>
> >>>
> >>>
> >>
> >> --
> >> Sincerely yours
> >> Mikhail Khludnev
> >> Principal Engineer,
> >> Grid Dynamics
> >>
> >> <http://www.griddynamics.com>
> >> <mk...@griddynamics.com>
> >>
> >
> >
> >
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
> <mk...@griddynamics.com>
>
Re: Solr join and lucene scoring
Posted by mm...@preselect-media.com.
Thank you for the clarification.
We really need scoring with solr joins, but as you can see I'm not a
specialist in solr development.
We would like to hire somebody with more experience to write a qparser
plugin for scoring in joins and donate the source code to the community.
Any suggestions where we could find somebody with the fitting experience?
Zitat von Mikhail Khludnev <mk...@griddynamics.com>:
> On Wed, Apr 9, 2014 at 1:33 PM, <mm...@preselect-media.com> wrote:
>
>> Hello Mikhail,
>>
>> thx for the clarification. I'm a little bit confused by the answer of
>> Alvaro, but my own tests didn't result in a proper score, so I think you're
>> right and it's still not implemented.
>>
>> What do you mean with the "impedance between Lucene and Solr"?
>>
> It's an old story, and unfortunately obvious. Using Lucene's code in Solr
> might not be straightforward. I haven't looked at this problem
> particularly, it's just a caveat.
>
>>
>> Why isn't the possibility of scoring in joins not implemented in Solr
>> anyways when Lucene offers a solution for that?
>>
> As you can see these are two separate implementation. It seems like Solr
> guys just didn't care about scoring (and here I share their point). It's
> just an exercise for someone, who needs it.
>
>
>>
>> Best regards,
>> Moritz
>>
>> Zitat von Mikhail Khludnev <mk...@griddynamics.com>:
>>
>> On Thu, Apr 3, 2014 at 1:42 PM, <mm...@preselect-media.com> wrote:
>>>
>>> Hello,
>>>>
>>>> referencing to this issue:
>>>> https://issues.apache.org/jira/browse/SOLR-4307
>>>>
>>>> Is it still not possible with the solr query time join to use scoring?
>>>>
>>>> It's not implemented still.
>>> https://github.com/apache/lucene-solr/blob/trunk/solr/
>>> core/src/java/org/apache/solr/search/JoinQParserPlugin.java#L549
>>>
>>>
>>> Do I still have to write my own plugin or is there a plugin somewhere I
>>>> could use?
>>>>
>>>> I never wrote a plugin for solr before, so I would prefer if I don't have
>>>> to start from scratch.
>>>>
>>>> The right approach from my POV is to use Lucene's join
>>> https://github.com/apache/lucene-solr/blob/trunk/lucene/
>>> join/src/java/org/apache/lucene/search/join/JoinUtil.javain
>>> new QParser, but solving the impedance between Lucene and Solr, might
>>> be
>>> tricky.
>>>
>>>
>>>
>>>
>>>> THX,
>>>> Moritz
>>>>
>>>>
>>>>
>>>
>>> --
>>> Sincerely yours
>>> Mikhail Khludnev
>>> Principal Engineer,
>>> Grid Dynamics
>>>
>>> <http://www.griddynamics.com>
>>> <mk...@griddynamics.com>
>>>
>>
>>
>>
>>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
> <mk...@griddynamics.com>
Re: Solr join and lucene scoring
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
On Wed, Apr 9, 2014 at 1:33 PM, <mm...@preselect-media.com> wrote:
> Hello Mikhail,
>
> thx for the clarification. I'm a little bit confused by the answer of
> Alvaro, but my own tests didn't result in a proper score, so I think you're
> right and it's still not implemented.
>
> What do you mean with the "impedance between Lucene and Solr"?
>
It's an old story, and unfortunately obvious. Using Lucene's code in Solr
might not be straightforward. I haven't looked at this problem
particularly, it's just a caveat.
>
> Why isn't the possibility of scoring in joins not implemented in Solr
> anyways when Lucene offers a solution for that?
>
As you can see these are two separate implementation. It seems like Solr
guys just didn't care about scoring (and here I share their point). It's
just an exercise for someone, who needs it.
>
> Best regards,
> Moritz
>
> Zitat von Mikhail Khludnev <mk...@griddynamics.com>:
>
> On Thu, Apr 3, 2014 at 1:42 PM, <mm...@preselect-media.com> wrote:
>>
>> Hello,
>>>
>>> referencing to this issue:
>>> https://issues.apache.org/jira/browse/SOLR-4307
>>>
>>> Is it still not possible with the solr query time join to use scoring?
>>>
>>> It's not implemented still.
>> https://github.com/apache/lucene-solr/blob/trunk/solr/
>> core/src/java/org/apache/solr/search/JoinQParserPlugin.java#L549
>>
>>
>> Do I still have to write my own plugin or is there a plugin somewhere I
>>> could use?
>>>
>>> I never wrote a plugin for solr before, so I would prefer if I don't have
>>> to start from scratch.
>>>
>>> The right approach from my POV is to use Lucene's join
>> https://github.com/apache/lucene-solr/blob/trunk/lucene/
>> join/src/java/org/apache/lucene/search/join/JoinUtil.javain
>> new QParser, but solving the impedance between Lucene and Solr, might
>> be
>> tricky.
>>
>>
>>
>>
>>> THX,
>>> Moritz
>>>
>>>
>>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>> Principal Engineer,
>> Grid Dynamics
>>
>> <http://www.griddynamics.com>
>> <mk...@griddynamics.com>
>>
>
>
>
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: Solr join and lucene scoring
Posted by mm...@preselect-media.com.
Hello Mikhail,
thx for the clarification. I'm a little bit confused by the answer of
Alvaro, but my own tests didn't result in a proper score, so I think
you're right and it's still not implemented.
What do you mean with the "impedance between Lucene and Solr"?
Why isn't the possibility of scoring in joins not implemented in Solr
anyways when Lucene offers a solution for that?
Best regards,
Moritz
Zitat von Mikhail Khludnev <mk...@griddynamics.com>:
> On Thu, Apr 3, 2014 at 1:42 PM, <mm...@preselect-media.com> wrote:
>
>> Hello,
>>
>> referencing to this issue:
>> https://issues.apache.org/jira/browse/SOLR-4307
>>
>> Is it still not possible with the solr query time join to use scoring?
>>
> It's not implemented still.
> https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/search/JoinQParserPlugin.java#L549
>
>
>> Do I still have to write my own plugin or is there a plugin somewhere I
>> could use?
>>
>> I never wrote a plugin for solr before, so I would prefer if I don't have
>> to start from scratch.
>>
> The right approach from my POV is to use Lucene's join
> https://github.com/apache/lucene-solr/blob/trunk/lucene/join/src/java/org/apache/lucene/search/join/JoinUtil.javain
> new QParser, but solving the impedance between Lucene and Solr, might
> be
> tricky.
>
>
>
>>
>> THX,
>> Moritz
>>
>>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
> <mk...@griddynamics.com>
Re: Solr join and lucene scoring
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
On Thu, Apr 3, 2014 at 1:42 PM, <mm...@preselect-media.com> wrote:
> Hello,
>
> referencing to this issue:
> https://issues.apache.org/jira/browse/SOLR-4307
>
> Is it still not possible with the solr query time join to use scoring?
>
It's not implemented still.
https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/search/JoinQParserPlugin.java#L549
> Do I still have to write my own plugin or is there a plugin somewhere I
> could use?
>
> I never wrote a plugin for solr before, so I would prefer if I don't have
> to start from scratch.
>
The right approach from my POV is to use Lucene's join
https://github.com/apache/lucene-solr/blob/trunk/lucene/join/src/java/org/apache/lucene/search/join/JoinUtil.javain
new QParser, but solving the impedance between Lucene and Solr, might
be
tricky.
>
> THX,
> Moritz
>
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: Solr join and lucene scoring
Posted by Alvaro Cabrerizo <to...@gmail.com>.
Hi,
The defect you are referencing is closed with a resolution of *Invalid*, so
it seems the scoring is working fine with the join. I've made the next two
tests on my own data and seems it is working:
*TestA*
- fl=id,score
- q=notebook
- fq={!join from=product_list to=id fromIndex=product}id:*
- rows=2
Gives me the next result with the score calculated:
<doc>
<str name="id">4ADCBA5F-B532-4154-8E12-47311DC0FD50</str>
<float name="score">*2.6598556*</float>
</doc>
<doc>
<str name="id">C861CC4A-6481-4754-946F-EA3903371C80</str>
<float name="score">*2.6598551*</float>
</doc>
</result>
*TESTB*
- fl=id,score
- q=notebook AND _query_:{!join from=product_list to=id
fromIndex=product}id:*
- rows=2
Gives me the next result with the score calcualted:
<doc>
<str name="id">5C449525-8A69-409B-829C-671E147BF6BB</str>
<float name="score">*0.1573925*</float>
</doc>
<doc>
<str name="id">D1A719E8-F843-4E8D-AD82-64AA88D78BBB</str>
<float name="score">*0.1571764*</float>
</doc>
Regards.
On Thu, Apr 3, 2014 at 11:42 AM, <mm...@preselect-media.com> wrote:
> Hello,
>
> referencing to this issue:
> https://issues.apache.org/jira/browse/SOLR-4307
>
> Is it still not possible with the solr query time join to use scoring?
> Do I still have to write my own plugin or is there a plugin somewhere I
> could use?
>
> I never wrote a plugin for solr before, so I would prefer if I don't have
> to start from scratch.
>
> THX,
> Moritz
>
>