You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Brian Lamb <br...@journalexperts.com> on 2011/05/19 20:45:19 UTC
Similarity class for an individual field
Hi all,
Based on advice I received on a previous email thread, I applied patch
https://issues.apache.org/jira/browse/SOLR-2338. My goal was to be able to
apply a similarity class to certain fields but not all fields.
I ran the following commands:
$ cd <your Solr trunk checkout dir>
$ svn up
$ wget https://issues.apache.org/jira/secure/attachment/12475027/SOLR-2338.patch
$ patch -p0 -i SOLR-2338.patch
And I did not get any errors. I then created my own SimilarityClass
listed below because it isn't very large:
package org.apache.lucene.misc;
import org.apache.lucene.search.DefaultSimilarity;
public class SimpleSimilarity extends DefaultSimilarity {
public SimpleSimilarity() { super(); }
public float idf(int dont, int care) { return 1; }
}
As you can see, it isn't very complicated. I'm just trying to remove
the idf from the scoring equation in certain cases.
Next, I make a change to the schema.xml file:
<fieldType name="string_noidf" class="solr.StrField"
sortMissingLast="true" omitNorms="true">
<similarity class="org.apache.lucene.misc.SimpleSimilarity"/>
</fieldType>
And apply that to the field in question:
<field name="string_noidf" multiValued="true" type="string_noidf"
indexed="true" stored="true" required="false" omitNorms="true" />
But I think something did not get applied correctly to the patch. I
restarted and did a full import but the scores are exactly the same.
Also, I tried using the existing SweetSpotSimilarity:
<fieldType name="string_noidf" class="solr.StrField"
sortMissingLast="true" omitNorms="true">
<similarity class="org.apache.lucene.misc.SweetSpotSimilarity"/>
</fieldType>
But the scores remained unchanged even in that case. At this point,
I'm not quite sure how to debug this to see whether the problem is
with the patch or the similarity class but given that the SweetSpot
similarity class didn't work either, I'm inclined to think it was a
problem with the patch.
Any thoughts on this one?
Thanks,
Brian Lamb
Re: Similarity class for an individual field
Posted by Brian Lamb <br...@journalexperts.com>.
I tried editing the SweetSpotSimilarity class located at
lucene/contrib/misc/src/java/org/apache/lucene/misc/SweetSpotSimilarity.java
to just return 1 for each function and the score does not change at all.
This has led me to believe that it does not recognize similarity at all. At
this point, all I have for similarity is the line at the end of the file to
apply similarity to all searches but that does not even work. So where am I
going wrong?
Thanks,
Brian Lamb
On Thu, May 19, 2011 at 3:41 PM, Brian Lamb
<br...@journalexperts.com>wrote:
> Also, I've tried adding:
>
> <similarity class="org.apache.lucene.misc.SweetSpotSimilarity"/>
>
> To the end of the schema file so that it is applied globally but it does
> not appear to change the score either. What am I doing incorrectly?
>
> Thanks,
>
> Brian Lamb
>
> On Thu, May 19, 2011 at 2:45 PM, Brian Lamb <brian.lamb@journalexperts.com
> > wrote:
>
>> Hi all,
>>
>> Based on advice I received on a previous email thread, I applied patch
>> https://issues.apache.org/jira/browse/SOLR-2338. My goal was to be able
>> to apply a similarity class to certain fields but not all fields.
>>
>> I ran the following commands:
>>
>> $ cd <your Solr trunk checkout dir>
>> $ svn up
>> $ wget https://issues.apache.org/jira/secure/attachment/12475027/SOLR-2338.patch
>> $ patch -p0 -i SOLR-2338.patch
>>
>> And I did not get any errors. I then created my own SimilarityClass listed below because it isn't very large:
>>
>> package org.apache.lucene.misc;
>> import org.apache.lucene.search.DefaultSimilarity;
>>
>> public class SimpleSimilarity extends DefaultSimilarity {
>> public SimpleSimilarity() { super(); }
>>
>>
>> public float idf(int dont, int care) { return 1; }
>> }
>>
>> As you can see, it isn't very complicated. I'm just trying to remove the idf from the scoring equation in certain cases.
>>
>>
>> Next, I make a change to the schema.xml file:
>>
>> <fieldType name="string_noidf" class="solr.StrField" sortMissingLast="true" omitNorms="true">
>>
>>
>> <similarity class="org.apache.lucene.misc.SimpleSimilarity"/>
>> </fieldType>
>>
>> And apply that to the field in question:
>>
>> <field name="string_noidf" multiValued="true" type="string_noidf" indexed="true" stored="true" required="false" omitNorms="true" />
>>
>>
>> But I think something did not get applied correctly to the patch. I restarted and did a full import but the scores are exactly the same. Also, I tried using the existing SweetSpotSimilarity:
>> <fieldType name="string_noidf" class="solr.StrField" sortMissingLast="true" omitNorms="true">
>> <similarity class="org.apache.lucene.misc.SweetSpotSimilarity"/>
>>
>>
>> </fieldType>
>>
>> But the scores remained unchanged even in that case. At this point, I'm not quite sure how to debug this to see whether the problem is with the patch or the similarity class but given that the SweetSpot similarity class didn't work either, I'm inclined to think it was a problem with the patch.
>>
>>
>> Any thoughts on this one?
>>
>> Thanks,
>>
>> Brian Lamb
>>
>>
>>
>
Re: Similarity class for an individual field
Posted by Brian Lamb <br...@journalexperts.com>.
Also, I've tried adding:
<similarity class="org.apache.lucene.misc.SweetSpotSimilarity"/>
To the end of the schema file so that it is applied globally but it does not
appear to change the score either. What am I doing incorrectly?
Thanks,
Brian Lamb
On Thu, May 19, 2011 at 2:45 PM, Brian Lamb
<br...@journalexperts.com>wrote:
> Hi all,
>
> Based on advice I received on a previous email thread, I applied patch
> https://issues.apache.org/jira/browse/SOLR-2338. My goal was to be able to
> apply a similarity class to certain fields but not all fields.
>
> I ran the following commands:
>
> $ cd <your Solr trunk checkout dir>
> $ svn up
> $ wget https://issues.apache.org/jira/secure/attachment/12475027/SOLR-2338.patch
> $ patch -p0 -i SOLR-2338.patch
>
> And I did not get any errors. I then created my own SimilarityClass listed below because it isn't very large:
>
> package org.apache.lucene.misc;
> import org.apache.lucene.search.DefaultSimilarity;
>
> public class SimpleSimilarity extends DefaultSimilarity {
> public SimpleSimilarity() { super(); }
>
> public float idf(int dont, int care) { return 1; }
> }
>
> As you can see, it isn't very complicated. I'm just trying to remove the idf from the scoring equation in certain cases.
>
> Next, I make a change to the schema.xml file:
>
> <fieldType name="string_noidf" class="solr.StrField" sortMissingLast="true" omitNorms="true">
>
> <similarity class="org.apache.lucene.misc.SimpleSimilarity"/>
> </fieldType>
>
> And apply that to the field in question:
>
> <field name="string_noidf" multiValued="true" type="string_noidf" indexed="true" stored="true" required="false" omitNorms="true" />
>
> But I think something did not get applied correctly to the patch. I restarted and did a full import but the scores are exactly the same. Also, I tried using the existing SweetSpotSimilarity:
> <fieldType name="string_noidf" class="solr.StrField" sortMissingLast="true" omitNorms="true">
> <similarity class="org.apache.lucene.misc.SweetSpotSimilarity"/>
>
> </fieldType>
>
> But the scores remained unchanged even in that case. At this point, I'm not quite sure how to debug this to see whether the problem is with the patch or the similarity class but given that the SweetSpot similarity class didn't work either, I'm inclined to think it was a problem with the patch.
>
> Any thoughts on this one?
>
> Thanks,
>
> Brian Lamb
>
>
>
Re: Similarity class for an individual field
Posted by Brian Lamb <br...@journalexperts.com>.
So what was my mistake? I still have not resolved this issue.
On Fri, May 20, 2011 at 11:22 AM, Brian Lamb
<br...@journalexperts.com>wrote:
> Yes. Was that not what I was supposed to do?
>
>
> On Thu, May 19, 2011 at 8:26 PM, Koji Sekiguchi <ko...@r.email.ne.jp>wrote:
>
>> (11/05/20 3:45), Brian Lamb wrote:
>>
>>> Hi all,
>>>
>>> Based on advice I received on a previous email thread, I applied patch
>>> https://issues.apache.org/jira/browse/SOLR-2338. My goal was to be able
>>> to
>>> apply a similarity class to certain fields but not all fields.
>>>
>>> I ran the following commands:
>>>
>>> $ cd<your Solr trunk checkout dir>
>>> $ svn up
>>> $ wget
>>> https://issues.apache.org/jira/secure/attachment/12475027/SOLR-2338.patch
>>> $ patch -p0 -i SOLR-2338.patch
>>>
>>> And I did not get any errors. I then created my own SimilarityClass
>>>
>>
>> Brian,
>>
>> I'm confused what you did because SOLR-2338 has been resolved in March and
>> committed
>> in trunk, but you did svn up & apply patch in your trunk?
>>
>> Koji
>> --
>> http://www.rondhuit.com/en/
>>
>
>
Re: Similarity class for an individual field
Posted by Brian Lamb <br...@journalexperts.com>.
Yes. Was that not what I was supposed to do?
On Thu, May 19, 2011 at 8:26 PM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:
> (11/05/20 3:45), Brian Lamb wrote:
>
>> Hi all,
>>
>> Based on advice I received on a previous email thread, I applied patch
>> https://issues.apache.org/jira/browse/SOLR-2338. My goal was to be able
>> to
>> apply a similarity class to certain fields but not all fields.
>>
>> I ran the following commands:
>>
>> $ cd<your Solr trunk checkout dir>
>> $ svn up
>> $ wget
>> https://issues.apache.org/jira/secure/attachment/12475027/SOLR-2338.patch
>> $ patch -p0 -i SOLR-2338.patch
>>
>> And I did not get any errors. I then created my own SimilarityClass
>>
>
> Brian,
>
> I'm confused what you did because SOLR-2338 has been resolved in March and
> committed
> in trunk, but you did svn up & apply patch in your trunk?
>
> Koji
> --
> http://www.rondhuit.com/en/
>
Re: Similarity class for an individual field
Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
(11/05/20 3:45), Brian Lamb wrote:
> Hi all,
>
> Based on advice I received on a previous email thread, I applied patch
> https://issues.apache.org/jira/browse/SOLR-2338. My goal was to be able to
> apply a similarity class to certain fields but not all fields.
>
> I ran the following commands:
>
> $ cd<your Solr trunk checkout dir>
> $ svn up
> $ wget https://issues.apache.org/jira/secure/attachment/12475027/SOLR-2338.patch
> $ patch -p0 -i SOLR-2338.patch
>
> And I did not get any errors. I then created my own SimilarityClass
Brian,
I'm confused what you did because SOLR-2338 has been resolved in March and committed
in trunk, but you did svn up & apply patch in your trunk?
Koji
--
http://www.rondhuit.com/en/