You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Christian Bongiorno <ch...@bongiorno.org> on 2009/08/12 21:46:39 UTC

term query boost problem

I have a situation where I have a series of terms queries as part of a
BooleanQuery.

example:

term: 'sole type' -> leather
  BooleanClause.SHOULD_OCCURR
term: 'title' -> 'Men's Golf shoes'
  BooleanClause.SHOULD_OCCURR
...

But, some terms are incredibly powerful indicators of match
term: 'band type' -> 'gold'

Normally, I would set this as a field boost in the query. Problem is, it
isn't producing ANY changes even when I set the boost score absurdly high. I
have also tried to set the BooleanClause.MUST on my really important term
and I get no results at all.

I tried a simple TermQuery for that (no clause) and got no results. Below is
a code snippet. I checked through the recent archives discussing field
boosts and I am pretty confident I am doing it right. So, now I am presuming
it's a problem with my query.

    private Query buildQuery(Map<String, String> CatalogInfo) {
        if (CatalogInfo != null && CatalogInfo.size() > 0) {
            BooleanQuery booleanQuery = new BooleanQuery();
            for (Map.Entry<String, String> attributeValue :
CatalogInfo.entrySet()) {
                String attributeName = attributeValue.getKey();
                String[] attributeValues;
                if (attributeValue.getValue().indexOf(VALUES_DELIMITER) ==
-1) {
                    attributeValues = new String[]
{attributeValue.getValue()};
                } else {
                    attributeValues =
attributeValue.getValue().split(VALUES_DELIMITER);
                }
                for (String attributeValue : attributeValues) {
                    String escapedValue =
QueryParser.escape(attributeValue).trim();
                    TermQuery termQuery = new TermQuery(new
Term(attributeName, escapedValue));


                    Float boostNumber = _boostMap.get(attributeName); //
this is where 'band type' gets it's boost
                    if (boostNumber != null) {
                        LOG.warn("Boost value found: " + boostNumber);
                        termQuery.setBoost(boostNumber);
                    }
                    booleanQuery.add(termQuery, BooleanClause.Occur.SHOULD);
                }
            }
            LOG.warn("Boolean query: " + booleanQuery.toString());
            return booleanQuery;
        }
        return null;
    }
-- 
Christian Bongiorno

Re: term query boost problem

Posted by Simon Willnauer <si...@googlemail.com>.
Chrisitan,
if you haven't done so you might find Luke
(http://www.getopt.org/luke/) very helpful so see what has been
indexed and how.

simon

On Thu, Aug 13, 2009 at 6:10 AM, Christian
Bongiorno<ch...@bongiorno.org> wrote:
> turns out the index is being built with lower-case terms which is why we
> aren't getting hits the way we expect. When I change my search terms to
> lower I see more of what I expect.
>
> Gonna keep working on this and post updates.
>
> On Wed, Aug 12, 2009 at 12:46 PM, Christian Bongiorno <
> christian@bongiorno.org> wrote:
>
>> I have a situation where I have a series of terms queries as part of a
>> BooleanQuery.
>>
>> example:
>>
>> term: 'sole type' -> leather
>>   BooleanClause.SHOULD_OCCURR
>> term: 'title' -> 'Men's Golf shoes'
>>   BooleanClause.SHOULD_OCCURR
>> ...
>>
>> But, some terms are incredibly powerful indicators of match
>> term: 'band type' -> 'gold'
>>
>> Normally, I would set this as a field boost in the query. Problem is, it
>> isn't producing ANY changes even when I set the boost score absurdly high. I
>> have also tried to set the BooleanClause.MUST on my really important term
>> and I get no results at all.
>>
>> I tried a simple TermQuery for that (no clause) and got no results. Below
>> is a code snippet. I checked through the recent archives discussing field
>> boosts and I am pretty confident I am doing it right. So, now I am presuming
>> it's a problem with my query.
>>
>>     private Query buildQuery(Map<String, String> CatalogInfo) {
>>         if (CatalogInfo != null && CatalogInfo.size() > 0) {
>>             BooleanQuery booleanQuery = new BooleanQuery();
>>             for (Map.Entry<String, String> attributeValue :
>> CatalogInfo.entrySet()) {
>>                 String attributeName = attributeValue.getKey();
>>                 String[] attributeValues;
>>                 if (attributeValue.getValue().indexOf(VALUES_DELIMITER) ==
>> -1) {
>>                     attributeValues = new String[]
>> {attributeValue.getValue()};
>>                 } else {
>>                     attributeValues =
>> attributeValue.getValue().split(VALUES_DELIMITER);
>>                 }
>>                 for (String attributeValue : attributeValues) {
>>                     String escapedValue =
>> QueryParser.escape(attributeValue).trim();
>>                     TermQuery termQuery = new TermQuery(new
>> Term(attributeName, escapedValue));
>>
>>
>>                     Float boostNumber = _boostMap.get(attributeName); //
>> this is where 'band type' gets it's boost
>>                     if (boostNumber != null) {
>>                         LOG.warn("Boost value found: " + boostNumber);
>>                         termQuery.setBoost(boostNumber);
>>                     }
>>                     booleanQuery.add(termQuery,
>> BooleanClause.Occur.SHOULD);
>>                 }
>>             }
>>             LOG.warn("Boolean query: " + booleanQuery.toString());
>>             return booleanQuery;
>>         }
>>         return null;
>>     }
>> --
>> Christian Bongiorno
>>
>>
>
>
> --
> Christian Bongiorno
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: term query boost problem

Posted by Christian Bongiorno <ch...@bongiorno.org>.
turns out the index is being built with lower-case terms which is why we
aren't getting hits the way we expect. When I change my search terms to
lower I see more of what I expect.

Gonna keep working on this and post updates.

On Wed, Aug 12, 2009 at 12:46 PM, Christian Bongiorno <
christian@bongiorno.org> wrote:

> I have a situation where I have a series of terms queries as part of a
> BooleanQuery.
>
> example:
>
> term: 'sole type' -> leather
>   BooleanClause.SHOULD_OCCURR
> term: 'title' -> 'Men's Golf shoes'
>   BooleanClause.SHOULD_OCCURR
> ...
>
> But, some terms are incredibly powerful indicators of match
> term: 'band type' -> 'gold'
>
> Normally, I would set this as a field boost in the query. Problem is, it
> isn't producing ANY changes even when I set the boost score absurdly high. I
> have also tried to set the BooleanClause.MUST on my really important term
> and I get no results at all.
>
> I tried a simple TermQuery for that (no clause) and got no results. Below
> is a code snippet. I checked through the recent archives discussing field
> boosts and I am pretty confident I am doing it right. So, now I am presuming
> it's a problem with my query.
>
>     private Query buildQuery(Map<String, String> CatalogInfo) {
>         if (CatalogInfo != null && CatalogInfo.size() > 0) {
>             BooleanQuery booleanQuery = new BooleanQuery();
>             for (Map.Entry<String, String> attributeValue :
> CatalogInfo.entrySet()) {
>                 String attributeName = attributeValue.getKey();
>                 String[] attributeValues;
>                 if (attributeValue.getValue().indexOf(VALUES_DELIMITER) ==
> -1) {
>                     attributeValues = new String[]
> {attributeValue.getValue()};
>                 } else {
>                     attributeValues =
> attributeValue.getValue().split(VALUES_DELIMITER);
>                 }
>                 for (String attributeValue : attributeValues) {
>                     String escapedValue =
> QueryParser.escape(attributeValue).trim();
>                     TermQuery termQuery = new TermQuery(new
> Term(attributeName, escapedValue));
>
>
>                     Float boostNumber = _boostMap.get(attributeName); //
> this is where 'band type' gets it's boost
>                     if (boostNumber != null) {
>                         LOG.warn("Boost value found: " + boostNumber);
>                         termQuery.setBoost(boostNumber);
>                     }
>                     booleanQuery.add(termQuery,
> BooleanClause.Occur.SHOULD);
>                 }
>             }
>             LOG.warn("Boolean query: " + booleanQuery.toString());
>             return booleanQuery;
>         }
>         return null;
>     }
> --
> Christian Bongiorno
>
>


-- 
Christian Bongiorno

Re: term query boost problem

Posted by Grant Ingersoll <gs...@apache.org>.
You have a bunch of log statements in there, what are they printing out?

Also, IndexSearcher.explain() is your friend for understanding why a  
doc matched the way it did.


On Aug 12, 2009, at 3:46 PM, Christian Bongiorno wrote:

> I have a situation where I have a series of terms queries as part of a
> BooleanQuery.
>
> example:
>
> term: 'sole type' -> leather
>  BooleanClause.SHOULD_OCCURR
> term: 'title' -> 'Men's Golf shoes'
>  BooleanClause.SHOULD_OCCURR
> ...
>
> But, some terms are incredibly powerful indicators of match
> term: 'band type' -> 'gold'
>
> Normally, I would set this as a field boost in the query. Problem  
> is, it
> isn't producing ANY changes even when I set the boost score absurdly  
> high. I
> have also tried to set the BooleanClause.MUST on my really important  
> term
> and I get no results at all.
>
> I tried a simple TermQuery for that (no clause) and got no results.  
> Below is
> a code snippet. I checked through the recent archives discussing field
> boosts and I am pretty confident I am doing it right. So, now I am  
> presuming
> it's a problem with my query.
>
>    private Query buildQuery(Map<String, String> CatalogInfo) {
>        if (CatalogInfo != null && CatalogInfo.size() > 0) {
>            BooleanQuery booleanQuery = new BooleanQuery();
>            for (Map.Entry<String, String> attributeValue :
> CatalogInfo.entrySet()) {
>                String attributeName = attributeValue.getKey();
>                String[] attributeValues;
>                if  
> (attributeValue.getValue().indexOf(VALUES_DELIMITER) ==
> -1) {
>                    attributeValues = new String[]
> {attributeValue.getValue()};
>                } else {
>                    attributeValues =
> attributeValue.getValue().split(VALUES_DELIMITER);
>                }
>                for (String attributeValue : attributeValues) {
>                    String escapedValue =
> QueryParser.escape(attributeValue).trim();
>                    TermQuery termQuery = new TermQuery(new
> Term(attributeName, escapedValue));
>
>
>                    Float boostNumber =  
> _boostMap.get(attributeName); //
> this is where 'band type' gets it's boost
>                    if (boostNumber != null) {
>                        LOG.warn("Boost value found: " + boostNumber);
>                        termQuery.setBoost(boostNumber);
>                    }
>                    booleanQuery.add(termQuery,  
> BooleanClause.Occur.SHOULD);
>                }
>            }
>            LOG.warn("Boolean query: " + booleanQuery.toString());
>            return booleanQuery;
>        }
>        return null;
>    }
> -- 
> Christian Bongiorno

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:
http://www.lucidimagination.com/search


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org