You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Christian Bongiorno <ch...@bongiorno.org> on 2009/08/12 21:46:39 UTC
term query boost problem
I have a situation where I have a series of terms queries as part of a
BooleanQuery.
example:
term: 'sole type' -> leather
BooleanClause.SHOULD_OCCURR
term: 'title' -> 'Men's Golf shoes'
BooleanClause.SHOULD_OCCURR
...
But, some terms are incredibly powerful indicators of match
term: 'band type' -> 'gold'
Normally, I would set this as a field boost in the query. Problem is, it
isn't producing ANY changes even when I set the boost score absurdly high. I
have also tried to set the BooleanClause.MUST on my really important term
and I get no results at all.
I tried a simple TermQuery for that (no clause) and got no results. Below is
a code snippet. I checked through the recent archives discussing field
boosts and I am pretty confident I am doing it right. So, now I am presuming
it's a problem with my query.
private Query buildQuery(Map<String, String> CatalogInfo) {
if (CatalogInfo != null && CatalogInfo.size() > 0) {
BooleanQuery booleanQuery = new BooleanQuery();
for (Map.Entry<String, String> attributeValue :
CatalogInfo.entrySet()) {
String attributeName = attributeValue.getKey();
String[] attributeValues;
if (attributeValue.getValue().indexOf(VALUES_DELIMITER) ==
-1) {
attributeValues = new String[]
{attributeValue.getValue()};
} else {
attributeValues =
attributeValue.getValue().split(VALUES_DELIMITER);
}
for (String attributeValue : attributeValues) {
String escapedValue =
QueryParser.escape(attributeValue).trim();
TermQuery termQuery = new TermQuery(new
Term(attributeName, escapedValue));
Float boostNumber = _boostMap.get(attributeName); //
this is where 'band type' gets it's boost
if (boostNumber != null) {
LOG.warn("Boost value found: " + boostNumber);
termQuery.setBoost(boostNumber);
}
booleanQuery.add(termQuery, BooleanClause.Occur.SHOULD);
}
}
LOG.warn("Boolean query: " + booleanQuery.toString());
return booleanQuery;
}
return null;
}
--
Christian Bongiorno
Re: term query boost problem
Posted by Simon Willnauer <si...@googlemail.com>.
Chrisitan,
if you haven't done so you might find Luke
(http://www.getopt.org/luke/) very helpful so see what has been
indexed and how.
simon
On Thu, Aug 13, 2009 at 6:10 AM, Christian
Bongiorno<ch...@bongiorno.org> wrote:
> turns out the index is being built with lower-case terms which is why we
> aren't getting hits the way we expect. When I change my search terms to
> lower I see more of what I expect.
>
> Gonna keep working on this and post updates.
>
> On Wed, Aug 12, 2009 at 12:46 PM, Christian Bongiorno <
> christian@bongiorno.org> wrote:
>
>> I have a situation where I have a series of terms queries as part of a
>> BooleanQuery.
>>
>> example:
>>
>> term: 'sole type' -> leather
>> BooleanClause.SHOULD_OCCURR
>> term: 'title' -> 'Men's Golf shoes'
>> BooleanClause.SHOULD_OCCURR
>> ...
>>
>> But, some terms are incredibly powerful indicators of match
>> term: 'band type' -> 'gold'
>>
>> Normally, I would set this as a field boost in the query. Problem is, it
>> isn't producing ANY changes even when I set the boost score absurdly high. I
>> have also tried to set the BooleanClause.MUST on my really important term
>> and I get no results at all.
>>
>> I tried a simple TermQuery for that (no clause) and got no results. Below
>> is a code snippet. I checked through the recent archives discussing field
>> boosts and I am pretty confident I am doing it right. So, now I am presuming
>> it's a problem with my query.
>>
>> private Query buildQuery(Map<String, String> CatalogInfo) {
>> if (CatalogInfo != null && CatalogInfo.size() > 0) {
>> BooleanQuery booleanQuery = new BooleanQuery();
>> for (Map.Entry<String, String> attributeValue :
>> CatalogInfo.entrySet()) {
>> String attributeName = attributeValue.getKey();
>> String[] attributeValues;
>> if (attributeValue.getValue().indexOf(VALUES_DELIMITER) ==
>> -1) {
>> attributeValues = new String[]
>> {attributeValue.getValue()};
>> } else {
>> attributeValues =
>> attributeValue.getValue().split(VALUES_DELIMITER);
>> }
>> for (String attributeValue : attributeValues) {
>> String escapedValue =
>> QueryParser.escape(attributeValue).trim();
>> TermQuery termQuery = new TermQuery(new
>> Term(attributeName, escapedValue));
>>
>>
>> Float boostNumber = _boostMap.get(attributeName); //
>> this is where 'band type' gets it's boost
>> if (boostNumber != null) {
>> LOG.warn("Boost value found: " + boostNumber);
>> termQuery.setBoost(boostNumber);
>> }
>> booleanQuery.add(termQuery,
>> BooleanClause.Occur.SHOULD);
>> }
>> }
>> LOG.warn("Boolean query: " + booleanQuery.toString());
>> return booleanQuery;
>> }
>> return null;
>> }
>> --
>> Christian Bongiorno
>>
>>
>
>
> --
> Christian Bongiorno
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: term query boost problem
Posted by Christian Bongiorno <ch...@bongiorno.org>.
turns out the index is being built with lower-case terms which is why we
aren't getting hits the way we expect. When I change my search terms to
lower I see more of what I expect.
Gonna keep working on this and post updates.
On Wed, Aug 12, 2009 at 12:46 PM, Christian Bongiorno <
christian@bongiorno.org> wrote:
> I have a situation where I have a series of terms queries as part of a
> BooleanQuery.
>
> example:
>
> term: 'sole type' -> leather
> BooleanClause.SHOULD_OCCURR
> term: 'title' -> 'Men's Golf shoes'
> BooleanClause.SHOULD_OCCURR
> ...
>
> But, some terms are incredibly powerful indicators of match
> term: 'band type' -> 'gold'
>
> Normally, I would set this as a field boost in the query. Problem is, it
> isn't producing ANY changes even when I set the boost score absurdly high. I
> have also tried to set the BooleanClause.MUST on my really important term
> and I get no results at all.
>
> I tried a simple TermQuery for that (no clause) and got no results. Below
> is a code snippet. I checked through the recent archives discussing field
> boosts and I am pretty confident I am doing it right. So, now I am presuming
> it's a problem with my query.
>
> private Query buildQuery(Map<String, String> CatalogInfo) {
> if (CatalogInfo != null && CatalogInfo.size() > 0) {
> BooleanQuery booleanQuery = new BooleanQuery();
> for (Map.Entry<String, String> attributeValue :
> CatalogInfo.entrySet()) {
> String attributeName = attributeValue.getKey();
> String[] attributeValues;
> if (attributeValue.getValue().indexOf(VALUES_DELIMITER) ==
> -1) {
> attributeValues = new String[]
> {attributeValue.getValue()};
> } else {
> attributeValues =
> attributeValue.getValue().split(VALUES_DELIMITER);
> }
> for (String attributeValue : attributeValues) {
> String escapedValue =
> QueryParser.escape(attributeValue).trim();
> TermQuery termQuery = new TermQuery(new
> Term(attributeName, escapedValue));
>
>
> Float boostNumber = _boostMap.get(attributeName); //
> this is where 'band type' gets it's boost
> if (boostNumber != null) {
> LOG.warn("Boost value found: " + boostNumber);
> termQuery.setBoost(boostNumber);
> }
> booleanQuery.add(termQuery,
> BooleanClause.Occur.SHOULD);
> }
> }
> LOG.warn("Boolean query: " + booleanQuery.toString());
> return booleanQuery;
> }
> return null;
> }
> --
> Christian Bongiorno
>
>
--
Christian Bongiorno
Re: term query boost problem
Posted by Grant Ingersoll <gs...@apache.org>.
You have a bunch of log statements in there, what are they printing out?
Also, IndexSearcher.explain() is your friend for understanding why a
doc matched the way it did.
On Aug 12, 2009, at 3:46 PM, Christian Bongiorno wrote:
> I have a situation where I have a series of terms queries as part of a
> BooleanQuery.
>
> example:
>
> term: 'sole type' -> leather
> BooleanClause.SHOULD_OCCURR
> term: 'title' -> 'Men's Golf shoes'
> BooleanClause.SHOULD_OCCURR
> ...
>
> But, some terms are incredibly powerful indicators of match
> term: 'band type' -> 'gold'
>
> Normally, I would set this as a field boost in the query. Problem
> is, it
> isn't producing ANY changes even when I set the boost score absurdly
> high. I
> have also tried to set the BooleanClause.MUST on my really important
> term
> and I get no results at all.
>
> I tried a simple TermQuery for that (no clause) and got no results.
> Below is
> a code snippet. I checked through the recent archives discussing field
> boosts and I am pretty confident I am doing it right. So, now I am
> presuming
> it's a problem with my query.
>
> private Query buildQuery(Map<String, String> CatalogInfo) {
> if (CatalogInfo != null && CatalogInfo.size() > 0) {
> BooleanQuery booleanQuery = new BooleanQuery();
> for (Map.Entry<String, String> attributeValue :
> CatalogInfo.entrySet()) {
> String attributeName = attributeValue.getKey();
> String[] attributeValues;
> if
> (attributeValue.getValue().indexOf(VALUES_DELIMITER) ==
> -1) {
> attributeValues = new String[]
> {attributeValue.getValue()};
> } else {
> attributeValues =
> attributeValue.getValue().split(VALUES_DELIMITER);
> }
> for (String attributeValue : attributeValues) {
> String escapedValue =
> QueryParser.escape(attributeValue).trim();
> TermQuery termQuery = new TermQuery(new
> Term(attributeName, escapedValue));
>
>
> Float boostNumber =
> _boostMap.get(attributeName); //
> this is where 'band type' gets it's boost
> if (boostNumber != null) {
> LOG.warn("Boost value found: " + boostNumber);
> termQuery.setBoost(boostNumber);
> }
> booleanQuery.add(termQuery,
> BooleanClause.Occur.SHOULD);
> }
> }
> LOG.warn("Boolean query: " + booleanQuery.toString());
> return booleanQuery;
> }
> return null;
> }
> --
> Christian Bongiorno
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org