You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org> on 2012/02/17 09:29:59 UTC
[jira] [Commented] (LUCENE-3796) Disallow setBoost() on
StringField, throw exception if boosts are set if norms are omitted
[ https://issues.apache.org/jira/browse/LUCENE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210134#comment-13210134 ]
Uwe Schindler commented on LUCENE-3796:
---------------------------------------
+1 to apply patch. This also effects NumericFields and any other norms-free field.
> Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3796
> URL: https://issues.apache.org/jira/browse/LUCENE-3796
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
> Attachments: LUCENE-3796.patch
>
>
> Occasionally users are confused why index-time boosts are not applied to their norms-omitted fields.
> This is because we silently discard the boost: there is no reason for this!
> The most absurd part: in 4.0 you can make a StringField and call setBoost and nothing complains... (more reasons to remove StringField totally in my opinion)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on
StringField, throw exception if boosts are set if norms are omitted
Posted by Robert Muir <rc...@gmail.com>.
On Fri, Feb 17, 2012 at 11:53 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>
> In my opinion, per document boosts should maybe removed in trunk and people
> should be instructed to use DocValues and CustomScoreQuery.
>
This is already the case since last summer: boosts are only per-field
in trunk (there is no concept of document boost in the apis, since it
never existed anyway)
--
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
Posted by Uwe Schindler <uw...@thetaphi.de>.
Yeah. I was not sure if you was referring to Robert's patch where we throw
ex when boost!=1.0f and norms omitted.
In general, boosting documents (not fields) was never really supported by
Lucene, it was always just a "norms hack". And it only works when TermQuery
is involved. If you really want to boost documents, the way to go is to use
FloatDocValues for the per-doc boosts and use a CustomScoreQuery wrapping
your final parsed query including constant scores,...) that multiplies the
docvalues boost in (or write a Collector wrapper around your
TopDocsCollector). I have used this quite often (with FieldCache), it works
as you would expect. Otherwise constant score queries (or other queries like
MatchAllDocs) were never using the boosts and this was always a problem
(because people don't understand that).
In my opinion, per document boosts should maybe removed in trunk and people
should be instructed to use DocValues and CustomScoreQuery.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
> Seeley
> Sent: Friday, February 17, 2012 5:16 PM
> To: dev@lucene.apache.org
> Subject: Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on
> StringField, throw exception if boosts are set if norms are omitted
>
> On Fri, Feb 17, 2012 at 11:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> > We dont prohibit anything. If the field type/field has norms omitted
> > (which is the default for those fields), applying a boost will never
> > work. That's all.
>
> That's a fine default, but your answer to Jan suggested that boosts
> *never* work. I was just pointing out, that's not strictly true.
>
> -Yonik
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
Posted by Jan Høydahl <ja...@cominvent.com>.
Created SOLR-3140
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com
On 17. feb. 2012, at 19:39, Yonik Seeley wrote:
> On Fri, Feb 17, 2012 at 1:19 PM, Jan Høydahl <ja...@cominvent.com> wrote:
>> If omitNorms is default for all numeric field types
>
> That's only true for Lucene, not Solr currently.
>
>> , could we simplify Solr's example schema and perhaps document the defaults in a comment?
>
> They may make sense to simplify the schema, but would require some
> additional code and a CHANGES entry (i.e. to make the default for
> omitNorms true for all except text field or something).
>
> -Yonik
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on
StringField, throw exception if boosts are set if norms are omitted
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Feb 17, 2012 at 1:19 PM, Jan Høydahl <ja...@cominvent.com> wrote:
> If omitNorms is default for all numeric field types
That's only true for Lucene, not Solr currently.
> , could we simplify Solr's example schema and perhaps document the defaults in a comment?
They may make sense to simplify the schema, but would require some
additional code and a CHANGES entry (i.e. to make the default for
omitNorms true for all except text field or something).
-Yonik
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
Posted by Jan Høydahl <ja...@cominvent.com>.
If omitNorms is default for all numeric field types, could we simplify Solr's example schema and perhaps document the defaults in a comment?
--
Jan Høydahl
Den 17. feb. 2012 kl. 17:16 skrev Yonik Seeley <yo...@lucidimagination.com>:
> On Fri, Feb 17, 2012 at 11:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>> We dont prohibit anything. If the field type/field has norms omitted (which
>> is the default for those fields), applying a boost will never work. That's
>> all.
>
> That's a fine default, but your answer to Jan suggested that boosts
> *never* work. I was just pointing out, that's not strictly true.
>
> -Yonik
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on
StringField, throw exception if boosts are set if norms are omitted
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Feb 17, 2012 at 11:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> We dont prohibit anything. If the field type/field has norms omitted (which
> is the default for those fields), applying a boost will never work. That's
> all.
That's a fine default, but your answer to Jan suggested that boosts
*never* work. I was just pointing out, that's not strictly true.
-Yonik
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Yonik,
We dont prohibit anything. If the field type/field has norms omitted (which
is the default for those fields), applying a boost will never work. That's
all.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
> Seeley
> Sent: Friday, February 17, 2012 4:58 PM
> To: dev@lucene.apache.org
> Subject: Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on
> StringField, throw exception if boosts are set if norms are omitted
>
> On Fri, Feb 17, 2012 at 5:50 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> > Index time boost is not applied, because e.g. NumericRangeQuery is a
> > ConstantScoreQuery.
>
> Someone could add boosts to these fields and use term queries though.
> I've seen it done in the past, and there's not a good reason to prohibit
that.
>
> -Yonik
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on
StringField, throw exception if boosts are set if norms are omitted
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Feb 17, 2012 at 5:50 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Index time boost is not applied, because e.g. NumericRangeQuery is a
> ConstantScoreQuery.
Someone could add boosts to these fields and use term queries though.
I've seen it done in the past, and there's not a good reason to
prohibit that.
-Yonik
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
Posted by Jan Høydahl <ja...@cominvent.com>.
So can we always omit norms for these types and remove the (then) unnecessary omitNorms="true" in Solr's schema?
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com
On 17. feb. 2012, at 11:50, Uwe Schindler wrote:
> Index time boost is not applied, because e.g. NumericRangeQuery is a
> ConstantScoreQuery.
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Jan Høydahl [mailto:jan.asf@cominvent.com]
>> Sent: Friday, February 17, 2012 11:47 AM
>> To: dev@lucene.apache.org
>> Subject: Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on
>> StringField, throw exception if boosts are set if norms are omitted
>>
>> A question related to this. Is it ever meaningful to include norms for
> "int",
>> "date", "long"... fieldTypes? Length normalization certainly doesn't make
> sense
>> and what happens if you add an index-time boost to an int field? Would the
>> boost be applied for queries like price:100 or price:[100 TO 200] ?
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> Solr Training - www.solrtraining.com
>>
>> On 17. feb. 2012, at 09:29, Uwe Schindler (Commented) (JIRA) wrote:
>>
>>>
>>> [
>>> https://issues.apache.org/jira/browse/LUCENE-3796?page=com.atlassian.j
>>> ira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=132
>>> 10134#comment-13210134 ]
>>>
>>> Uwe Schindler commented on LUCENE-3796:
>>> ---------------------------------------
>>>
>>> +1 to apply patch. This also effects NumericFields and any other
> norms-free
>> field.
>>>
>>>> Disallow setBoost() on StringField, throw exception if boosts are set
>>>> if norms are omitted
>>>> ---------------------------------------------------------------------
>>>> ---------------------
>>>>
>>>> Key: LUCENE-3796
>>>> URL: https://issues.apache.org/jira/browse/LUCENE-3796
>>>> Project: Lucene - Java
>>>> Issue Type: Bug
>>>> Reporter: Robert Muir
>>>> Priority: Blocker
>>>> Fix For: 4.0
>>>>
>>>> Attachments: LUCENE-3796.patch
>>>>
>>>>
>>>> Occasionally users are confused why index-time boosts are not applied
> to
>> their norms-omitted fields.
>>>> This is because we silently discard the boost: there is no reason for
> this!
>>>> The most absurd part: in 4.0 you can make a StringField and call
>>>> setBoost and nothing complains... (more reasons to remove StringField
>>>> totally in my opinion)
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> If you think it was sent incorrectly, please contact your JIRA
>>> administrators:
>>> https://issues.apache.org/jira/secure/ContactAdministrators!default.js
>>> pa For more information on JIRA, see:
>>> http://www.atlassian.com/software/jira
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>> additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>> commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
Posted by Uwe Schindler <uw...@thetaphi.de>.
Index time boost is not applied, because e.g. NumericRangeQuery is a
ConstantScoreQuery.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Jan Høydahl [mailto:jan.asf@cominvent.com]
> Sent: Friday, February 17, 2012 11:47 AM
> To: dev@lucene.apache.org
> Subject: Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on
> StringField, throw exception if boosts are set if norms are omitted
>
> A question related to this. Is it ever meaningful to include norms for
"int",
> "date", "long"... fieldTypes? Length normalization certainly doesn't make
sense
> and what happens if you add an index-time boost to an int field? Would the
> boost be applied for queries like price:100 or price:[100 TO 200] ?
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> Solr Training - www.solrtraining.com
>
> On 17. feb. 2012, at 09:29, Uwe Schindler (Commented) (JIRA) wrote:
>
> >
> > [
> > https://issues.apache.org/jira/browse/LUCENE-3796?page=com.atlassian.j
> > ira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=132
> > 10134#comment-13210134 ]
> >
> > Uwe Schindler commented on LUCENE-3796:
> > ---------------------------------------
> >
> > +1 to apply patch. This also effects NumericFields and any other
norms-free
> field.
> >
> >> Disallow setBoost() on StringField, throw exception if boosts are set
> >> if norms are omitted
> >> ---------------------------------------------------------------------
> >> ---------------------
> >>
> >> Key: LUCENE-3796
> >> URL: https://issues.apache.org/jira/browse/LUCENE-3796
> >> Project: Lucene - Java
> >> Issue Type: Bug
> >> Reporter: Robert Muir
> >> Priority: Blocker
> >> Fix For: 4.0
> >>
> >> Attachments: LUCENE-3796.patch
> >>
> >>
> >> Occasionally users are confused why index-time boosts are not applied
to
> their norms-omitted fields.
> >> This is because we silently discard the boost: there is no reason for
this!
> >> The most absurd part: in 4.0 you can make a StringField and call
> >> setBoost and nothing complains... (more reasons to remove StringField
> >> totally in my opinion)
> >
> > --
> > This message is automatically generated by JIRA.
> > If you think it was sent incorrectly, please contact your JIRA
> > administrators:
> > https://issues.apache.org/jira/secure/ContactAdministrators!default.js
> > pa For more information on JIRA, see:
> > http://www.atlassian.com/software/jira
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (LUCENE-3796) Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
Posted by Jan Høydahl <ja...@cominvent.com>.
A question related to this. Is it ever meaningful to include norms for "int", "date", "long"... fieldTypes? Length normalization certainly doesn't make sense and what happens if you add an index-time boost to an int field? Would the boost be applied for queries like price:100 or price:[100 TO 200] ?
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com
On 17. feb. 2012, at 09:29, Uwe Schindler (Commented) (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/LUCENE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210134#comment-13210134 ]
>
> Uwe Schindler commented on LUCENE-3796:
> ---------------------------------------
>
> +1 to apply patch. This also effects NumericFields and any other norms-free field.
>
>> Disallow setBoost() on StringField, throw exception if boosts are set if norms are omitted
>> ------------------------------------------------------------------------------------------
>>
>> Key: LUCENE-3796
>> URL: https://issues.apache.org/jira/browse/LUCENE-3796
>> Project: Lucene - Java
>> Issue Type: Bug
>> Reporter: Robert Muir
>> Priority: Blocker
>> Fix For: 4.0
>>
>> Attachments: LUCENE-3796.patch
>>
>>
>> Occasionally users are confused why index-time boosts are not applied to their norms-omitted fields.
>> This is because we silently discard the boost: there is no reason for this!
>> The most absurd part: in 4.0 you can make a StringField and call setBoost and nothing complains... (more reasons to remove StringField totally in my opinion)
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org