You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Tomás Fernández Löbbe (Created JIRA)" <ji...@apache.org> on 2012/03/13 22:08:40 UTC
[jira] [Created] (SOLR-3241) Document boost fail if a field copy
omit the norms
Document boost fail if a field copy omit the norms
--------------------------------------------------
Key: SOLR-3241
URL: https://issues.apache.org/jira/browse/SOLR-3241
Project: Solr
Issue Type: Bug
Reporter: Tomás Fernández Löbbe
Fix For: 4.0
Attachments: SOLR-3241.patch
After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
<field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
<field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
<copyField source="author" dest="author_display"/>
I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228930#comment-13228930 ]
Robert Muir commented on SOLR-3241:
-----------------------------------
{quote}
but if the schema says to copyField from foo->bar where bar has omitNorms==true then i think that will currently cause an from the lucene low level check, corret?
{quote}
I have no idea.. does copyField actually copy field boosts too?
The attached test in the patch is a case of document boosting + copyField only.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Hoss Man (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229298#comment-13229298 ]
Hoss Man commented on SOLR-3241:
--------------------------------
patch looks fine ... i wish there was a way to make it easier for poly fields so they wouldn't have do do the check themselves, but when i tried the idea i had it didn't work, so better to go with this for now and maybe refactor a helper method later.
the few changes i would make:
1) make the new tests grab the IndexSchema obejct and assert that every field (that the cares about) has the expected omitNorms value -- future proof ourselves against someone nuetering the test w/o realizing by tweaking the test schema because they don't know that there is a specific reason for those omitNorm settings
2) add a test that explicitly verifies the failure case of someone setting field boost on a field with omitNorms==true, assert that we get the expected error mesg (doesn't look like this was added when LUCENE-3796 was commited, and we want to make sure we don't inadvertantly break that error check)
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved SOLR-3241.
-------------------------------
Resolution: Fixed
Assignee: Robert Muir
Thanks Tomás!
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Assignee: Robert Muir
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Tomás Fernández Löbbe (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229134#comment-13229134 ]
Tomás Fernández Löbbe commented on SOLR-3241:
---------------------------------------------
I found the issue with copyfields as you mentioned Robert. foo is omitNorms=false, and bar is omitNorms=true. I have a copyfield foo->bar and I add a document like:
<document boost=X>
<field name=foo>AAAAAA</field>
</document>
This case is fixed by the patch. Testing it, i found a similar situation where a field1 is a poly type with omitNorms=false, and the subtype if it has omitNorms=true. In this case, it fails even without a copyfield just by adding a document like:
<document boost=X>
<field name=poly>AAA,BBB</field>
</document>
I don't know if it makes sense to have a poly field where the subtype have a different value in the "omitNorms" attribute, probably this should fail even before the document is added.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Tomás Fernández Löbbe (Updated JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tomás Fernández Löbbe updated SOLR-3241:
----------------------------------------
Attachment: SOLR-3241.patch
I'm attaching another patch with some more tests.
Also updated the DocumentBuilder to use the existing logic instead of replicating it where the fix is applied.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229243#comment-13229243 ]
Robert Muir commented on SOLR-3241:
-----------------------------------
Thanks, patch looks good!
I'll wait a bit for any other input.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Updated] (SOLR-3241) Document boost fail if a field copy omit the norms
Posted by Bill Bell <bi...@gmail.com>.
Do we have this backported too?
Sent from my Mobile device
720-256-8076
On Mar 14, 2012, at 10:56 AM, "Robert Muir (Updated) (JIRA)" <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Robert Muir updated SOLR-3241:
> ------------------------------
>
> Attachment: SOLR-3241.patch
>
> updated patch with hossman's suggested test improvements. I'll commit soon.
>
>> Document boost fail if a field copy omit the norms
>> --------------------------------------------------
>>
>> Key: SOLR-3241
>> URL: https://issues.apache.org/jira/browse/SOLR-3241
>> Project: Solr
>> Issue Type: Bug
>> Reporter: Tomás Fernández Löbbe
>> Fix For: 3.6, 4.0
>>
>> Attachments: SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch
>>
>>
>> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
>> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
>> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
>> <copyField source="author" dest="author_display"/>
>> I'm attaching a possible fix.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-3241:
------------------------------
Attachment: SOLR-3241.patch
updated patch with hossman's suggested test improvements. I'll commit soon.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Hoss Man (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228914#comment-13228914 ]
Hoss Man commented on SOLR-3241:
--------------------------------
part of me things we should just remove the error checking for {{omitNorms && boost != 1.0F}} from DocumentBuilder.toDocument (added in /LUCENE-3796) and just silently ignore any boost on a SolrInputField where omitNorms==true (ie: maybe log a warning, but don't throw an Exception). This would be consistent with the behavior in past releases (except for the warning log if we add that), and wouldn't cause any confusing errors for things like LatLonType (even if they come from third-party plugins we can't contro/test)
On the other hand... that feels really dirty, and it would be nice to fail fast and loud if the client tries to set a boost on an omitNorms field
Perhaps a better fix would be to leave DocumentBuilder exactly as it is today, and instead change FieldType.createField to (silently) ignore the boost if omitNorms==true for that SchemaField. if i'm thinking about this right, that would mean the error checking of the SolrInputDocument (and all it's SolrInputFields) in DocumentBuilder.toDocument would still work as designed -- so you'd get an error if any client or "high level" plugin like an UpdateProcessor tried to use a field boost on an omitNorms field; but any fields added at a lower level (ie: by copyField or a poly field) would silently ignore those boosts if they were copied/cloned to a field where omitNorms==true.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Tomás Fernández Löbbe (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228841#comment-13228841 ]
Tomás Fernández Löbbe commented on SOLR-3241:
---------------------------------------------
This still doesn't work for Types with subtypes, like LatLonType and CurrencyType
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-3241:
------------------------------
Attachment: SOLR-3241.patch
Updated patch with fixes for the polyfield case (untested!)
After reviewing the code: Tomás had the correct fix for the copyField case, his patch fixes a logic bug, nothing more!
The polyField case is different: its too late in DocumentBuilder to do anything here after the creation of IndexableFields: moreover we cannot nuke the whole boost for the field because we cannot assume anything just because isPolyField() == true, for example a custom field type might not even be instanceof AbstractSubTypeField!
Because of this I think these fieldtypes should really treat the fact they use 'real fields' as an impl detail. so I added the logic to their subfield creation.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch, SOLR-3241.patch, SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-3241:
------------------------------
Attachment: SOLR-3241.patch
updating the patch from Tomás to include an additional test for the field boost+copyField.
we still need to add tests for this polyField case, and any other possibilities (can you polyField+copyField?)
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch, SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228921#comment-13228921 ]
Robert Muir commented on SOLR-3241:
-----------------------------------
{quote}
Perhaps a better fix would be to leave DocumentBuilder exactly as it is today, and instead change FieldType.createField to (silently) ignore the boost if omitNorms==true for that SchemaField. if i'm thinking about this right, that would mean the error checking of the SolrInputDocument (and all it's SolrInputFields) in DocumentBuilder.toDocument would still work as designed – so you'd get an error if any client or "high level" plugin like an UpdateProcessor tried to use a field boost on an omitNorms field; but any fields added at a lower level (ie: by copyField or a poly field) would silently ignore those boosts if they were copied/cloned to a field where omitNorms==true.
{quote}
Hoss: this sounds good to me? we can add tests that the basic case is still working.
The reason the logic was somewhat complicated in DocumentBuilder is because, from the lucene indexer its easy to detect this case, but:
# As mentioned before, "document boost" is something only in solr now, so the idea was not to fail just because you had omitNorms field and solr multiplied document boost into field boosts. But this copyField/polyField is an exception as you mention
# explicit handling was added to DocumentBuilder because its better for Solr to throw an exception here than rely upon lucene, lucene's exception
is "nastier" in the sense its a lower-level non-aborting exception that will produce a deleted document.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Hoss Man (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228924#comment-13228924 ]
Hoss Man commented on SOLR-3241:
--------------------------------
bq. The reason the logic was somewhat complicated in DocumentBuilder is because, from the lucene indexer its easy to detect this case, but:
sure ... but i think it's not actually just "Document Boost" is it? if field "foo" is declared with omitNorms==false, and a client sends a doc with a field "foo" using a fieldBoost then that should be totally fine -- but if the schema says to copyField from foo->bar where bar has omitNorms==true then i think that will currently cause an from the lucene low level check, corret? (i haven't tried it, i'm going based on tomas's path) likewise if "foo" is a LatLonField (or any other polyfield) and the underlying dynamic field used has omitNorms==true then won't that same low level lucene code throw an error there?
so multiple error paths from totally sane usage none of which has anything to do with doc boost, right?
(Truth be told, i didn't even notice the "Document boost" part of the summary, i was just looking at tomas's patch and skimming the summary)
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated SOLR-3241:
------------------------------
Fix Version/s: 3.6
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Tomás Fernández Löbbe (Updated JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tomás Fernández Löbbe updated SOLR-3241:
----------------------------------------
Attachment: SOLR-3241.patch
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3241) Document boost fail if a field copy
omit the norms
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228916#comment-13228916 ]
Robert Muir commented on SOLR-3241:
-----------------------------------
Thanks Tomás (and thanks for the patch!) I'll look into this.
The concept of document boost doesn't really exist anymore, and I don't think we should fail for an omitNorms field
just because of the document boost. It just won't be applied to that field.
The point is only to fail if someone explicitly boosts an omitNorms field itself.
> Document boost fail if a field copy omit the norms
> --------------------------------------------------
>
> Key: SOLR-3241
> URL: https://issues.apache.org/jira/browse/SOLR-3241
> Project: Solr
> Issue Type: Bug
> Reporter: Tomás Fernández Löbbe
> Fix For: 4.0
>
> Attachments: SOLR-3241.patch
>
>
> After https://issues.apache.org/jira/browse/LUCENE-3796, it is not possible to set a boost to a field that has the "omitNorms" set to true. This is making Solr's document index-time boost to fail when a field that doesn't omit norms is copied (with copyField) to a field that does omit them and document boost is used. For example:
> <field name="author" type="text" indexed="true" stored="false" omitNorms="false"/>
> <field name="author_display" type="string" indexed="true" stored="true" omitNorms="true"/>
> <copyField source="author" dest="author_display"/>
> I'm attaching a possible fix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org