You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Adrien Grand (JIRA)" <ji...@apache.org> on 2012/09/04 18:30:07 UTC

[jira] [Created] (LUCENE-4359) Allow DocValues to be unstored

Adrien Grand created LUCENE-4359:
------------------------------------

             Summary: Allow DocValues to be unstored
                 Key: LUCENE-4359
                 URL: https://issues.apache.org/jira/browse/LUCENE-4359
             Project: Lucene - Core
          Issue Type: Sub-task
    Affects Versions: 5.0
            Reporter: Adrien Grand


Lucene trunk currently forces DocValues to be stored:
 - FieldType.setDocValueType sets stored to true,
 - DocFieldProcessor.processDocument handles DocValues in a {{for (StorableField field: docState.doc.storableFields())}} loop, meaning that DocValues won't be consumed if they are not stored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Commented] (LUCENE-4359) Allow DocValues to be unstored

Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448090#comment-13448090 ] 

Adrien Grand commented on LUCENE-4359:
--------------------------------------

Robert, thanks for fixing this issue so quickly!
                
> Allow DocValues to be unstored
> ------------------------------
>
>                 Key: LUCENE-4359
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4359
>             Project: Lucene - Core
>          Issue Type: Sub-task
>    Affects Versions: 5.0
>            Reporter: Adrien Grand
>            Assignee: Robert Muir
>             Fix For: 5.0
>
>         Attachments: LUCENE-4359.patch, LUCENE-4359_test.patch
>
>
> Lucene trunk currently forces DocValues to be stored:
>  - FieldType.setDocValueType sets stored to true,
>  - DocFieldProcessor.processDocument handles DocValues in a {{for (StorableField field: docState.doc.storableFields())}} loop, meaning that DocValues won't be consumed if they are not stored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Commented] (LUCENE-4359) Allow DocValues to be unstored

Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447796#comment-13447796 ] 

Robert Muir commented on LUCENE-4359:
-------------------------------------

Nice catch: I think this is related to LUCENE-4349, in that we should treat DocValues differently (neither stored nor indexed, its just its own thing)

                
> Allow DocValues to be unstored
> ------------------------------
>
>                 Key: LUCENE-4359
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4359
>             Project: Lucene - Core
>          Issue Type: Sub-task
>    Affects Versions: 5.0
>            Reporter: Adrien Grand
>
> Lucene trunk currently forces DocValues to be stored:
>  - FieldType.setDocValueType sets stored to true,
>  - DocFieldProcessor.processDocument handles DocValues in a {{for (StorableField field: docState.doc.storableFields())}} loop, meaning that DocValues won't be consumed if they are not stored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Resolved] (LUCENE-4359) Allow DocValues to be unstored

Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir resolved LUCENE-4359.
---------------------------------

       Resolution: Fixed
    Fix Version/s: 5.0
    
> Allow DocValues to be unstored
> ------------------------------
>
>                 Key: LUCENE-4359
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4359
>             Project: Lucene - Core
>          Issue Type: Sub-task
>    Affects Versions: 5.0
>            Reporter: Adrien Grand
>            Assignee: Robert Muir
>             Fix For: 5.0
>
>         Attachments: LUCENE-4359.patch, LUCENE-4359_test.patch
>
>
> Lucene trunk currently forces DocValues to be stored:
>  - FieldType.setDocValueType sets stored to true,
>  - DocFieldProcessor.processDocument handles DocValues in a {{for (StorableField field: docState.doc.storableFields())}} loop, meaning that DocValues won't be consumed if they are not stored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Updated] (LUCENE-4359) Allow DocValues to be unstored

Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-4359:
--------------------------------

    Attachment: LUCENE-4359_test.patch

just adding a test: i ran into another bug (i think i saw this on the issue tracker, will look) that confused me for a second, as my test initially passed with assertNull(document.get("dv")):

{noformat}
doc 0
  numfields 2
  field 1
    name dv
    type long
    value 0
  field 0
    name docId
    type string
    value 0
{noformat}

This second bug is in Document/StoredDocument for numeric fields:
{code}
   /** Returns the string value of the field with the given name if any exist in
    * this document, or null.  If multiple fields exist with this name, this
    * method returns the first value added. If only binary fields with this name
    * exist, returns null.
    * For {@link IntField}, {@link LongField}, {@link
    * FloatField} and {@link DoubleField} it returns the string value of the number. If you want
    * the actual numeric field instance back, use {@link #getField}.
    */
   public final String get(String name) {
     for (StorableField field : fields) {
       if (field.name().equals(name) && field.stringValue() != null) {
         return field.stringValue();
       }
     }
     return null;
   }
{code}

It clearly does not return the string value of a number for numeric fields. Ill separate this into another issue (or reuse the existing one if i find it)
                
> Allow DocValues to be unstored
> ------------------------------
>
>                 Key: LUCENE-4359
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4359
>             Project: Lucene - Core
>          Issue Type: Sub-task
>    Affects Versions: 5.0
>            Reporter: Adrien Grand
>         Attachments: LUCENE-4359_test.patch
>
>
> Lucene trunk currently forces DocValues to be stored:
>  - FieldType.setDocValueType sets stored to true,
>  - DocFieldProcessor.processDocument handles DocValues in a {{for (StorableField field: docState.doc.storableFields())}} loop, meaning that DocValues won't be consumed if they are not stored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Assigned] (LUCENE-4359) Allow DocValues to be unstored

Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir reassigned LUCENE-4359:
-----------------------------------

    Assignee: Robert Muir
    
> Allow DocValues to be unstored
> ------------------------------
>
>                 Key: LUCENE-4359
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4359
>             Project: Lucene - Core
>          Issue Type: Sub-task
>    Affects Versions: 5.0
>            Reporter: Adrien Grand
>            Assignee: Robert Muir
>         Attachments: LUCENE-4359_test.patch
>
>
> Lucene trunk currently forces DocValues to be stored:
>  - FieldType.setDocValueType sets stored to true,
>  - DocFieldProcessor.processDocument handles DocValues in a {{for (StorableField field: docState.doc.storableFields())}} loop, meaning that DocValues won't be consumed if they are not stored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Commented] (LUCENE-4359) Allow DocValues to be unstored

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447951#comment-13447951 ] 

Michael McCandless commented on LUCENE-4359:
--------------------------------------------

+1
                
> Allow DocValues to be unstored
> ------------------------------
>
>                 Key: LUCENE-4359
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4359
>             Project: Lucene - Core
>          Issue Type: Sub-task
>    Affects Versions: 5.0
>            Reporter: Adrien Grand
>            Assignee: Robert Muir
>         Attachments: LUCENE-4359.patch, LUCENE-4359_test.patch
>
>
> Lucene trunk currently forces DocValues to be stored:
>  - FieldType.setDocValueType sets stored to true,
>  - DocFieldProcessor.processDocument handles DocValues in a {{for (StorableField field: docState.doc.storableFields())}} loop, meaning that DocValues won't be consumed if they are not stored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Updated] (LUCENE-4359) Allow DocValues to be unstored

Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-4359:
--------------------------------

    Attachment: LUCENE-4359.patch

here's a patch. it doesn't fix all the issues but I think we should commit it.

I think its really confusing that setDocValuesType sets stored as a side effect.

                
> Allow DocValues to be unstored
> ------------------------------
>
>                 Key: LUCENE-4359
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4359
>             Project: Lucene - Core
>          Issue Type: Sub-task
>    Affects Versions: 5.0
>            Reporter: Adrien Grand
>            Assignee: Robert Muir
>         Attachments: LUCENE-4359.patch, LUCENE-4359_test.patch
>
>
> Lucene trunk currently forces DocValues to be stored:
>  - FieldType.setDocValueType sets stored to true,
>  - DocFieldProcessor.processDocument handles DocValues in a {{for (StorableField field: docState.doc.storableFields())}} loop, meaning that DocValues won't be consumed if they are not stored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org