You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Simon Willnauer (JIRA)" <ji...@apache.org> on 2011/06/13 15:14:51 UTC
[jira] [Created] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Optimize FixedStraightBytes for bytes size == 1
-----------------------------------------------
Key: LUCENE-3196
URL: https://issues.apache.org/jira/browse/LUCENE-3196
Project: Lucene - Java
Issue Type: Improvement
Components: core/index, core/search
Affects Versions: 4.0
Reporter: Simon Willnauer
Priority: Minor
Fix For: 4.0
Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048615#comment-13048615 ]
Simon Willnauer commented on LUCENE-3196:
-----------------------------------------
bq. Do we have a test (eg a random test that picks random fixed byte[] size) that covers this...?
yes the fixed length is selected at random in the tests I fixed that in the patch too.
> Optimize FixedStraightBytes for bytes size == 1
> -----------------------------------------------
>
> Key: LUCENE-3196
> URL: https://issues.apache.org/jira/browse/LUCENE-3196
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-3196.patch
>
>
> Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048606#comment-13048606 ]
Michael McCandless commented on LUCENE-3196:
--------------------------------------------
Looks good Simon!
Probably other smallish sizes (2, 3, 4, ...) could be a single array too, ie paged or not should be separately controllable, but we can do that later; this is a great baby step since we need this for norms cutover.
Do we have a test (eg a random test that picks random fixed byte[] size) that covers this...?
> Optimize FixedStraightBytes for bytes size == 1
> -----------------------------------------------
>
> Key: LUCENE-3196
> URL: https://issues.apache.org/jira/browse/LUCENE-3196
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-3196.patch
>
>
> Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048787#comment-13048787 ]
Uwe Schindler commented on LUCENE-3196:
---------------------------------------
Looks fine, using this approach, separate norms impl can hopefully go away quite fast *g*
For the PreFlex codec I even have an idea for the codec and backwards compatibility: The old norms file could be exposed as standard DocValues field by PreFlex codec. The r/w StandardCodec would never write separate norms files, instead simply write docvalues using this 1 byte approach (of course configureable to have e.g. read float norms, and other additional BM25 statistics or whatever).
Just ideas, Uwe
> Optimize FixedStraightBytes for bytes size == 1
> -----------------------------------------------
>
> Key: LUCENE-3196
> URL: https://issues.apache.org/jira/browse/LUCENE-3196
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-3196.patch
>
>
> Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048757#comment-13048757 ]
Simon Willnauer commented on LUCENE-3196:
-----------------------------------------
I am planning to commit this soon if nobody objects.
> Optimize FixedStraightBytes for bytes size == 1
> -----------------------------------------------
>
> Key: LUCENE-3196
> URL: https://issues.apache.org/jira/browse/LUCENE-3196
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-3196.patch
>
>
> Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer resolved LUCENE-3196.
-------------------------------------
Resolution: Fixed
Committed in revision 1135293.
> Optimize FixedStraightBytes for bytes size == 1
> -----------------------------------------------
>
> Key: LUCENE-3196
> URL: https://issues.apache.org/jira/browse/LUCENE-3196
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-3196.patch
>
>
> Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048616#comment-13048616 ]
Michael McCandless commented on LUCENE-3196:
--------------------------------------------
Ahh yes great! selckin's random number generator should hit 1 frequently ;)
> Optimize FixedStraightBytes for bytes size == 1
> -----------------------------------------------
>
> Key: LUCENE-3196
> URL: https://issues.apache.org/jira/browse/LUCENE-3196
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-3196.patch
>
>
> Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-3196:
------------------------------------
Attachment: LUCENE-3196.patch
here is a patch
> Optimize FixedStraightBytes for bytes size == 1
> -----------------------------------------------
>
> Key: LUCENE-3196
> URL: https://issues.apache.org/jira/browse/LUCENE-3196
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-3196.patch
>
>
> Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (LUCENE-3196) Optimize FixedStraightBytes for
bytes size == 1
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer reassigned LUCENE-3196:
---------------------------------------
Assignee: Simon Willnauer
> Optimize FixedStraightBytes for bytes size == 1
> -----------------------------------------------
>
> Key: LUCENE-3196
> URL: https://issues.apache.org/jira/browse/LUCENE-3196
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
>
> Currently we read all the bytes in a PagedBytes instance wich is unneeded for single byte values like norms. For fast access this should simply be a straight array.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org