You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Toke Eskildsen (JIRA)" <ji...@apache.org> on 2010/11/02 22:52:31 UTC
[jira] Commented: (LUCENE-2735) First Cut at GroupVarInt with
FixedIntBlockIndexInput / Output
[ https://issues.apache.org/jira/browse/LUCENE-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927635#action_12927635 ]
Toke Eskildsen commented on LUCENE-2735:
----------------------------------------
I tried making an extra test in GVintMicroBenchmark that created the same output as groupintsTest but just read all the bytes directly back, no decoding, using IndexOutput. It's called GroupVarIntRead in the output below and as can be seen, most of the processing seems to take place outside of GVint decoding. Sorry no patch, as I messed up the formatting.
{code}
Running 4 Million random ints with max value: 64
GroupVarInt time per value: 4ns - time to decode 4M ints: 18ms
GroupVarIntRead time per value: 3ns - time to decode 4M ints: 15ms
Vint time per value: 6ns - time to decode 4M ints: 25ms
Running 4 Million random ints with max value: 1024
GroupVarInt time per value: 9ns - time to decode 4M ints: 39ms
GroupVarIntRead time per value: 5ns - time to decode 4M ints: 23ms
Vint time per value: 11ns - time to decode 4M ints: 44ms
Running 4 Million random ints with max value: 524288
GroupVarInt time per value: 12ns - time to decode 4M ints: 51ms
GroupVarIntRead time per value: 9ns - time to decode 4M ints: 38ms
Vint time per value: 14ns - time to decode 4M ints: 56ms
Running 4 Million random ints with max value: 67108864
GroupVarInt time per value: 14ns - time to decode 4M ints: 59ms
GroupVarIntRead time per value: 11ns - time to decode 4M ints: 45ms
Vint time per value: 15ns - time to decode 4M ints: 61ms
{code}
> First Cut at GroupVarInt with FixedIntBlockIndexInput / Output
> --------------------------------------------------------------
>
> Key: LUCENE-2735
> URL: https://issues.apache.org/jira/browse/LUCENE-2735
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-2735.patch, LUCENE-2735.patch, LUCENE-2735_alt.patch
>
>
> I have hacked together a FixedIntBlockIndex impl with Group VarInt encoding - this does way worse than standard codec in benchmarks but I guess that is mainly due to the FixedIntBlockIndex limitations. Once LUCENE-2723 is in / or builds with trunk again I will update and run some tests. The isolated microbenchmark shows that there could be improvements over vint even in java though and I am sure we can make it faster impl. wise.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org