You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Adrien Grand (JIRA)" <ji...@apache.org> on 2012/05/31 19:34:23 UTC
[jira] [Created] (LUCENE-4098) Efficient bulk operations for packed
integer arrays
Adrien Grand created LUCENE-4098:
------------------------------------
Summary: Efficient bulk operations for packed integer arrays
Key: LUCENE-4098
URL: https://issues.apache.org/jira/browse/LUCENE-4098
Project: Lucene - Java
Issue Type: Improvement
Components: core/other
Reporter: Adrien Grand
Priority: Minor
Fix For: 4.1
There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4098) Efficient bulk operations for
packed integer arrays
Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287581#comment-13287581 ]
Adrien Grand commented on LUCENE-4098:
--------------------------------------
The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{{code}}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{{code}}
should be replaced with
{{code}}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= len);
{{code}}
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Comment Edited] (LUCENE-4098) Efficient bulk operations for
packed integer arrays
Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287581#comment-13287581 ]
Adrien Grand edited comment on LUCENE-4098 at 6/1/12 6:24 PM:
--------------------------------------------------------------
The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{code}
should be replaced with
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= got);
{code}
was (Author: jpountz):
The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{code}
should be replaced with
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= len);
{code}
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed
integer arrays
Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrien Grand updated LUCENE-4098:
---------------------------------
Attachment: (was: LUCENE-4098.patch)
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed
integer arrays
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-4098:
---------------------------------------
Attachment: LUCENE-4098.patch
The patch looks great!
I wrote a new random test (in attached patch) that just randomly bulk copies a bunch of slices from one packed ints array to another and then asserts the values are correct ... but it's failing. I'm not sure why yet, and it's entirely possible it's a test bug! If you run with -Dtests.verbose=true it prints details...
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed
integer arrays
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-4098:
---------------------------------------
Attachment: LUCENE-4098.patch
Patch, w/ the fix for the new random test (now passes).
I also tweaked the random test to sometimes use the PackedInts.copy method.
Finally, I changed PackedInts.copy to alloc a long[] of size min(capacity, len) so we don't over-allocate if the incoming mem is larger than it needs to be.
I think it's ready! Thanks Adrien.
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch, LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed
integer arrays
Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrien Grand updated LUCENE-4098:
---------------------------------
Attachment: LUCENE-4098.patch
Here is the patch for the proposed modifications. All {{Mutable}} implementations have a new efficient {{fill}} method and Packed64SingleBlock* classes also have efficient bulk get and set.
For example, the execution time of the following (unrealistic) microbenchmark is more than twice better with the patch applied on my computer thanks to the use of {{PackedInts.copy}} instead of naive copy (see {{GrowableWriter#ensureCapacity}}).
{code}
for (int k = 0; k < 50; ++k) {
long start = System.nanoTime();
GrowableWriter wrt = new GrowableWriter(1, 1 << 22, PackedInts.DEFAULT);
for (int i = 0; i < 1 << 22; ++i) {
wrt.set(i,i);
}
long end = System.nanoTime();
System.out.println((end - start) / 1000000);
long sum = 0;
for (int i = 0; i < wrt.size(); ++i) {
sum += wrt.get(i);
}
System.out.println(sum);
}
{code}
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-4098) Efficient bulk operations for
packed integer arrays
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-4098.
----------------------------------------
Resolution: Fixed
Fix Version/s: (was: 4.1)
5.0
4.0
Thanks Adrien!
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4098.patch, LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed
integer arrays
Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrien Grand updated LUCENE-4098:
---------------------------------
Attachment: LUCENE-4098.patch
Slightly updated patch that fixes the computation of maxValue in GrowableWriter and uses | instead of Math.max to compute the max value when performing a bulk set (since GrowableWriter only works with unsigned integers).
As a side note, the reason why I didn't write bulk get and set methods for Packed64 is that I didn't find how to do it efficiently (= with less instructions and no conditional) without writing specialized methods for every bitsPerValue.
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4098) Efficient bulk operations for
packed integer arrays
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287584#comment-13287584 ]
Michael McCandless commented on LUCENE-4098:
--------------------------------------------
Aha, phew! A bug in the test ... with that change the test looks like it's passing so far ... I'll beast it for a while. Thanks Adrien.
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Comment Edited] (LUCENE-4098) Efficient bulk operations for
packed integer arrays
Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287581#comment-13287581 ]
Adrien Grand edited comment on LUCENE-4098 at 6/1/12 6:23 PM:
--------------------------------------------------------------
The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{code}
should be replaced with
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= len);
{code}
was (Author: jpountz):
The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{{code}}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{{code}}
should be replaced with
{{code}}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= len);
{{code}}
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (LUCENE-4098) Efficient bulk operations for
packed integer arrays
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-4098:
------------------------------------------
Assignee: Michael McCandless
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
> Key: LUCENE-4098
> URL: https://issues.apache.org/jira/browse/LUCENE-4098
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/other
> Reporter: Adrien Grand
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 4.1
>
> Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org