You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Adrien Grand (JIRA)" <ji...@apache.org> on 2012/05/31 19:34:23 UTC

[jira] [Created] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Adrien Grand created LUCENE-4098:
------------------------------------

             Summary: Efficient bulk operations for packed integer arrays
                 Key: LUCENE-4098
                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
             Project: Lucene - Java
          Issue Type: Improvement
          Components: core/other
            Reporter: Adrien Grand
            Priority: Minor
             Fix For: 4.1


There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.

For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.

However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Commented] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287581#comment-13287581 ] 

Adrien Grand commented on LUCENE-4098:
--------------------------------------

The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{{code}}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{{code}}
should be replaced with
{{code}}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= len);
{{code}}
                
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Comment Edited] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287581#comment-13287581 ] 

Adrien Grand edited comment on LUCENE-4098 at 6/1/12 6:24 PM:
--------------------------------------------------------------

The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{code}
should be replaced with
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= got);
{code}
                
      was (Author: jpountz):
    The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{code}
should be replaced with
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= len);
{code}
                  
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand updated LUCENE-4098:
---------------------------------

    Attachment:     (was: LUCENE-4098.patch)
    
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless updated LUCENE-4098:
---------------------------------------

    Attachment: LUCENE-4098.patch

The patch looks great!

I wrote a new random test (in attached patch) that just randomly bulk copies a bunch of slices from one packed ints array to another and then asserts the values are correct ... but it's failing.  I'm not sure why yet, and it's entirely possible it's a test bug!  If you run with -Dtests.verbose=true it prints details...
                
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless updated LUCENE-4098:
---------------------------------------

    Attachment: LUCENE-4098.patch

Patch, w/ the fix for the new random test (now passes).

I also tweaked the random test to sometimes use the PackedInts.copy method.

Finally, I changed PackedInts.copy to alloc a long[] of size min(capacity, len) so we don't over-allocate if the incoming mem is larger than it needs to be.

I think it's ready!  Thanks Adrien.
                
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch, LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand updated LUCENE-4098:
---------------------------------

    Attachment: LUCENE-4098.patch

Here is the patch for the proposed modifications. All {{Mutable}} implementations have a new efficient {{fill}} method and Packed64SingleBlock* classes also have efficient bulk get and set.

For example, the execution time of the following (unrealistic) microbenchmark is more than twice better with the patch applied on my computer thanks to the use of {{PackedInts.copy}} instead of naive copy (see {{GrowableWriter#ensureCapacity}}).

{code}
for (int k = 0; k < 50; ++k) {
    long start = System.nanoTime();
    GrowableWriter wrt = new GrowableWriter(1, 1 << 22, PackedInts.DEFAULT);
    for (int i = 0; i < 1 << 22; ++i) {
        wrt.set(i,i);
    }
    long end = System.nanoTime();
    System.out.println((end - start) / 1000000);
    long sum = 0;
    for (int i = 0; i < wrt.size(); ++i) {
        sum += wrt.get(i);
    }
    System.out.println(sum);
}
{code}

                
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Resolved] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless resolved LUCENE-4098.
----------------------------------------

       Resolution: Fixed
    Fix Version/s:     (was: 4.1)
                   5.0
                   4.0

Thanks Adrien!
                
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 4.0, 5.0
>
>         Attachments: LUCENE-4098.patch, LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand updated LUCENE-4098:
---------------------------------

    Attachment: LUCENE-4098.patch

Slightly updated patch that fixes the computation of maxValue in GrowableWriter and uses | instead of Math.max to compute the max value when performing a bulk set (since GrowableWriter only works with unsigned integers).

As a side note, the reason why I didn't write bulk get and set methods for Packed64 is that I didn't find how to do it efficiently (= with less instructions and no conditional) without writing specialized methods for every bitsPerValue.
                
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Commented] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287584#comment-13287584 ] 

Michael McCandless commented on LUCENE-4098:
--------------------------------------------

Aha, phew!  A bug in the test ... with that change the test looks like it's passing so far ... I'll beast it for a while.  Thanks Adrien.
                
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Comment Edited] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Adrien Grand (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287581#comment-13287581 ] 

Adrien Grand edited comment on LUCENE-4098 at 6/1/12 6:23 PM:
--------------------------------------------------------------

The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{code}
should be replaced with
{code}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= len);
{code}
                
      was (Author: jpountz):
    The bulk get and set are not guaranteed to return/set exactly len longs (so that they can stay at a block boundary to make the subsequent reads/writes faster). So I think
{{code}}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, len);
assertTrue(sot <= len);
{{code}}
should be replaced with
{{code}}
int got = packed1.get(start, buffer, offset, len);
assertTrue(got <= len);
int sot = packed2.set(start, buffer, offset, got);
assertTrue(sot <= len);
{{code}}
                  
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch, LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Assigned] (LUCENE-4098) Efficient bulk operations for packed integer arrays

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless reassigned LUCENE-4098:
------------------------------------------

    Assignee: Michael McCandless
    
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations, this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}} time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}}, allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org