[jira] [Created] (KUDU-1465) Large allocations for scanner result buffers harm allocator thread caching

["Todd Lipcon (JIRA)" <ji...@apache.org>]

Todd Lipcon created KUDU-1465:
---------------------------------

             Summary: Large allocations for scanner result buffers harm allocator thread caching
                 Key: KUDU-1465
                 URL: https://issues.apache.org/jira/browse/KUDU-1465
             Project: Kudu
          Issue Type: Bug
          Components: perf
    Affects Versions: 0.8.0
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon


I was looking at the performance of a random-read stress test on a 70 node cluster and found that threads were often spending time in allocator contention, particularly when deallocating RpcSidecar objects. After a bit of analysis, I determined this is because we always preallocate buffers of 1MB (the default batch size) even if the response is only going to be a single row.

As a simple test, I used the set_flag command to drop the default batch size to 4KB, and the read throughput (reads/second) increased substantially.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)