You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "shyam narayan singh (JIRA)" <ji...@apache.org> on 2018/12/10 14:33:00 UTC

[jira] [Created] (ARROW-3987) Compact ArrowBuf to reduce its heap footprint

shyam narayan singh created ARROW-3987:
------------------------------------------

             Summary: Compact ArrowBuf to reduce its heap footprint
                 Key: ARROW-3987
                 URL: https://issues.apache.org/jira/browse/ARROW-3987
             Project: Apache Arrow
          Issue Type: Task
            Reporter: shyam narayan singh


This is to record numbers for a test that has been done internally to compact ArrowBuf heap foot print by moving variables (udle, refcnt, isempty) to buffer ledger. It also removes the debug fields in ledger (buffers, historical log) to a different class.

Running a test that does 20000 allocations of arrow bufs and with ravindra's ongoing fix that does the slicing of arrow bufs, below are the results (without/with above fix)
Without above fix :
 
       Total Bytes: 13,028,109
       Total Classes: 2,367
       Total Instances: 193,471
       Classloaders: 77
       GC Roots: 1,641
       Number of Objects Pending for Finalization: 0
  
With above fix :
 
       Total Bytes: 12,700,115
       Total Classes: 2,373
       Total Instances: 193,635
       Classloaders: 82
       GC Roots: 1,642
       Number of Objects Pending for Finalization: 0
 
 
ArrowBuf size decreased from 109 bytes to 88 bytes and BufferLedger size increased from 80 bytes to 89 bytes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)