You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by John Morris <jm...@maprtech.com> on 2014/02/25 00:46:49 UTC

Re: Review Request 17165: Extend netty memory manager (jira DRILL-336)

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17165/
-----------------------------------------------------------

(Updated Feb. 24, 2014, 11:46 p.m.)


Review request for drill.


Changes
-------

Removed the "feature" of throwing an exception for large memory copy on resize. While we would like to avoid large copies in Drill, internal Netty routines were unwittingly raising the exception.


Repository: drill-git


Description
-------

Modified the netty direct memory manager to:
   1) Efficiently reduce ("trim") the size of a memory buffer, releasing the extra memory back to the memory manager.
   2) Opportunistically allocate larger buffers if it can be done efficiently.   
   3) Raise a warning exception if resizing a buffer copies more than 1 page of data.
   4) Added Javadocs and some in-code comments.
   5) Created a Junit test to verify basic functionality.

The "trim" and allocation changes are primarily in PoolChunkL,
which breaks a "chunk" into runs of pages using a buddy system.


Diffs (updated)
-----

  exec/bufferl/src/main/java/io/netty/buffer/PoolArenaL.java cd9ec3f 
  exec/bufferl/src/main/java/io/netty/buffer/PoolChunkL.java 91a13a9 
  exec/bufferl/src/main/java/io/netty/buffer/PoolChunkListL.java cfa6fc6 
  exec/bufferl/src/main/java/io/netty/buffer/PoolSubpageL.java 2956dd6 
  exec/bufferl/src/main/java/io/netty/buffer/PooledByteBufAllocatorL.java f3b3afb 
  exec/bufferl/src/main/java/io/netty/buffer/PooledByteBufL.java 595cbc9 
  exec/bufferl/src/main/java/io/netty/buffer/PooledUnsafeDirectByteBufL.java efc13b5 
  exec/bufferl/src/main/java/io/netty/buffer/package-info.java PRE-CREATION 
  exec/bufferl/src/test/java/io/netty/buffer/TestPoolChunkTrim.java PRE-CREATION 

Diff: https://reviews.apache.org/r/17165/diff/


Testing
-------

Created a basic Junit test to verify the functionality as well as some of the boundary conditions.


Thanks,

John Morris