[GitHub] [carbondata] jackylk edited a comment on issue #3638: [CARBONDATA-3731] Avoid data copy in Writer

[GitBox <gi...@apache.org>]

jackylk edited a comment on issue #3638: [CARBONDATA-3731] Avoid data copy in Writer
URL: https://github.com/apache/carbondata/pull/3638#issuecomment-594351151
 
 
   @ajantha-bhat 
   In original code, 5 copies are:
   
   ```
   TablePage.convertToColumnarAndAddToPages:
   	byte[] valueWithLength = addShortLengthToByteArray((byte[]) noDictAndComplex[i]);  // alloc and copy
   	noDictDimensionPages[i].putData(rowId, valueWithLength); // safe copy to unsafe
   
   HighCardDictDimensionIndexCodec.encodeIndexStorage:
   	byte[][] data = input.getByteArrayPage();  // copy unsafe to safe
   	byte[] flattened = ByteUtil.flatten(indexStorage.getDataPage());   // alloc and copy
   
   SnappyCompressor.compressByte:
   	Snappy.rawCompress(unCompInput, unCompInput.length); // one copy and one compression write inside, shoud use direct buffer which has only one compression write
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services