[kudu] branch master updated: bitshuffle: stop using uninitialized data as padding

[ad...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git


The following commit(s) were added to refs/heads/master by this push:
     new 882ae58  bitshuffle: stop using uninitialized data as padding
882ae58 is described below

commit 882ae5861ef5de451b39568547f7c77b6909ec05
Author: Adar Dembo <ad...@cloudera.com>
AuthorDate: Fri Apr 3 16:36:56 2020 -0700

    bitshuffle: stop using uninitialized data as padding
    
    The resize() incorrectly accounted for a block header that was never actually
    written to data_. The result was that added padding was actually kHeaderSize
    bytes "to the right", and the call to compress_lz4() read uninitialized data
    from this part of data_ rather than the added padding.
    
    What's the effect? Up to padding_bytes of uninitialized data gets
    bitshuffled, compressed, and written to the block. At read time, it is
    decompressed, debitshuffled, but ultimately ignored, as it was expected to
    be just padded zeroes.
    
    I observed this in cfile-test's TestMetadata built with MSAN
    instrumentation, but oddly enough not in any other test in cfile-test, even
    though others use bitshuffle and padding.
    
    Change-Id: I25eba027ba356774173b2313c68436d7baddaddc
    Reviewed-on: http://gerrit.cloudera.org:8080/15647
    Reviewed-by: Andrew Wong <aw...@cloudera.com>
    Tested-by: Kudu Jenkins
---
 src/kudu/cfile/bshuf_block.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/kudu/cfile/bshuf_block.h b/src/kudu/cfile/bshuf_block.h
index eaeb6e0..52e8639 100644
--- a/src/kudu/cfile/bshuf_block.h
+++ b/src/kudu/cfile/bshuf_block.h
@@ -174,7 +174,7 @@ class BShufBlockBuilder final : public BlockBuilder {
   }
 
   Slice Finish(rowid_t ordinal_pos, int final_size_of_type) {
-    data_.resize(kHeaderSize + final_size_of_type * count_);
+    data_.resize(final_size_of_type * count_);
 
     // Do padding so that the input num of element is multiple of 8.
     int num_elems_after_padding = KUDU_ALIGN_UP(count_, 8);