incubator-impala git commit: IMPALA-5798: ASAN use-after-poison in Parquet decoder

[ta...@apache.org]

Repository: incubator-impala
Updated Branches:
  refs/heads/master d8bc570b6 -> cb645b1bc


IMPALA-5798: ASAN use-after-poison in Parquet decoder

In ParquetLevelDecoder::Init() for RLE encoding, we read the metadata
size and advance the data buffer past it. If the metadata size is
corrupted, it can cause us to incorrectly read past the end of the
buffer.

This patch checks that the metadata size is less than the total size
of the buffer, and returns an error if it isn't.

Testing:
- Ran test_scanners_fuzz.py under ASAN 500 times without hitting the
  use-after-poison (previously it would usually hit in < 100 runs).

Change-Id: I3f3d0d998f7581c7c935d98fde886f145efd61a8
Reviewed-on: http://gerrit.cloudera.org:8080/7769
Reviewed-by: Alex Behm <al...@cloudera.com>
Reviewed-by: Matthew Jacobs <mj...@cloudera.com>
Tested-by: Impala Public Jenkins


Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/cb645b1b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/cb645b1b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/cb645b1b

Branch: refs/heads/master
Commit: cb645b1bcb3e18123bee0916da9fbcf7ff55050d
Parents: d8bc570
Author: Thomas Tauber-Marshall <tm...@cloudera.com>
Authored: Mon Aug 21 17:58:09 2017 -0700
Committer: Impala Public Jenkins <im...@gerrit.cloudera.org>
Committed: Wed Aug 23 20:15:37 2017 +0000

----------------------------------------------------------------------
 be/src/exec/parquet-column-readers.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/cb645b1b/be/src/exec/parquet-column-readers.cc
----------------------------------------------------------------------
diff --git a/be/src/exec/parquet-column-readers.cc b/be/src/exec/parquet-column-readers.cc
index 9f00762..d55b545 100644
--- a/be/src/exec/parquet-column-readers.cc
+++ b/be/src/exec/parquet-column-readers.cc
@@ -90,7 +90,7 @@ Status ParquetLevelDecoder::Init(const string& filename,
       if (!ReadWriteUtil::Read(data, data_size, &num_bytes, &status)) {
         return status;
       }
-      if (num_bytes < 0) {
+      if (num_bytes < 0 || num_bytes > *data_size) {
         return Status(TErrorCode::PARQUET_CORRUPT_RLE_BYTES, filename, num_bytes);
       }
       int bit_width = BitUtil::Log2Ceiling64(max_level + 1);