You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2018/11/20 07:39:40 UTC

[Impala-ASF-CR] IMPALA-7869: break up parquet-column-readers.cc

Tim Armstrong has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/11949 )

Change subject: IMPALA-7869: break up parquet-column-readers.cc
......................................................................

IMPALA-7869: break up parquet-column-readers.cc

Move CollectionColumnReader and ParquetLevelDecoder into separate files.

Switch BOOLEAN decoding to use composition instead of inheritance. This
lets the boolean decoding use the faster batched implementations in
ScalarColumnReader and avoids some confusing aspects of the class
hierarchy, like the ReadValueBatch() implementation on the base class
that was shared between BoolColumnReader and CollectionColumnReader.

Improve compile times by instantiating BitPacking templates in a
separate file (this looks to give a 30s+ speedup for
compiling parquet-column-readers.cc).

Testing:
Ran exhaustive tests.

Change-Id: I0efd5c50b781fe9e3c022b33c66c06cfb529c0b8
---
M be/src/benchmarks/bit-packing-benchmark.cc
M be/src/exec/CMakeLists.txt
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
A be/src/exec/parquet-bool-decoder.cc
A be/src/exec/parquet-bool-decoder.h
A be/src/exec/parquet-collection-column-reader.cc
A be/src/exec/parquet-collection-column-reader.h
M be/src/exec/parquet-column-readers.cc
M be/src/exec/parquet-column-readers.h
M be/src/exec/parquet-common.h
A be/src/exec/parquet-level-decoder.cc
A be/src/exec/parquet-level-decoder.h
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
A be/src/util/bit-packing.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils.inline.h
M be/src/util/dict-test.cc
M be/src/util/rle-test.cc
M common/thrift/generate_error_codes.py
M testdata/workloads/functional-query/queries/QueryTest/parquet-num-values-def-levels-mismatch.test
23 files changed, 983 insertions(+), 683 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/11949/7
-- 
To view, visit http://gerrit.cloudera.org:8080/11949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0efd5c50b781fe9e3c022b33c66c06cfb529c0b8
Gerrit-Change-Number: 11949
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>