You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org> on 2019/08/29 12:23:26 UTC

[Impala-ASF-CR] WIP: IMPALA-6433: Part 1: Extract page reading logic from ParquetColumnReader

Csaba Ringhofer has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/13329 )

Change subject: WIP: IMPALA-6433: Part 1: Extract page reading logic from ParquetColumnReader
......................................................................

WIP: IMPALA-6433: Part 1: Extract page reading logic from ParquetColumnReader

Moved some responsibilities from parquet-column-readers.cc to
a new class 'ParquetPageReader':
- reading pages from ScanRange
- decompress data if needed

The main motivation is to make the implementation of V2 data page
reading simpler by moving most parts that will differ between V1 and V2
into a class with manageable complexity.

TODOs:
- The current implementation tries to change the existing logic as
  little as possible. The interface of ParquetPageReader could be
  simplified by changing the logic a bit e.g. reorder handling of
  errors.
- The comments could be extended / polished once the interface
  is more or less final.

Testing:
- ran parquet related scanner tests

Change-Id: Ic0289394adcb97a3529313030930c9c5b85aaa12
---
M be/src/exec/parquet/CMakeLists.txt
M be/src/exec/parquet/hdfs-parquet-scanner.h
M be/src/exec/parquet/parquet-column-readers.cc
M be/src/exec/parquet/parquet-column-readers.h
A be/src/exec/parquet/parquet-page-reader.cc
A be/src/exec/parquet/parquet-page-reader.h
6 files changed, 674 insertions(+), 406 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/13329/5
-- 
To view, visit http://gerrit.cloudera.org:8080/13329
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic0289394adcb97a3529313030930c9c5b85aaa12
Gerrit-Change-Number: 13329
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>