[Impala-ASF-CR] IMPALA-11039: Fix incorrect page jumping in late materialization of Parquet

["Quanlong Huang (Code Review)" <ge...@cloudera.org>]

Hello Zoltan Borok-Nagy, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18372

to look at the new patch set (#2).

Change subject: IMPALA-11039: Fix incorrect page jumping in late materialization of Parquet
......................................................................

IMPALA-11039: Fix incorrect page jumping in late materialization of Parquet

The current calculation of LastRowIdxInCurrentPage() is incorrect. It
uses the first row index of the next candidate page instead of the next
valid page. The next candidate page could be far away from the current
page. Thus giving a number larger than the current page size. Skipping
rows in the current page could overflow the boundary due to this. This
patch fixes LastRowIdxInCurrentPage() to use the next valid page.

When skip_row_id is set (>0), the current approach of
SkipRowsInternal<false>() expects jumping to a page containing this row
and then skipping rows in that page. However, the expected row might
not be in the candidate pages. When we jump to the next candidate page,
the target row could already be skipped. In this case, we don't need to
skip rows in the current page.

Tests:
 - Add a test on alltypes_empty_pages to reveal the bug.
 - Add more batch_size values in test_page_index.
 - Pass tests/query_test/test_parquet_stats.py locally.

Change-Id: I3a783115ba8faf1a276e51087f3a70f79402c21d
---
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/parquet/parquet-column-readers.cc
M be/src/exec/parquet/parquet-column-readers.h
M be/src/exec/parquet/parquet-common.cc
M be/src/exec/parquet/parquet-common.h
M testdata/workloads/functional-query/queries/QueryTest/parquet-page-index.test
M tests/query_test/test_parquet_stats.py
7 files changed, 92 insertions(+), 28 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/18372/2
-- 
To view, visit http://gerrit.cloudera.org:8080/18372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3a783115ba8faf1a276e51087f3a70f79402c21d
Gerrit-Change-Number: 18372
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>