[jira] [Assigned] (IMPALA-15157) Parquet scanner unnecessarily decompresses a fully-skipped page at a page boundary

["Quanlong Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quanlong Huang reassigned IMPALA-15157:
---------------------------------------

    Assignee: YifanZhang

> Parquet scanner unnecessarily decompresses a fully-skipped page at a page boundary
> ----------------------------------------------------------------------------------
>
>                 Key: IMPALA-15157
>                 URL: https://issues.apache.org/jira/browse/IMPALA-15157
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: YifanZhang
>            Assignee: YifanZhang
>            Priority: Major
>             Fix For: Impala 5.0.0
>
>
> When late materialization / row-level skipping skips a set of rows that ends exactly on a Parquet page boundary, {{BaseScalarColumnReader::SkipRowsInternal()}} exits the page-skipping loop and calls {{{}ReadCurrentDataPage(){}}}, decompressing that final page just to skip all of its values. The whole page could instead be discarded without decompression. 
> *Root cause*
> The whole-page-skip loop discarded pages only while the number of rows to skip was _strictly greater_ than the values buffered in the current page:
> {code:java}
> while (num_rows > num_buffered_values_) {
>   ...
>   if (!col_chunk_reader_.SkipPageData().ok() || !AdvanceNextPageHeader()) {
>     return false;
>   }
> } {code}
> When {{num_rows == num_buffered_values_}} (the skip ends exactly at a page boundary), the loop exited and the code proceeded to decompress the current page and skip values in-page. In the boundary case where nothing remains after the current page, this path still calls {{AdvanceNextPageHeader()}} for a page header that may not exist, and needlessly decompresses a page that is going to be fully skipped.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org