You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Zoltán Borók-Nagy (Jira)" <ji...@apache.org> on 2022/12/07 12:57:00 UTC
[jira] [Created] (IMPALA-11780) Wrong FILE__POSITION values for multi row group Parquet files when page filtering is used
Zoltán Borók-Nagy created IMPALA-11780:
------------------------------------------
Summary: Wrong FILE__POSITION values for multi row group Parquet files when page filtering is used
Key: IMPALA-11780
URL: https://issues.apache.org/jira/browse/IMPALA-11780
Project: IMPALA
Issue Type: Bug
Components: Backend
Reporter: Zoltán Borók-Nagy
Impala generates wrong values for the FILE__POSITION column when the Parquet file contains multiple row groups and page filtering is being used.
We are using the value of 'current_row_' to populate the file position slot:
https://github.com/apache/impala/blob/8dbc8b733d3712035f8615f477f74c66ec9d63ea/be/src/exec/parquet/parquet-column-readers.cc#L906
Problem is that 'current_row_' denotes the index of the row within the row group and not withing the file. We cannot change 'current_row_' as page filtering depends on its value, and the page index also uses the row group-based indexes of the rows, not the file indexes.
The value of FILE__POSITION is critical for Iceberg V2 tables as position delete files store file positions of the deleted rows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)