You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Gabor Kaszab (Jira)" <ji...@apache.org> on 2022/06/16 09:58:00 UTC

[jira] [Created] (IMPALA-11364) Use Late Materialization when structs are in the select list

Gabor Kaszab created IMPALA-11364:
-------------------------------------

             Summary: Use Late Materialization when structs are in the select list
                 Key: IMPALA-11364
                 URL: https://issues.apache.org/jira/browse/IMPALA-11364
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend
    Affects Versions: Impala 4.1.0
            Reporter: Gabor Kaszab


IMPALA-9496 introduced querying structs in the select list even for Parquet tables. However, it has a restriction that in this case late materialization (LM) can't be used.

The reason is that LM expects the column readers to be used through the batched reading interface, however, currently struct column readers use the non-batched reading interface of its children. As a result after reading, the column readers are not in a state as SkipRows() of LM expects and then results in a query failure because it's not able to skip the rows for non-filter readers.


Once IMPALA-11363 is implemented and the struct will also use the ReadValueBatch() interface of its children then late materialization could be turned on even if structs are in the select list.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org