[jira] [Commented] (IMPALA-10485) Support Iceberg field-id based column resolution in the ORC scanner

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287470#comment-17287470 ] 

ASF subversion and git services commented on IMPALA-10485:
----------------------------------------------------------

Commit 7c8e3eb4400f5924c2ed12be92899d254d5b412b in impala's branch refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=7c8e3eb ]

IMPALA-10485: part(1): make ORC column reader creation independent of schema resolution

Currently ORC schema resolution is position-based only. Then the
positional information is reused during ORC column reader creation.
This prevents adding other column resolution strategies, e.g.
column resolution by name or Iceberg field id. It also prevents
schema evolution, as table metadata and file metadata should be
in sync.

This patch makes column reader creation independent of schema
resolution. It does this by creating a mapping between slot/tuple
descriptors and ORC type ids during schema resolution and use
this mapping during column reader creation.

Now further patches just need to add support for other column
resolution strategies, the column readers will be created
accordingly.

Testing:
 * no additional tests as it is neither a bug fix nor a new feature

Change-Id: I0f7d521f9397c5188fadc7996cee0bd1650d363e
Reviewed-on: http://gerrit.cloudera.org:8080/17039
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Support Iceberg field-id based column resolution in the ORC scanner
> -------------------------------------------------------------------
>
>                 Key: IMPALA-10485
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10485
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>            Reporter: Zoltán Borók-Nagy
>            Assignee: Zoltán Borók-Nagy
>            Priority: Major
>              Labels: impala-iceberg
>
> Currently the ORC scanner only support position-based column resolution.
> We need to add Iceberg field-id based column resolution to support schema evolution.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org