[jira] [Resolved] (IMPALA-11339) Implement LOAD DATA INPATH for Iceberg tables

["Tamas Mate (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tamas Mate resolved IMPALA-11339.
---------------------------------
     Fix Version/s: Impala 4.3.0
    Target Version: Impala 4.3.0
        Resolution: Implemented

> Implement LOAD DATA INPATH for Iceberg tables
> ---------------------------------------------
>
>                 Key: IMPALA-11339
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11339
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>            Reporter: Zoltán Borók-Nagy
>            Assignee: Tamas Mate
>            Priority: Major
>              Labels: impala-iceberg
>             Fix For: Impala 4.3.0
>
>
> Currently Impala doesn't support LOAD DATA statements for Iceberg tables.
> Some user workflows still use this statement, so it would be nice to implement it in some way.
> The parameter to LOAD DATA can be a directory or a single file.
> A possible solution would be to
>  # Create an external table
>  ## If the parameter is a single file, then we can use IMPALA-10934 to define an external table on this single file
>  ## If the parameter is a directory, then we need to create an external table using the directory as table location. To get the table schema we could use CREATE TABLE LIKE PARQUET/ORC
>  # run an {{insert into iceberg_table select * from tmp_table}}
>  # drop the tmp table (not sure if we want to keep or remove the original files)
> It does some copying, but probably this would be the safest solution.
> Users might specify the partition columns in the [PARTITION (partcol1=val1, partcol2=val2 ...)] clause. In this case the data files don't necessarily contain the partition values, i.e. we need to create the tmp table with proper partitioning.
> It's possible to create child queries for a single statement, see https://github.com/apache/impala/blob/master/be/src/service/child-query.h
> Currently only COMPUTE STATS uses this. They are probably executed in parallel, but in this task we need to execute the above statements sequentially.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org