You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tamas Mate (Jira)" <ji...@apache.org> on 2023/01/25 14:27:00 UTC
[jira] [Created] (IMPALA-11864) LOAD DATA should not try to load hidden files for Iceberg tables
Tamas Mate created IMPALA-11864:
-----------------------------------
Summary: LOAD DATA should not try to load hidden files for Iceberg tables
Key: IMPALA-11864
URL: https://issues.apache.org/jira/browse/IMPALA-11864
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 4.3.0
Reporter: Riddhi jain
Assignee: Tamas Mate
Fix For: Impala 4.3.0
Steps to reproduce: # Create an iceberg table
# Try to load data into the table from folder which has both hidden files as well *.parq file
Example queries ran:
{code:java}
CREATE TABLE iceberg_partitioned_table1 (id int,
bool_col boolean,
timestamp_col timestamp)
stored as iceberg;{code}
{code:java}
LOAD DATA INPATH 's3a://dwx-testdata/impala/sql_test/tests/load_data_inpath/runtime_data/0690a6fa9bfb11ed920c164053429bec/load_data_test/A/impala_data/impala_alltypessmall_data/alltypessmall_parquet_iceberg/year=2009/month=1/' OVERWRITE INTO TABLE iceberg_partitioned_table1;{code}
It is trying to load hidden file instead of ignoring it and hence throw error saying:
{code:java}
AnalysisException: INPATH contains unsupported LOAD format, file '
s3a://dwx-testdata/impala/sql_test/tests/load_data_inpath/runtime_data/0690a6fa9bfb11ed920c164053429bec/load_data_test/A/impala_data/impala_alltypessmall_data/alltypessmall_parquet_iceberg/year=2009/month=1/.hiddenfileforloaddatatest
' has 'This' magic string. {code}
However when there are no hidden files in the folder it loads successfully.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org