[jira] [Created] (IMPALA-11618) Impala fails to read Iceberg snapshot tables created by Spark using call spark_catalog.system.snapshot()

["David Rorke (Jira)" <ji...@apache.org>]

David Rorke created IMPALA-11618:
------------------------------------

             Summary: Impala fails to read Iceberg snapshot tables created by Spark using call spark_catalog.system.snapshot() 
                 Key: IMPALA-11618
                 URL: https://issues.apache.org/jira/browse/IMPALA-11618
             Project: IMPALA
          Issue Type: Bug
          Components: Catalog
    Affects Versions: Impala 4.1.0
            Reporter: David Rorke


I get an exception from catalog when Impala tries to read an Iceberg "snapshot" table created by a spark job using this API:  [https://iceberg.apache.org/docs/latest/spark-procedures/#snapshot]


{noformat}
Query submitted at: 2022-09-27 09:43:15 (Coordinator: http://coordinator-0.coordinator-int.impala-1662757723-t7hk.svc.cluster.local:25000)
ERROR: AnalysisException: Failed to load metadata for table: 'customer_iceberg_snapshot'
CAUSED BY: TableLoadingException: Could not load table tpcds_3000_string_parquet_external.customer_iceberg_snapshot from catalog
CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[IcebergTableLoadingException: Error loading metadata for Iceberg table s3a://drorke-dwxperf2/warehouse/tablespace/external/hive/tpcds_3000_string_parquet_external.db/customer_iceberg_snapshot
CAUSED BY: RuntimeException: FileSystem returned an unexpected path s3a://drorke-dwxperf2/warehouse/tablespace/external/hive/tpcds_3000_string_parquet_external.db/customer/0b443b4d91612be0-af2edb1b00000000_1194661597_data.0.parq for a file within s3a://drorke-dwxperf2/warehouse/tablespace/external/hive/tpcds_3000_string_parquet_external.db/customer_iceberg_snapshot]), lookup_status:OK)
Looks like Impala gets confused in FileSystemUtils.relativizePath() because the final directory component in the new snapshot table name (customer_iceberg_snapshot) is different from the corresponding path component in the file.
{noformat}

It looks like Impala gets confused in FileSystemUtils.relativizePath() because the final directory component in the new snapshot table name (customer_iceberg_snapshot) is different from the corresponding path component in the file.





--
This message was sent by Atlassian Jira
(v8.20.10#820010)