You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tamas Mate (Jira)" <ji...@apache.org> on 2023/01/10 11:14:00 UTC
[jira] [Commented] (IMPALA-11834) Iceberg LOAD DATA hdfsDelete can crash JVM
[ https://issues.apache.org/jira/browse/IMPALA-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656588#comment-17656588 ]
Tamas Mate commented on IMPALA-11834:
-------------------------------------
{{hdfsConnect("default", 0)}} will create an FS object based on core-site.xml {{fs.defaultFS}} property. In case of S3 this can be a bucket, this can cause problems and fails during path checking in the hadoop library.
> Iceberg LOAD DATA hdfsDelete can crash JVM
> ------------------------------------------
>
> Key: IMPALA-11834
> URL: https://issues.apache.org/jira/browse/IMPALA-11834
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, Frontend
> Affects Versions: Impala 4.3.0
> Reporter: Tamas Mate
> Assignee: Tamas Mate
> Priority: Major
> Labels: iceberg
>
> It is probably caused by the incorrectly initialising the {{hdfsFS}} object before executing the delete.
> {code:none}
> hdfsDelete(path=s3a://bucket-external/path/to/table/icebergtable1/.tmp_8cd3828f-27dd-4d49-be7a-4453207ecfb7, recursive=1): FileSystem#delete error:IllegalArgumentException: Wrong FS s3a://bucket-external//path/to/table/icebergtable1/.tmp_8cd3828f-27dd-4d49-be7a-4453207ecfb7 -expected s3a://bucket-managed at org.apache.hadoop.fs.s3native.S3xLoginHelper.checkPath(S3xLoginHelper.java:224) at org.apache.hadoop.fs.s3a.S3AFileSystem.checkPath(S3AFileSystem.java:1375) at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:647) at org.apache.hadoop.fs.s3a.S3AFileSystem.makeQualified(S3AFileSystem.java:1337) at org.apache.hadoop.fs.s3a.S3AFileSystem.qualify(S3AFileSystem.java:1363) at org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:3132) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org