You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/09/10 03:20:02 UTC
[GitHub] [iceberg] shengkui edited a comment on issue #3079: java.io.IOException: Mkdirs failed to create file:/user/hive/warehouse/bench/metadata
shengkui edited a comment on issue #3079:
URL: https://github.com/apache/iceberg/issues/3079#issuecomment-916599585
@openinx I've tried following the document you provided, and failed again. Following is picked from the log of flink sql-client:
```
Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Got exception: org.apache.hadoop.fs.s3a.AWSBadRequestException PUT 0-byte object on lsk.db/: com.amazonaws.services.s3.model.AmazonS3Exception: null (Service: Amazon S3; Status Code: 400; Error Code: InvalidRequest; Request ID: tx000000000000000265ae0-006139a08d-5efa-default; S3 Extended Request ID: 5efa-default-default), S3 Extended Request ID: 5efa-default-default:InvalidRequest: null (Service: Amazon S3; Status Code: 400; Error Code: InvalidRequest; Request ID: tx000000000000000265ae0-006139a08d-5efa-default; S3 Extended Request ID: 5efa-default-default)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_database_result$create_database_resultStandardScheme.read(ThriftHiveMetastore.java:39343) ~[flink-sql-connector-hive-3.1.2_2.12-1.12.5.jar:1.12.5]
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_database_result$create_database_resultStandardScheme.read(ThriftHiveMetastore.java:39311) ~[flink-sql-connector-hive-3.1.2_2.12-1.12.5.jar:1.12.5]
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_database_result.read(ThriftHiveMetastore.java:39245) ~[flink-sql-connector-hive-3.1.2_2.12-1.12.5.jar:1.12.5]
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) ~[flink-sql-connector-hive-3.1.2_2.12-1.12.5.jar:1.12.5]
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_database(ThriftHiveMetastore.java:1106) ~[flink-sql-connector-hive-3.1.2_2.12-1.12.5.jar:1.12.5]
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_database(ThriftHiveMetastore.java:1093) ~[flink-sql-connector-hive-3.1.2_2.12-1.12.5.jar:1.12.5]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:809) ~[flink-sql-connector-hive-3.1.2_2.12-1.12.5.jar:1.12.5]
at org.apache.iceberg.hive.HiveCatalog.lambda$createNamespace$7(HiveCatalog.java:243) ~[iceberg-flink-runtime-5f90476.jar:?]
at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:51) ~[iceberg-flink-runtime-5f90476.jar:?]
at org.apache.iceberg.hive.CachedClientPool.run(CachedClientPool.java:76) ~[iceberg-flink-runtime-5f90476.jar:?]
at org.apache.iceberg.hive.HiveCatalog.createNamespace(HiveCatalog.java:242) ~[iceberg-flink-runtime-5f90476.jar:?]
at org.apache.iceberg.flink.FlinkCatalog.createDatabase(FlinkCatalog.java:203) ~[iceberg-flink-runtime-5f90476.jar:?]
at org.apache.iceberg.flink.FlinkCatalog.createDatabase(FlinkCatalog.java:196) ~[iceberg-flink-runtime-5f90476.jar:?]
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeOperation(TableEnvironmentImpl.java:968) ~[flink-table_2.12-1.12.5.jar:1.12.5]
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:666) ~[flink-table_2.12-1.12.5.jar:1.12.5]
at org.apache.flink.table.client.gateway.local.LocalExecutor.lambda$executeSql$1(LocalExecutor.java:315) ~[flink-sql-client_2.12-1.12.5.jar:1.12.5]
at org.apache.flink.table.client.gateway.local.ExecutionContext.wrapClassLoader(ExecutionContext.java:256) ~[flink-sql-client_2.12-1.12.5.jar:1.12.5]
at org.apache.flink.table.client.gateway.local.LocalExecutor.executeSql(LocalExecutor.java:315) ~[flink-sql-client_2.12-1.12.5.jar:1.12.5]
... 8 more
```
I've tried hadoop catalog with S3(without hive-metastore), it works well. But there is someone said, iceberg need hive-metastore for S3 storage(#1468). I know iceberg have implemented Glue Catalog, but it for AWS. Is any solution to use S3 without hive-metastore?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org