You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/10/27 08:00:11 UTC
[GitHub] [iceberg] 1272870698 edited a comment on issue #1668: Do we need to provide a more detailed document for HIVE ?
1272870698 edited a comment on issue #1668:
URL: https://github.com/apache/iceberg/issues/1668#issuecomment-717058924
I tried to create external table (which was created in hive catalog by flink sql client) in hive sql client.
```sql
add jar iceberg-hive-runtime-apache-iceberg-0.9.0-rc4-238-g779dafd.dirty.jar;
set iceberg.mr.catalog=hive;
set hive.metastore.uris=thrift://....;
CREATE EXTERNAL TABLE iceberg_db_01.iceberg_001(
userid int,
f_random_str string
)
STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
LOCATION 'hdfs://nameservice1/user/hive/warehouse/iceberg_db_01.db/iceberg_001/';
```
The error message is here:
```
20/10/27 15:15:28 INFO ql.Driver: Compiling command(queryId=root_20201027151528_7ddc1a58-4c8a-44ef-9ac2-55ec510f2e7d): CREATE EXTERNAL TABLE iceberg_db_01.iceberg_001(
userid int,
f_random_str string
)
STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
LOCATION 'hdfs://nameservice1/user/hive/warehouse/iceberg_db_01.db/iceberg_001/'
20/10/27 15:15:28 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
20/10/27 15:15:28 INFO parse.SemanticAnalyzer: Creating table iceberg_db_01.iceberg_001 position=22
20/10/27 15:15:28 INFO ql.Driver: Semantic Analysis Completed
20/10/27 15:15:28 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
20/10/27 15:15:28 INFO ql.Driver: Completed compiling command(queryId=root_20201027151528_7ddc1a58-4c8a-44ef-9ac2-55ec510f2e7d); Time taken: 0.076 seconds
20/10/27 15:15:28 INFO ql.Driver: Executing command(queryId=root_20201027151528_7ddc1a58-4c8a-44ef-9ac2-55ec510f2e7d): CREATE EXTERNAL TABLE iceberg_db_01.iceberg_001(
userid int,
f_random_str string
)
STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
LOCATION 'hdfs://nameservice1/user/hive/warehouse/iceberg_db_01.db/iceberg_001/'
20/10/27 15:15:28 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
20/10/27 15:15:28 INFO plan.CreateTableDesc: Use StorageHandler-supplied org.apache.iceberg.mr.hive.HiveIcebergSerDe for table iceberg_001
20/10/27 15:15:28 INFO exec.DDLTask: creating table iceberg_db_01.iceberg_001 on hdfs://nameservice1/user/hive/warehouse/iceberg_db_01.db/iceberg_001
20/10/27 15:15:28 INFO mr.Catalogs: Loaded Hive Metastore catalog HiveCatalog{name=hive, uri=thrift://...:9083}
20/10/27 15:15:28 INFO iceberg.BaseMetastoreTableOperations: Refreshing table metadata from new version: hdfs://nameservice1/user/hive/warehouse/iceberg_db_01.db/iceberg_001/metadata/00008-6e9a36e4-7dcb-4dfb-8b9c-74bcf5080e8a.metadata.json
20/10/27 15:15:28 INFO iceberg.BaseMetastoreCatalog: Table loaded by catalog: hive.iceberg_db_01.iceberg_001
20/10/27 15:15:28 INFO mr.Catalogs: Loaded Hive Metastore catalog HiveCatalog{name=hive, uri=thrift://...:9083}
20/10/27 15:15:28 INFO iceberg.BaseMetastoreTableOperations: Refreshing table metadata from new version: hdfs://nameservice1/user/hive/warehouse/iceberg_db_01.db/iceberg_001/metadata/00008-6e9a36e4-7dcb-4dfb-8b9c-74bcf5080e8a.metadata.json
20/10/27 15:15:28 INFO iceberg.BaseMetastoreCatalog: Table loaded by catalog: hive.iceberg_db_01.iceberg_001
20/10/27 15:15:28 ERROR exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException: Please provide a table schema
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:868)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:873)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4291)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:343)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2200)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1843)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1563)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1339)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1328)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:409)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:836)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:772)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:699)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
Caused by: java.lang.NullPointerException: Please provide a table schema
at org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkNotNull(Preconditions.java:897)
at org.apache.iceberg.mr.hive.HiveIcebergMetaHook.preCreateTable(HiveIcebergMetaHook.java:93)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:822)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:813)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:154)
at com.sun.proxy.$Proxy34.createTable(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2562)
at com.sun.proxy.$Proxy34.createTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:858)
... 22 more
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException: Please provide a table schema
20/10/27 15:15:28 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException: Please provide a table schema
20/10/27 15:15:28 INFO ql.Driver: Completed executing command(queryId=root_20201027151528_7ddc1a58-4c8a-44ef-9ac2-55ec510f2e7d); Time taken: 0.142 seconds
20/10/27 15:15:28 INFO conf.HiveConf: Using the default value passed in for log id: d1298959-7aa5-4ecb-883e-fc302e1714c3
20/10/27 15:15:28 INFO session.SessionState: Resetting thread name to main
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org