You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/01/03 00:20:00 UTC
[jira] [Work logged] (HIVE-26693) HS2 can not read/write hive_catalog iceberg table created by other engines
[ https://issues.apache.org/jira/browse/HIVE-26693?focusedWorklogId=836474&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-836474 ]
ASF GitHub Bot logged work on HIVE-26693:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Jan/23 00:19
Start Date: 03/Jan/23 00:19
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on PR #3726:
URL: https://github.com/apache/hive/pull/3726#issuecomment-1369287186
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
Issue Time Tracking
-------------------
Worklog Id: (was: 836474)
Time Spent: 50m (was: 40m)
> HS2 can not read/write hive_catalog iceberg table created by other engines
> --------------------------------------------------------------------------
>
> Key: HIVE-26693
> URL: https://issues.apache.org/jira/browse/HIVE-26693
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2, StorageHandler
> Affects Versions: 4.0.0-alpha-2
> Reporter: zhangbutao
> Assignee: zhangbutao
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Step to reproduce:
> # Create hive_catalog iceberg table by Trino/Presto/Flink (Spark with _iceberg.engine.hive.enabled_ disabled)
> # show table info with hive beeline:
> {code:java}
> +----------------------------------------------------+
> | createtab_stmt |
> +----------------------------------------------------+
> | CREATE EXTERNAL TABLE `iceberg_hive`.`testtrinoice`( |
> | `id` int) |
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.mapred.FileInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.mapred.FileOutputFormat' |
> | LOCATION |
> | 'hdfs://localhost:8020/iceberg_hive.db/testtrinoice-08642c05e622415ab3e2da4b4c35224d' |
> | TBLPROPERTIES ( |
> | 'metadata_location'='hdfs://localhost:8020/iceberg_hive.db/testtrinoice-08642c05e622415ab3e2da4b4c35224d/metadata/00000-3303dd99-e4d1-4cb0-9d12-9744cbe0a1c9.metadata.json', |
> | 'table_type'='iceberg', |
> | 'transient_lastDdlTime'='1667292082') |
> +----------------------------------------------------+
> {code}
> You can see that the iceberg table created by trino has no iceberg inputformat/outputformat which is used to read/write iceberg data for HS2.
> # Query this iceberg table with HS2:
> {code:java}
> select * from iceberg_hive.testtrinoice; {code}
>
> {code:java}
> ERROR : Failed with exception java.io.IOException:java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.FileInputFormat as specified in mapredWork!
> java.io.IOException: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.FileInputFormat as specified in mapredWork!
> at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:624)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:531)
> at org.apache.hadoop.hive.ql.exec.FetchTask.executeInner(FetchTask.java:197)
> at org.apache.hadoop.hive.ql.exec.FetchTask.execute(FetchTask.java:98)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:212)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:154)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:149)
> at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:185)
> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:234)
> at org.apache.hive.service.cli.operation.SQLOperation.access$500(SQLOperation.java:88)
> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:337)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:357)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.FileInputFormat as specified in mapredWork!
> at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:233)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:374)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:306)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:562)
> ... 19 more
> Caused by: java.lang.RuntimeException: java.lang.InstantiationException
> at org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:85)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:229)
> ... 22 more
> Caused by: java.lang.InstantiationException
> at sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:83)
> ... 23 more {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)