You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/09/18 07:56:34 UTC
[GitHub] [iceberg] zaq1xsw2cde3vf opened a new issue #3147: beeline (HDP hive 3.1) read iceberg table NoClassDefFoundError
zaq1xsw2cde3vf opened a new issue #3147:
URL: https://github.com/apache/iceberg/issues/3147
where use beeline read iceberg table on HDFS, cast error
`java.lang.NoClassDefFoundError: org/apache/iceberg/shaded/com/github/benmanes/caffeine/cache/RemovalCause$1 (state=,code=0)`
ENV:
- HDP hive 3.1
- spark 3.1.2
- iceberg-hive-runtime-0.12.0
OP:
1. spark generate a iceberg table and insert data on HDFS. catalog = hadoop. OK
2. read table by spark code. OK
3. create external table on hive. OK. It can auto load schema from iceberg table when check by describe extended tablename.
`
hive> add jar iceberg-hive-runtime.jar;
hive> SET iceberg.catalog.hadoop_prod.type=hadoop;
hive> SET iceberg.catalog.hadoop_prod.warehouse=hdfs://path/to/dir;
hive> CREATE EXTERNAL TABLE iceberg.table1 STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' TBLPROPERTIES ('iceberg.catalog'='hadoop_prod');
`
4. run. error ->
`
0: jdbc:hive2://cs-nn-02.rdsp.com:2181,cs-nn-> select * from table1;
INFO : Compiling command(queryId=hive_20210918153256_50630344-2cb5-4e8a-8d79-77534a837288): select * from table1
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:table1.c1, type:int, comment:null), FieldSchema(name:table1.c2, type:string, comment:null), FieldSchema(name:table1.c3, type:string, comment:null)], properties:null)
INFO : Completed compiling command(queryId=hive_20210918153256_50630344-2cb5-4e8a-8d79-77534a837288); Time taken: 3.899 seconds
INFO : Executing command(queryId=hive_20210918153256_50630344-2cb5-4e8a-8d79-77534a837288): select * from table1
INFO : Completed executing command(queryId=hive_20210918153256_50630344-2cb5-4e8a-8d79-77534a837288); Time taken: 0.014 seconds
INFO : OK
Error: java.lang.NoClassDefFoundError: org/apache/iceberg/shaded/com/github/benmanes/caffeine/cache/RemovalCause$1 (state=,code=0)
`
Thanks very much!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on issue #3147: beeline (HDP hive 3.1) read iceberg table NoClassDefFoundError
Posted by GitBox <gi...@apache.org>.
pvary commented on issue #3147:
URL: https://github.com/apache/iceberg/issues/3147#issuecomment-922648255
There is a different version of Caffeine included in Hive and Iceberg.
We fixed it in the Hive code:
https://issues.apache.org/jira/browse/HIVE-25201
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary edited a comment on issue #3147: beeline (HDP hive 3.1) read iceberg table NoClassDefFoundError
Posted by GitBox <gi...@apache.org>.
pvary edited a comment on issue #3147:
URL: https://github.com/apache/iceberg/issues/3147#issuecomment-922648255
There is a different version of Caffeine included in HDP Hive and Iceberg.
We fixed it in the Hive code:
https://issues.apache.org/jira/browse/HIVE-25201
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zaq1xsw2cde3vf commented on issue #3147: beeline (HDP hive 3.1) read iceberg table NoClassDefFoundError
Posted by GitBox <gi...@apache.org>.
zaq1xsw2cde3vf commented on issue #3147:
URL: https://github.com/apache/iceberg/issues/3147#issuecomment-924520031
> There is a different version of Caffeine included in HDP Hive and Iceberg.
>
> We fixed it in the Hive code:
> https://issues.apache.org/jira/browse/HIVE-25201
If I can't change any thing on hive, what things I can do to run iceberg 0.12 with Hive 3.1 ? Thanks for any advices!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zaq1xsw2cde3vf closed issue #3147: beeline (HDP hive 3.1) read iceberg table NoClassDefFoundError
Posted by GitBox <gi...@apache.org>.
zaq1xsw2cde3vf closed issue #3147:
URL: https://github.com/apache/iceberg/issues/3147
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on issue #3147: beeline (HDP hive 3.1) read iceberg table NoClassDefFoundError
Posted by GitBox <gi...@apache.org>.
pvary commented on issue #3147:
URL: https://github.com/apache/iceberg/issues/3147#issuecomment-924584934
> If I can't change any thing on hive, what things I can do to run iceberg 0.12 with Hive 3.1 ? Thanks for any advices!
You can use the Apache Hive 3.1.2 version, or you can ask your vendor about the possibilities. I know that Cloudera is working on integrating Iceberg to the CDP stack, which might be useful for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zaq1xsw2cde3vf commented on issue #3147: beeline (HDP hive 3.1) read iceberg table NoClassDefFoundError
Posted by GitBox <gi...@apache.org>.
zaq1xsw2cde3vf commented on issue #3147:
URL: https://github.com/apache/iceberg/issues/3147#issuecomment-924595298
> > If I can't change any thing on hive, what things I can do to run iceberg 0.12 with Hive 3.1 ? Thanks for any advices!
>
> You can use the Apache Hive 3.1.2 version, or you can ask your vendor about the possibilities. I know that Cloudera is working on integrating Iceberg to the CDP stack, which might be useful for you.
Thanks for your help!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org