You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/11/24 05:14:57 UTC
[GitHub] [iceberg] kowshikdremio opened a new issue, #6263: Unable to create Iceberg Table using HIVE
kowshikdremio opened a new issue, #6263:
URL: https://github.com/apache/iceberg/issues/6263
### Apache Iceberg version
_No response_
### Query engine
Hive
### Please describe the bug 🐞
### Environment:
**HIVE:** I am running HIVE with Local storage ( Not HADOOP). Regular HIVE tables are getting created. MariaDB is holding HIVE metastore. Please view the hive-site.xml given below
**Java:** OpenJDK Version 1.11
**Iceberg:** Cloned the latest from github and build using gradle script
### Ran the Following commands on HIVE command line:
add jar /home/ec2-user/iceberg/hive-runtime/build/libs/iceberg-hive-runtime-1.2.0-SNAPSHOT.jar;
add jar /home/ec2-user/iceberg/core/build/libs/iceberg-core-1.2.0-SNAPSHOT.jar;
add jar /home/ec2-user/iceberg/mr/build/libs/iceberg-mr-1.2.0-SNAPSHOT.jar;
### Error Description:
hive> create table test (name String) stored by 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler';
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: MetaException(message:java.lang.NoClassDefFoundError Could not initialize class org.apache.iceberg.mr.hive.serde.objectinspector.IcebergObjectInspector)
Ideally as per document, we need to add only iceberg-hive-runtime-1.2.0-SNAPSHOT.jar. I did add the other two jars as well and it made me overcome some other class not found error. However, stuck with the above error now.
Please let me know what other jars I need to add to HIVE to make this work.
Regards,
Kowshik
### Configuration of hive-site.xml:
`<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true</value>
<description>metadata is stored in a MySQL server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>MySQL JDBC driver class</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hiveuser</value>
<description>user name for connecting to mysql server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>???????</value>
<description>hivepassword for connecting to mysql server</description>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>file:///home/ec2-user/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
<property>
<name>fs.default.name</name>
<value>file:///home/ec2-user/warehouse</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://localhost:9083</value>
<description>Thrift URI for the remote metastore.</description>
</property>
<property>
<name>hive.server2.enable.doAs</name>
<value>false</value>
</property>
<property>
<name>iceberg.engine.hive.enabled</name>
<value>true</value>
</property>
</configuration>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kowshikdremio commented on issue #6263: Unable to create Iceberg Table using HIVE
Posted by GitBox <gi...@apache.org>.
kowshikdremio commented on issue #6263:
URL: https://github.com/apache/iceberg/issues/6263#issuecomment-1336829158
Hi,
I added the below jars to the HIVE lib directory and it all works fine now.
1.iceberg-hive-runtime-1.2.0-SNAPSHOT.jar ( Got it by running ./gradlew build -x test -x integrationTest on Iceberg download)
2. iceberg-hive3-1.1.0.jar (Downloaded it from Maven Central https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-hive3/1.1.0/iceberg-hive3-1.1.0.jar)
All running good now.
Thanks,
Kowshik
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra closed issue #6263: Unable to create Iceberg Table using HIVE
Posted by GitBox <gi...@apache.org>.
nastra closed issue #6263: Unable to create Iceberg Table using HIVE
URL: https://github.com/apache/iceberg/issues/6263
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on issue #6263: Unable to create Iceberg Table using HIVE
Posted by GitBox <gi...@apache.org>.
pvary commented on issue #6263:
URL: https://github.com/apache/iceberg/issues/6263#issuecomment-1326823416
What is the execution engine that you are using?
Could you use Hive 4.0.0 instead? Iceberg is already integrated there, and should work OOB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on issue #6263: Unable to create Iceberg Table using HIVE
Posted by GitBox <gi...@apache.org>.
nastra commented on issue #6263:
URL: https://github.com/apache/iceberg/issues/6263#issuecomment-1326060759
It is correct to only add `iceberg-hive-runtime-1.2.0-SNAPSHOT.jar` (if you inspect its content, then it will also contain `org.apache.iceberg.mr.hive.serde.objectinspector.IcebergObjectInspector`). The other two jars should not be necessary. Which Hive version are you running?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kowshikdremio commented on issue #6263: Unable to create Iceberg Table using HIVE
Posted by GitBox <gi...@apache.org>.
kowshikdremio commented on issue #6263:
URL: https://github.com/apache/iceberg/issues/6263#issuecomment-1326073724
I am using Hive 3.1.3....
> It is correct to only add `iceberg-hive-runtime-1.2.0-SNAPSHOT.jar` (if you inspect its content, then it will also contain `org.apache.iceberg.mr.hive.serde.objectinspector.IcebergObjectInspector`). The other two jars should not be necessary. Which Hive version are you running?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org