You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2021/02/15 17:19:00 UTC
[jira] [Comment Edited] (HUDI-1609) Issues w/ using hive metastore
by disabling jdbc
[ https://issues.apache.org/jira/browse/HUDI-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284866#comment-17284866 ]
sivabalan narayanan edited comment on HUDI-1609 at 2/15/21, 5:18 PM:
---------------------------------------------------------------------
hive sync w/ metastore approach fails locally with docker set up.
Tried to follow the docker demo here.
When trying to do the hive sync, running into ClassNotFoundError w/ CommonTree in antlr.
{code:java}
2021-02-15 17:05:33,033 INFO [main] hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 0
Exception in thread "main" java.lang.NoClassDefFoundError: org/antlr/runtime/tree/CommonTree
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQLs(HoodieHiveClient.java:401)
at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQLUsingHiveDriver(HoodieHiveClient.java:384)
at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:374)
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:122)
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:94)
at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:231)
Caused by: java.lang.ClassNotFoundException: org.antlr.runtime.tree.CommonTree
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 11 more
{code}
maven dep tree for antlr-runtime
[https://gist.github.com/nsivabalan/4a308f3a55b1520edee067996b373215]
Looks like, when this jar is transitively fetched via hive-metastore, scope it chosen as "provided". Eg: *hudi-spark-bundle_2.11*
But if this jar is transitively fetched via "org.antlr:stringtemplate", scope is chosen as "compile".
May be this could the issue.
was (Author: shivnarayan):
hive sync w/ metastore approach fails locally with docker set up.
Tried to follow the docker demo here.
When trying to do the hive sync, running into ClassNotFoundError w/ CommonTree in antlr.
```
2021-02-15 17:05:33,033 INFO [main] hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 0
Exception in thread "main" java.lang.NoClassDefFoundError: org/antlr/runtime/tree/CommonTree
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQLs(HoodieHiveClient.java:401)
at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQLUsingHiveDriver(HoodieHiveClient.java:384)
at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:374)
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:122)
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:94)
at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:231)
Caused by: java.lang.ClassNotFoundException: org.antlr.runtime.tree.CommonTree
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 11 more
```
maven dep tree for antlr-runtime
[https://gist.github.com/nsivabalan/4a308f3a55b1520edee067996b373215]
Looks like, when this jar is transitively fetched via hive-metastore, scope it chosen as "provided". Eg: *hudi-spark-bundle_2.11*
But if this jar is transitively fetched via "org.antlr:stringtemplate", scope is chosen as "compile".
May be this could the issue.
> Issues w/ using hive metastore by disabling jdbc
> ------------------------------------------------
>
> Key: HUDI-1609
> URL: https://issues.apache.org/jira/browse/HUDI-1609
> Project: Apache Hudi
> Issue Type: Task
> Components: Hive Integration
> Reporter: sivabalan narayanan
> Priority: Major
> Labels: sev:critical, user-support-issues
> Fix For: 0.8.0
>
>
> Ref: https://github.com/apache/hudi/issues/1679
--
This message was sent by Atlassian Jira
(v8.3.4#803005)