You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2022/06/10 18:42:00 UTC
[jira] [Work started] (IMPALA-11342) Hive UDFs are unable to use classes from the same Jar in catalogd
[ https://issues.apache.org/jira/browse/IMPALA-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on IMPALA-11342 started by Csaba Ringhofer.
------------------------------------------------
> Hive UDFs are unable to use classes from the same Jar in catalogd
> -----------------------------------------------------------------
>
> Key: IMPALA-11342
> URL: https://issues.apache.org/jira/browse/IMPALA-11342
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Major
>
> While it is a known limitations that Hive UDFs in Impala cannot load dependencies from classpath, they should be able to load classes from the same jar (and thus creating an shaded jar with the dependencies should be a valid workaround). Meanwhile I got an error when the parent class of the UDF was a class defined in another package in the jar.
> The exception occurred when catalogd tried to instantiate the UDF class:
> {code}
> I0608 14:50:21.520296 22464 jni-util.cc:286] java.lang.NoClassDefFoundError: com/esri/core/geometry/Point
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.impala.hive.executor.HiveLegacyJavaFunction.instantiateUDFInstance(HiveLegacyJavaFunction.java:115)
> at org.apache.impala.hive.executor.HiveLegacyJavaFunction.<init>(HiveLegacyJavaFunction.java:74)
> at org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:50)
> at org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:67)
> at org.apache.impala.service.CatalogOpExecutor.createFunction(CatalogOpExecutor.java:2117)
> at org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:453)
> at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:245)
> Caused by: java.lang.ClassNotFoundException: com.esri.core.geometry.Point
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 11 more
> {code}
> The issue seems to be cause by closing the UrlClassLoader too early, after loading the class but before doing the actual instantiation:
> https://github.com/apache/impala/blob/13bbff4e4e5fc5d459cc6f7a5512f84ceba897cd/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfLoader.java#L71
> Note that this is an issue only in the catalogd, not the executors, as the class loader is kept open longer in that case.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org