You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Yuchen Fan (Jira)" <ji...@apache.org> on 2023/02/09 11:36:00 UTC

[jira] [Created] (IMPALA-11910) Java UDF with resource file may cause catalogd start failed

Yuchen Fan created IMPALA-11910:
-----------------------------------

             Summary: Java UDF with resource file may cause catalogd start failed
                 Key: IMPALA-11910
                 URL: https://issues.apache.org/jira/browse/IMPALA-11910
             Project: IMPALA
          Issue Type: Bug
          Components: Catalog, Frontend
    Affects Versions: Impala 4.1.0
            Reporter: Yuchen Fan
         Attachments: image-2023-02-09-19-15-10-624.png

In IMPALA-10997, UDF implementation is refactored for supporting GenericUDF. After Impala-4.1.0, catalogd will start failed due to some irreguler UDFs, which load some files or libraries in UDF class initialization. For example, some uses' UDF may declare like this:

 
{code:java}
public class TestUdf extends UDF {

TestUdf() {}

static {
  // 'resource.db' is attached in UDF jar
  InputStream is = XXX.class.getClassLoader().getResourceAsStream("resource.db");
}
...
// some UDF implementation
...

} {code}
When loading these UDFs, catalogd will crash and report error.

 

!image-2023-02-09-19-15-10-624.png!

We found UDF loading codes delete UDF jar file too early. Related code shown below:

 
{code:java}
  public static HiveUdfLoader createWithLocalPath(String localLibPath, Function fn) {
    ...
    // copy jar to local     
    FileSystemUtil.copyToLocal(new Path(uri), localJarPath); 
    ...
    // contruct HiveUdfLoader
    ...
    // delete local jar file
    FileSystemUtil.deleteIfExists(localJarPath);
}
  
  // Now local jar file is deleted
  private UDF instantiateUDFInstance(Class<?> udfClass) {
    ...
    try {
      Constructor<?> ctor = udfClass.getConstructor();
      // static block containing open jar resource of UDF class will fail here       
      return (UDF) ctor.newInstance();
    } catch (...Exception e) {
      ...
    }
  } {code}
To solve this, we should delay delete jar until UDF class finished to instantiate.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org