You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by mo...@apache.org on 2023/06/14 08:03:06 UTC
[doris] branch master updated: [fix](hudi) set default class loader for hudi serializer (#20680)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 062641e8f8 [fix](hudi) set default class loader for hudi serializer (#20680)
062641e8f8 is described below
commit 062641e8f835e91cfe4254d6373998c2bd8cb430
Author: Ashin Gau <As...@users.noreply.github.com>
AuthorDate: Wed Jun 14 16:02:56 2023 +0800
[fix](hudi) set default class loader for hudi serializer (#20680)
hudi serializer `org.apache.hudi.common.util.SerializationUtils$KryoInstantiator.newKryo` throws error like `java.lang.IllegalArgumentException: classLoader cannot be null`. Set the default class loader for scan thread.
```
public Kryo newKryo() {
Kryo kryo = new Kryo();
...
// Thread.currentThread().getContextClassLoader() returns null
kryo.setClassLoader(Thread.currentThread().getContextClassLoader());
...
return kryo;
}
```
---
.../src/main/java/org/apache/doris/hudi/HudiJniScanner.java | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/fe/be-java-extensions/hudi-scanner/src/main/java/org/apache/doris/hudi/HudiJniScanner.java b/fe/be-java-extensions/hudi-scanner/src/main/java/org/apache/doris/hudi/HudiJniScanner.java
index d7078dc9ca..ce86b27ad6 100644
--- a/fe/be-java-extensions/hudi-scanner/src/main/java/org/apache/doris/hudi/HudiJniScanner.java
+++ b/fe/be-java-extensions/hudi-scanner/src/main/java/org/apache/doris/hudi/HudiJniScanner.java
@@ -155,6 +155,11 @@ public class HudiJniScanner extends JniScanner {
InputFormat<?, ?> inputFormatClass = HudiScanUtils.createInputFormat(jobConf, hudiScanParam.getInputFormat());
+ // org.apache.hudi.common.util.SerializationUtils$KryoInstantiator.newKryo
+ // throws error like `java.lang.IllegalArgumentException: classLoader cannot be null`.
+ // Set the default class loader
+ Thread.currentThread().setContextClassLoader(classLoader);
+
// RecordReader will use ProcessBuilder to start a hotspot process, which may be stuck,
// so use another process to kill this stuck process.
// TODO(gaoxin): better way to solve the stuck process?
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org