You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Awsmsniper (via GitHub)" <gi...@apache.org> on 2024/04/16 02:44:14 UTC
[I] [Bug] ClassNotFoundException error occurred when using Function development in the kubernetes cluster [pulsar]
Awsmsniper opened a new issue, #22513:
URL: https://github.com/apache/pulsar/issues/22513
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### Version
- kubernetes: 1.28
- jdk: 17
- pulsar: 3.2.1
- pulsar-functions-pulsarctl-java-runner: 3.2.1.2
### Minimal reproduce step
1. Develop using the Extended Pulsar Functions SDK for Java and connect to the Redis cluster
2. Edit yaml file
3. kubectl apply -f function.yaml
### What did you expect to see?
The function can connect to the Redis cluster and work properly.
### What did you see instead?
```java
2024-04-15T06:18:32,716+0000 [public/default/demo-function-las-auth-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - Sink open produced uncaught exception:
java.lang.ClassNotFoundException: com.alibaba.fastjson2.JSONObject
at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
at org.apache.pulsar.common.util.Reflections.loadClass(Reflections.java:310) ~[io.streamnative-pulsar-common-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:437) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:362) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:1070) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:261) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:305) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at java.lang.Thread.run(Thread.java:840) ~[?:?]
2024-04-15T06:18:32,725+0000 [public/default/demo-function-las-auth-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - [public/default/demo-function-las-auth:0] Uncaught exception in Java Instance
java.lang.ClassNotFoundException: com.alibaba.fastjson2.JSONObject
at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
at org.apache.pulsar.common.util.Reflections.loadClass(Reflections.java:310) ~[io.streamnative-pulsar-common-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:437) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:362) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:1070) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:261) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:305) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at java.lang.Thread.run(Thread.java:840) ~[?:?]
```
### Anything else?
```yaml
apiVersion: compute.functionmesh.io/v1alpha1
kind: Function
metadata:
name: demo-function-las-auth
namespace: pulsar
spec:
image: streamnative/pulsar-functions-pulsarctl-java-runner:3.2.1.2 # using java function runner
className: com.audit.function.handler.LasAuthFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 5
logTopic: persistent://demo-tenants/log-namespace/function-las-auth-logs
input:
topics:
- persistent://demo-tenants/log-namespace/las-auth
typeClassName: "[Ljava.lang.String;"
output:
topic: persistent://demo-tenants/log-namespace/las-auth-done
typeClassName: "com.alibaba.fastjson2.JSONObject"
funcConfig:
redis.url: redis-cluster.redis-cluster.svc.cluster.local
redis.password: ******
secretsMap:
"redis-ca-certs":
path: "redis-ca-certs"
key: "ca.crt"
"redis-tls-cert":
path: "redis-tls-cert"
key: "tls.crt"
"redis-tls-key":
path: "redis-tls-key"
key: "tls.key"
pulsar:
pulsarConfig: "demo-pulsar"
authSecret: "pulsar-auth"
tlsConfig:
enabled: true
allowInsecure: true
hostnameVerification: false
certSecretName: pulsar-tls-proxy
certSecretKey: ""
resources:
requests:
cpu: "2"
memory: 4G
limits:
cpu: "2"
memory: 4G
java:
extraDependenciesDir: random-dir/
jar: demo-function-1.1-SNAPSHOT.jar # the package will download as this filename.
log:
level: "debug"
jarLocation: function://demo-tenants/log-namespace/demo-las-auth-function@1.1 # function package URL
javaOpts:
- "--add-opens java.base/sun.net=ALL-UNNAMED"
- "--add-opens java.management/sun.management=ALL-UNNAMED"
autoAck: true
---
apiVersion: v1
kind: ConfigMap
metadata:
name: demo-pulsar
namespace: pulsar
data:
webServiceURL: https://pulsar-proxy.pulsar.svc.cluster.local:443
brokerServiceURL: pulsar+ssl://pulsar-proxy.pulsar.svc.cluster.local:6651
---
apiVersion: v1
metadata:
name: pulsar-auth
namespace: pulsar
data:
clientAuthenticationPlugin: ******
clientAuthenticationParameters: ******
kind: Secret
type: Opaque
```
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] ClassNotFoundException error occurred when using Function development in the kubernetes cluster [pulsar]
Posted by "Awsmsniper (via GitHub)" <gi...@apache.org>.
Awsmsniper commented on issue #22513:
URL: https://github.com/apache/pulsar/issues/22513#issuecomment-2058133221
@lhotari @nlu90
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] ClassNotFoundException error occurred when using Function development in the kubernetes cluster [pulsar]
Posted by "Awsmsniper (via GitHub)" <gi...@apache.org>.
Awsmsniper commented on issue #22513:
URL: https://github.com/apache/pulsar/issues/22513#issuecomment-2058133839
@lhotari @nlu90 Hope to receive help, thank you very much!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] ClassNotFoundException error occurred when using Function development in the kubernetes cluster [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #22513:
URL: https://github.com/apache/pulsar/issues/22513#issuecomment-6381519014
Thanks for the report, @Awsmsniper. The output type class (`typeClassName`) is loaded with the function's own class loader (`PulsarSink.initializeSchema` on master), so `com.alibaba.fastjson2.JSONObject` must be packaged in the function jar (for example a shaded/fat jar) or in the configured extra-dependencies directory. Your setup uses Function Mesh and a StreamNative runner image on 3.2, which this project does not maintain, and 3.2 is no longer supported. If this still fails with fastjson2 bundled in the jar on a supported Apache Pulsar release (4.0.x, 4.2.x or 5.0.x), please share a minimal function and the exact deployment steps. Without an update, this issue will be closed in a few weeks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org