You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/04/18 09:11:04 UTC
Slack digest for #general - 2020-04-18
2020-04-17 09:13:56 UTC - Frans Guelinckx: Hi, i’m running into issues when trying out the functions SDK in version 2.5.0. It throws a `java.lang.ClassNotFoundException: org.apache.pulsar.client.impl.schema.KeyValueSchema`
----
2020-04-17 09:14:44 UTC - Frans Guelinckx: stacktrace:
```08:31:26.055 [public/default/employerFunction-0] INFO org.apache.pulsar.functions.instance.JavaInstanceRunnable - Encountered user exception when processing message PulsarRecord(topicName=Optional[<persistent://public/default/everesst.employer.DOMAIN_EVENT_RECORD>], partition=0, message=Optional[org.apache.pulsar.client.impl.MessageImpl@6972c5a6], failFunction=org.apache.pulsar.functions.source.PulsarSource$$Lambda$162/1486417897@be7fd19, ackFunction=org.apache.pulsar.functions.source.PulsarSource$$Lambda$161/1923546188@36053bec)
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.pulsar.client.impl.schema.KeyValueSchema
at org.apache.pulsar.client.internal.ReflectionUtils.catchExceptions(ReflectionUtils.java:46) ~[java-instance.jar:?]
at org.apache.pulsar.client.internal.DefaultImplementation.newKeyValueBytesSchema(DefaultImplementation.java:267) ~[java-instance.jar:?]
at org.apache.pulsar.client.api.Schema.KV_BYTES(Schema.java:277) ~[java-instance.jar:?]
at be.everesst.pulsar.functions.employer.function.EmployeeRoutingFunction.lambda$process$0(EmployeeRoutingFunction.java:37) ~[?:?]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_232]
at be.everesst.pulsar.functions.employer.function.EmployeeRoutingFunction.process(EmployeeRoutingFunction.java:35) ~[?:?]
at be.everesst.pulsar.functions.employer.function.EmployeeRoutingFunction.process(EmployeeRoutingFunction.java:20) ~[?:?]
at org.apache.pulsar.functions.instance.JavaInstance.handleMessage(JavaInstance.java:63) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:269) [org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.pulsar.client.impl.schema.KeyValueSchema
at org.apache.pulsar.client.internal.ReflectionUtils.newClassInstance(ReflectionUtils.java:63) ~[java-instance.jar:?]
at org.apache.pulsar.client.internal.ReflectionUtils.getStaticMethod(ReflectionUtils.java:78) ~[java-instance.jar:?]
at org.apache.pulsar.client.internal.DefaultImplementation.lambda$newKeyValueBytesSchema$30(DefaultImplementation.java:268) ~[java-instance.jar:?]
at org.apache.pulsar.client.internal.ReflectionUtils.catchExceptions(ReflectionUtils.java:35) ~[java-instance.jar:?]
... 9 more
Caused by: java.lang.ClassNotFoundException: org.apache.pulsar.client.impl.schema.KeyValueSchema
at <http://java.net|java.net>.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_232]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_232]
at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_232]
at org.apache.pulsar.client.internal.ReflectionUtils.newClassInstance(ReflectionUtils.java:60) ~[java-instance.jar:?]
at org.apache.pulsar.client.internal.ReflectionUtils.getStaticMethod(ReflectionUtils.java:78) ~[java-instance.jar:?]
at org.apache.pulsar.client.internal.DefaultImplementation.lambda$newKeyValueBytesSchema$30(DefaultImplementation.java:268) ~[java-instance.jar:?]
at org.apache.pulsar.client.internal.ReflectionUtils.catchExceptions(ReflectionUtils.java:35) ~[java-instance.jar:?]
... 9 more```
----
2020-04-17 09:16:06 UTC - Frans Guelinckx: I only depend on `pulsar-functions-api` and `pulsar-client-api` , but a class in the client-api seems to want to load a class from client-impl
----
2020-04-17 09:16:19 UTC - Frans Guelinckx: Am i doing something wrong or is this an issue?
----
2020-04-17 09:51:20 UTC - JG: yes, for preso I didnt find any exemple of java client, only presto cli.
----
2020-04-17 10:52:13 UTC - Frans Guelinckx: With pulsar 2.4.0 it works, so is an issue in 2.5.0 i guess?
----
2020-04-17 13:18:35 UTC - Gilles Barbier: Hi! Is that an expected behavior that after a
```./bin/pulsar-admin schemas delete <persistent://public/default/topic>```
I still have access to other version ? Eg. with
``` ./bin/pulsar-admin schemas get <persistent://public/default/topic> --version 1```
----
2020-04-17 13:20:58 UTC - Penghui Li: Which version of Pulsar are you using?
----
2020-04-17 13:22:36 UTC - Gilles Barbier: 2.5.0 standalone
----
2020-04-17 13:26:38 UTC - Penghui Li: Can you get any information if do not specify --version ?
```./bin/pulsar-admin schemas get <persistent://public/default/topic>```
----
2020-04-17 13:27:00 UTC - Gilles Barbier: No - I have then a 404
----
2020-04-17 13:31:46 UTC - Penghui Li: Ok, looks some problem with query schema by schema version. Let me check if it is fixed in the master branch.
----
2020-04-17 13:50:47 UTC - Penghui Li: Ok, the problem is still there. I have created an issue to track this bug <https://github.com/apache/pulsar/issues/6754>.
----
2020-04-17 13:53:15 UTC - Gilles Barbier: Thx!
----
2020-04-17 14:04:22 UTC - Penghui Li: Do you use key value schema? You need add the client-impl dependency because function instance need client-impl to create schemas.
----
2020-04-17 14:12:50 UTC - Gilles Barbier: In the same spirit, I do
```./bin/pulsar-admin topics delete <persistent://public/default/topic> --force```
then
```/bin/pulsar-admin topics list public/default```
gives me
```"<persistent://public/default/topic>"```
----
2020-04-17 14:14:04 UTC - Gilles Barbier: (I have a function on this topic)
----
2020-04-17 14:23:04 UTC - Penghui Li: By default, when producers or consumers connects to a broker, the broker will create topic if the topic is absent. If uses `--force` option, the broker close the client connection and then delete the topic. But the client will try to reconnect after disconnecting, so the topic created again. If you don’t want topic to be created automatically, you can set `allowAutoTopicCreation=false` in the broker.conf
+1 : David Kjerrumgaard
----
2020-04-17 14:29:11 UTC - Gilles Barbier: Thanks for taking the time. I have deleted the function, deleted the topic, it worked
+1 : Penghui Li
----
2020-04-17 15:01:39 UTC - David Kjerrumgaard: It appears that your ultimate goal for this use case is to "fan-in" data from 50+ sources, it might be best to do some of the aggregation in Pulsar first, i.e. all sensors of the same type publish to the same topic and carry a meta-tag to indicate which site they came from. Then you can dump that data periodically into a DB or use Pulsar SQL to access it.
----
2020-04-17 15:12:57 UTC - Frans Guelinckx: Is there a `pulsar-client-impl` dependency? I cant seem to find it in the public maven repository.
----
2020-04-17 15:15:42 UTC - Frans Guelinckx: adding `pulsar-client` doesnt work either: it throws a ClassCastException
----
2020-04-17 15:19:16 UTC - Frans Guelinckx: @Penghui Li yes we use KeyValue. We started with the same dependencies as this pom file: <https://github.com/apache/pulsar/blob/master/pulsar-functions/java-examples/pom.xml>
----
2020-04-17 15:35:47 UTC - Aaron Verachtert: @Aaron Verachtert has joined the channel
----
2020-04-17 16:07:13 UTC - Ebere Abanonu: @Sijie Guo @Penghui Li is this right?
```"name" : "authoritative",
"in" : "query",
"description" : "messageId where to create the subscription. It can be 'latest', 'earliest' or (ledgerId:entryId)",
"required" : false,
"type" : "boolean",
"default" : "latest",
"enum" : [ "latest", "earliest", "ledgerId:entryId" ]```
----
2020-04-17 16:07:53 UTC - Ebere Abanonu: That is pulsar admin swagger definition
----
2020-04-17 18:41:30 UTC - Christian Oelsner: @Christian Oelsner has joined the channel
----
2020-04-18 01:59:40 UTC - Penghui Li: Can you show me some details about the ClassCastException?
----
2020-04-18 02:17:18 UTC - Penghui Li: No, The wrong description information is set in the API description. I have created an issue <https://github.com/apache/pulsar/issues/6759> to track this problem. If you are interested in fix this problem, welcome.
----
2020-04-18 03:27:52 UTC - PercyGauguin: @PercyGauguin has joined the channel
----
2020-04-18 06:41:37 UTC - Gautam Lodhiya: I am using pulsar’s websocket api and want to send negative ack. For normal ack we have to send signal with {messageId: “”}. But, how I can signal pulsar with negative ack so that broker re-enqueue that msg again?
----
2020-04-18 07:22:23 UTC - Maxi: As far as I understand it, there is no need for negative ack. Pulsar holds messages that dont get ack in backlog and automatically re-enqueues them.
----
2020-04-18 07:27:06 UTC - Gautam Lodhiya: @Maxi - But that timeout will delay the execution like say 20 mins or something whatever is being set.
Is there any way to negative ack broker which will reequeue that msg again immediately.
----
2020-04-18 08:15:53 UTC - Greg Methvin: As far as I can tell the websocket client is not implementing negative acknowledgements, at least based on this code: <https://github.com/apache/pulsar/blob/466ba07a04f75f532180b0ac5cf5af52b372baa7/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/ConsumerHandler.java#L243|https://github.com/apache/pulsar/blob/466ba07a04f75f532180b0ac5cf5af52b372baa7/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/ConsumerHandler.java#L243>. It doesn’t seem like it’d be too hard to add though. It probably makes sense to have a new type of command to negatively acknowledge.
----
2020-04-18 08:18:17 UTC - Greg Methvin: The websocket client claims to support all features of the regular clients so it should support nacks too
----