You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/09/26 07:54:54 UTC
[GitHub] [pulsar] nicoloboschi opened a new pull request, #17835: [fix][connector] KCA: use reflection to get pulsar-client impl classes
nicoloboschi opened a new pull request, #17835:
URL: https://github.com/apache/pulsar/pull/17835
### Motivation
With the [latest changes to KCA](https://github.com/datastax/pulsar/commit/625da657cf509e547d934cf3b825786866f0386d) now you get this error if the message is batched:
```
java.lang.ClassCastException: class org.apache.pulsar.client.impl.BatchMessageIdImpl cannot be cast to class org.apache.pulsar.client.impl.MessageIdImpl (org.apache.pulsar.client.impl.BatchMessageIdImpl is in unnamed module of loader java.net.URLClassLoader @2ff4acd0; org.apache.pulsar.client.impl.MessageIdImpl is in unnamed module of loader org.apache.pulsar.common.nar.NarClassLoader @6ff415ad)
at org.apache.pulsar.io.kafka.connect.KafkaConnectSink.getMessageOffset(KafkaConnectSink.java:327) ~[pulsar-io-kafka-connect-adaptor-?.jar]
at org.apache.pulsar.io.kafka.connect.KafkaConnectSink.toSinkRecord(KafkaConnectSink.java:408) ~[pulsar-io-kafka-connect-adaptor-?.jar:]
at org.apache.pulsar.io.kafka.connect.KafkaConnectSink.write(KafkaConnectSink.java:121) [pulsar-io-kafka-connect-adaptor-?.jar:]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.sendOutputMessage(JavaInstanceRunnable.java:368) [com.datastax.oss-pulsar-functions-instance-?-SNAPSHOT.jar:?-SNAPSHOT]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.handleResult(JavaInstanceRunnable.java:351) [com.datastax.oss-pulsar-functions-instance-?-SNAPSHOT.jar:?-SNAPSHOT]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:300) [com.datastax.oss-pulsar-functions-instance-?-SNAPSHOT.jar:?-SNAPSHOT]
```
The root problem is that connectors should not bundle the Pulsar client impl because it's provided at runtime by the Java Functions framework.
### Modifications
* Use reflection to get the custom sequenceId
- [x] `doc-not-needed`
(Please explain why)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #17835: [fix][connector] KCA: use reflection to get pulsar-client impl classes
Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on PR #17835:
URL: https://github.com/apache/pulsar/pull/17835#issuecomment-1644131745
Hi Nico, I met a similar error (see https://github.com/BewareMyPower/pulsar-sink-demo) when I'm trying the connector.
> because it's provided at runtime by the Java Functions framework.
Could you explain a bit more? You mean the Pulsar client impl JAR is in the classpath and the connector is loaded by the Java Functions framework so the connector cannot access classes in Pulsar broker's classpath?
BTW, if a connector cannot access methods from modules like `pulsar-client-original` directly, it would be a bad experience for Pulsar connector developers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi merged pull request #17835: [fix][connector] KCA: use reflection to get pulsar-client impl classes
Posted by GitBox <gi...@apache.org>.
nicoloboschi merged PR #17835:
URL: https://github.com/apache/pulsar/pull/17835
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org