You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/02/11 10:04:05 UTC
[GitHub] [pulsar] BohanZhang0222 opened a new issue #14239: 使用Kafka source connector导数报错：NoClassDefFoundError
BohanZhang0222 opened a new issue #14239:
URL: https://github.com/apache/pulsar/issues/14239
#### Expected behavior
按照官方示例：https://pulsar.apache.org/docs/en/io-kafka-source/
通过 sources localrun 的方式，试图完成kafka数据导入到pulsar。
#### Actual behavior
实际上，我的操作并没有成功。
出现报错：
java.lang.RuntimeException: User class must be in class path
at org.apache.pulsar.common.util.Reflections.createInstance(Reflections.java:113) ~[org.apache.pulsar-pulsar-common-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupInput(JavaInstanceRunnable.java:735) ~[org.apache.pulsar-pulsar-functions-instance-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:232) ~[org.apache.pulsar-pulsar-functions-instance-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:260) ~[org.apache.pulsar-pulsar-functions-instance-2.9.1.jar:2.9.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
Caused by: java.lang.NoClassDefFoundError: org/apache/kafka/clients/consumer/Consumer
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_162]
at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_162]
at org.apache.pulsar.common.util.Reflections.createInstance(Reflections.java:111) ~[org.apache.pulsar-pulsar-common-2.9.1.jar:2.9.1]
... 4 more
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.consumer.Consumer
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_162]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_162]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_162]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_162]
at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_162]
at org.apache.pulsar.common.util.Reflections.createInstance(Reflections.java:111) ~[org.apache.pulsar-pulsar-common-2.9.1.jar:2.9.1]
... 4 more
#### Steps to reproduce
**我创建了目录connectors，并且放入了文件pulsar-io-kafka-2.9.1.nar。**
**我的kafka-source配置文件：**
{
"bootstrapServers": "bigdata-kafka-01-ontest.chj.cloud:6667,bigdata-kafka-02-ontest.chj.cloud:6667,bigdata-kafka-03-ontest.chj.cloud:6667",
"groupId": "test-pulsar-io",
"topic": "test-zbh",
"sessionTimeoutMs": "10000",
"autoCommitEnabled": false
}
**我执行的命令：**
bin/pulsar-admin sources localrun --archive /chj/pulsarCluster/connectors/pulsar-io-kafka-2.9.1.nar --classname org.apache.pulsar.io.kafka.KafkaBytesSource --tenant public --namespace default --name kafka --destination-topic-name test-zbh --source-config-file ./sourceConfig/kafka-source.json --parallelism 1
执行就会报出User class must be in class path，
由于Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.consumer.Consumer
**所以我认为是pulsar的lib中，可能不包含我使用的kafka版本的client，于是我在pulsar/lib中上传了一个kafka-client的jar包**
**重新执行sources localrun命令后，我得到了不同的报错结果。**
#### System configuration
**我的来源kafka：**
版本：1.1.1
3个节点的集群部署
**我的Pulsar集群：**
版本：2.9.1
3个节点的集群部署（zk、broker、bookie）。
另外，kafka和pulsar使用了不同的zk。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] freeznet commented on issue #14239: Kafka Source Connector "User class must be in class path" in pulsar sources localrun
Posted by GitBox <gi...@apache.org>.
freeznet commented on issue #14239:
URL: https://github.com/apache/pulsar/issues/14239#issuecomment-1038515410
if you are following the pulsar's docs, you can try to remove the `--className` parameter when `localrun` or `create` the source connector.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] freeznet commented on issue #14239: Kafka Source Connector "User class must be in class path" in pulsar sources localrun
Posted by GitBox <gi...@apache.org>.
freeznet commented on issue #14239:
URL: https://github.com/apache/pulsar/issues/14239#issuecomment-1038515410
if you are following the pulsar's docs, you can try to remove the `--className` parameter when `localrun` or `create` the source connector.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BohanZhang0222 commented on issue #14239: Kafka Source Connector "User class must be in class path" in pulsar sources localrun
Posted by GitBox <gi...@apache.org>.
BohanZhang0222 commented on issue #14239:
URL: https://github.com/apache/pulsar/issues/14239#issuecomment-1038684978
去掉--className参数后，问题解决。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BohanZhang0222 closed issue #14239: Kafka Source Connector "User class must be in class path" in pulsar sources localrun
Posted by GitBox <gi...@apache.org>.
BohanZhang0222 closed issue #14239:
URL: https://github.com/apache/pulsar/issues/14239
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org