You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2020/12/25 07:26:19 UTC
[GitHub] [camel-kafka-connector] iamrsaravana opened a new issue #817: Reading data from AMQP protocal using Strimzi kafka connector
iamrsaravana opened a new issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817
I am planning to pull the data from Hono using Strimzi kafka(Version 0.20.0) connector (Camel AMQP source connector). I have followed the below steps to read data from Hono.
I downloaded Camel-amqp-kafka-connector, JMS Jar files from below link:
https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-amqp-kafka-connector/0.7.0/camel-amqp-kafka-connector-0.7.0-package.tar.gz
https://downloads.apache.org/qpid/jms/0.51.0/apache-qpid-jms-0.51.0-bin.tar.gz
After downloaded the above tar and unzipped and created docker image file using below command
cat <<EOF >Dockerfile
FROM strimzi/kafka:0.20.1-kafka-2.6.0
USER root:root
RUN mkdir -p /opt/kafka/plugins/camel
COPY ./camel-activemq-kafka-connector/* /opt/kafka/plugins/camel/
USER 1001
EOF
Docker build -f ./Dockerfile -t localhost:5000/my-connector-amqp_new .
docker push localhost:5000/my-connector-amqp_new
using below command i have created kafkaConnect( here used my local image created above)
apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaConnect
metadata:
name: my-connect-cluster
annotations:
strimzi.io/use-connector-resources: "true"
spec:
image: 10.128.0.6:5000/my-connector-amqp_new
replicas: 1
bootstrapServers: my-cluster-kafka-bootstrap:9092
config:
group.id: connect-cluster
offset.storage.topic: connect-cluster-offsets
config.storage.topic: connect-cluster-configs
status.storage.topic: connect-cluster-status
config.storage.replication.factor: 1
offset.storage.replication.factor: 1
status.storage.replication.factor: 1
using below command to pull the data from hono message queue:
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
name: camelamqpsourceconnector
labels:
strimzi.io/cluster: my-connect-cluster
spec:
class: org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
tasksMax: 1
config:
camel.component.amqp.includeAmqpAnnotations: true
camel.component.amqp.connectionFactory.class: org.apache.qpid.jms.JmsConnectionFactory
camel.component.amqp.connectionFactory.remoteURI: $( kubectl get service eclipse-hono-dispatch-router-ext --output="jsonpath={.spec.clusterIP}" -n hono):15672
camel.component.amqp.username: consumer@HONO
camel.component.amqp.password: verysecret
camel.component.amqp.testConnectionOnStartup: true
camel.source.kafka.topic: mytopic
camel.source.path.destinationType: queue
camel.source.path.destinationName: test-queue
i am not getting any data in my above kafka topics (mytopic)
Could you check is there any i need to modify to get the data in my kafka topics. I have searched various forum but i could not get any clear direction.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751688302
I already answered. They are not confusing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751365170
I was referring the below link and gave my inputs
https://github.com/apache/camel-kafka-connector/blob/master/examples/CamelAmqpSourceConnector.properties
Values are in this file as mentioned it below:
name=CamelAmqpSourceConnector
topics=mytopic
tasks.max=1
connector.class=org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
camel.source.path.destinationType=queue
camel.source.path.destinationName=test-queue
camel.component.amqp.includeAmqpAnnotations=true
**camel.component.amqp.connectionFactory=#class:org.apache.qpid.jms.JmsConnectionFactory
camel.component.amqp.connectionFactory.remoteURI=amqp://localhost:5672**
camel.component.amqp.username=admin
camel.component.amqp.password=admin
camel.component.amqp.testConnectionOnStartup=true
I am using the below configurations. Could you suggest me what is the correct value for this property?
**camel.component.amqp.connectionFactor**
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
name: camelamqpsourceconnector
labels:
strimzi.io/cluster: my-connect-cluster-new
spec:
class: org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
tasksMax: 1
config:
camel.component.amqp.includeAmqpAnnotations: true
**camel.component.amqp.connectionFactory: org.apache.qpid.jms.JmsConnectionFactory
camel.component.amqp.connectionFactory.remoteURI: amqp://$( kubectl get service eclipse-hono-dispatch-router-ext --output="jsonpath={.spec.clusterIP}" -n hono):15671**
camel.component.amqp.username: consumer@HONO
camel.component.amqp.password: verysecret
camel.component.amqp.testConnectionOnStartup: true
camel.source.kafka.topic: mytopic
camel.source.path.destinationType: queue
camel.source.path.destinationName: test-queue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751442174
Yes, or you have to ask to strimzi how to reference the service
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751342154
after download the tar file from below link
https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-amqp-kafka-connector/0.7.0/camel-amqp-kafka-connector-0.7.0-package.tar.gz
**_unzipped folder does not have any jar file related to AMQP_**
I pushed all the jar files into one folder inside PODs:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana closed issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana closed issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751682082
Are you pointing a queue or a topic? What's the name of queue/topic? I don't know Hono anyway.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751646678
I gave input in yaml file as you mentioned it below:
camel.component.amqp.connectionFactory: #class:org.apache.qpid.jms.JmsConnectionFactory
camel.component.amqp.connectionFactory.remoteURI: amqp://10.106.88.243:15671
Still i am getting below error message:
Spec:
Class: org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
Config:
camel.component.amqp.connectionFactory: <nil>
camel.component.amqp.connectionFactory.remoteURI: amqp://10.106.88.243:15671
camel.component.amqp.includeAmqpAnnotations: true
camel.component.amqp.password: verysecret
camel.component.amqp.testConnectionOnStartup: true
camel.component.amqp.username: consumer@HONO
camel.source.kafka.topic: mytopic
camel.source.path.destinationName: test-queue
camel.source.path.destinationType: queue
Tasks Max: 1
Status:
Conditions:
Last Transition Time: 2020-12-28T09:16:21.630894Z
Status: True
Type: Ready
Connector Status:
Connector:
State: FAILED
Trace: java.lang.ClassCastException: Non-string value found in original settings for key **camel.component.amqp.connectionFactory: null**
at org.apache.kafka.common.config.AbstractConfig.originalsStrings(AbstractConfig.java:241)
at org.apache.kafka.connect.runtime.WorkerConnector.<init>(WorkerConnector.java:88)
at org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:282)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:1321)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1400(DistributedHerder.java:127)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:1329)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:1325)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
worker_id: 10.244.1.137:8083
Name: camelamqpsourceconnector
Tasks:
Type: source
Observed Generation: 1
Tasks Max: 1
Events: <none>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751337694
i have installed using these two jar file
https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-amqp-kafka-connector/0.7.0/camel-amqp-kafka-connector-0.7.0-package.tar.gz
https://downloads.apache.org/qpid/jms/0.51.0/apache-qpid-jms-0.51.0-bin.tar.gz
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751698862
Would you mind open a PR on the camel-kafka-connector-examples repository with your hono examples and steps? Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751408058
do you want me to hardcode this value in configuration file instead of experssion?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751322445
Strimzi operator logs say below error message:
2020-12-26 05:54:31 INFO AbstractConnectOperator:436 - Reconciliation #37(timer) KafkaConnect(kafka/my-connect-cluster): creating/updating connector: camelamqpsourceconnector
2020-12-26 05:54:31 WARN AbstractConnectOperator:573 - Reconciliation #37(timer) KafkaConnect(kafka/my-connect-cluster): **Error reconciling connector camelamqpsourceconnect**or
io.strimzi.operator.cluster.operator.assembly.ConnectRestException: PUT /connectors/camelamqpsourceconnector/config returned 500 (Internal Server Error): Failed to find any class that implements Connector and which name matches org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector, available connectors are: PluginDesc{klass=class org.apache.camel.kafkaconnector.CamelSinkConnector, name='org.apache.camel.kafkaconnector.CamelSinkConnector', version='0.6.1', encodedVersion=0.6.1, type=sink, typeName='sink', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.camel.kafkaconnector.CamelSourceConnector, name='org.apache.camel.kafkaconnector.CamelSourceConnector', version='0.6.1', encodedVersion=0.6.1, type=source, typeName='source', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.camel.kafkaconnector.activemq.CamelActivemqSinkConnector, name='org.apache.camel.kafkaconnector.activemq.CamelActivemqSinkConnector', version='0.6
.1', encodedVersion=0.6.1, type=sink, typeName='sink', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.camel.kafkaconnector.activemq.CamelActivemqSourceConnector, name='org.apache.camel.kafkaconnector.activemq.CamelActivemqSourceConnector', version='0.6.1', encodedVersion=0.6.1, type=source, typeName='source', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnect
or', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='2.6.0', encodedVersion=2.6.0, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=sink, typeName='sink', location='classpath'}, Plugin
Desc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}
at io.strimzi.operator.cluster.operator.assembly.KafkaConnectApiImpl.lambda$createOrUpdatePutRequest$1(KafkaConnectApiImpl.java:84) ~[io.strimzi.cluster-operator-0.20.1.jar:0.20.1]
at io.vertx.core.http.impl.HttpClientResponseImpl$BodyHandler.notifyHandler(HttpClientResponseImpl.java:292) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.http.impl.HttpClientResponseImpl.lambda$bodyHandler$0(HttpClientResponseImpl.java:193) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.http.impl.HttpClientResponseImpl.handleEnd(HttpClientResponseImpl.java:248) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.http.impl.Http1xClientConnection$StreamImpl.lambda$beginResponse$0(Http1xClientConnection.java:483) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.streams.impl.InboundBuffer.handleEvent(InboundBuffer.java:237) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.streams.impl.InboundBuffer.write(InboundBuffer.java:127) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.http.impl.Http1xClientConnection$StreamImpl.endResponse(Http1xClientConnection.java:502) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.http.impl.Http1xClientConnection$StreamImpl.access$000(Http1xClientConnection.java:241) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.http.impl.Http1xClientConnection.handleResponseEnd(Http1xClientConnection.java:641) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.http.impl.Http1xClientConnection.handleHttpMessage(Http1xClientConnection.java:601) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.http.impl.Http1xClientConnection.handleMessage(Http1xClientConnection.java:575) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:366) [io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43) [io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:229) [io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:173) [io.vertx.vertx-core-3.9.1.jar:3.9.1]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [io.netty.netty-codec-4.1.50.Final.jar:4.1.50.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [io.netty.netty-codec-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [io.netty.netty-handler-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [io.netty.netty-transport-4.1.50.Final.jar:4.1.50.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [io.netty.netty-common-4.1.50.Final.jar:4.1.50.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty.netty-common-4.1.50.Final.jar:4.1.50.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.50.Final.jar:4.1.50.Final]
at java.lang.Thread.run(Thread.java:834) [?:?]
2020-12-26 05:54:31 INFO AbstractOperator:457 - Reconciliation #37(timer) KafkaConnect(kafka/my-connect-cluster): reconciled
2020-12-26 05:54:34 INFO AbstractOperator:457 - Reconciliation #35(timer) Kafka(kafka/my-cluster): reconciled
**but my connector cluster is running without any error message**. i have attached (pod/my-connect-cluster-connect-5c84678dd9-dhjj7 ) logs
[my-connect-cluster-connect-5c84678dd9-dhjj7 .txt](https://github.com/apache/camel-kafka-connector/files/5742738/my-connect-cluster-connect-5c84678dd9-dhjj7.txt)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana edited a comment on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana edited a comment on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751408058
Do you want me to hardcode this value in configuration file instead of expression?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751680997
Thanks for quick response.
The below two I have given hono AMQP end point.
camel.source.path.destinationType: topic
camel.source.path.destinationName: mytopic
2020-12-28 11:16:26,047 INFO A JMS MessageConsumer has been closed: JmsConsumerInfo: { ID:30cbf23a-e57b-4507-bc48-044bf3881586:282:1:1, destination = mytopic } (org.apache.qpid.jms.JmsSession) [QpidJMS Connection Executor: ID:30cbf23a-e57b-4507-bc48-044bf3881586:282]
2020-12-28 11:16:31,094 INFO Connection ID:afcc0ed5-cdae-45ae-a5a0-2e72380327dc:283 connected to server: amqp://10.106.88.243:15672 (org.apache.qpid.jms.JmsConnection) [AmqpProvider :(1077):[amqp://10.106.88.243:15672]]
what these two parameter values?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751342690
It must be in a separated folder, not in plugin.
Il sab 26 dic 2020, 11:24 iamrsaravana <no...@github.com> ha
scritto:
> after download the tar file from below link
>
>
> https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-amqp-kafka-connector/0.7.0/camel-amqp-kafka-connector-0.7.0-package.tar.gz
>
> *unzipped folder does not have any jar file related to AMQP*
>
> [image: image]
> <https://user-images.githubusercontent.com/7235459/103149755-79408180-4792-11eb-8c6e-962ee41ee086.png>
>
> I pushed all the jar files into one folder inside PODs:
> [image: image]
> <https://user-images.githubusercontent.com/7235459/103149717-118a3680-4792-11eb-8a15-10070ba7169e.png>
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751342154>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABG6XV2SLIPLXTBZNEIR2SLSWW2VVANCNFSM4VI5DDCQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751338059
The connector needs to be unzipped in a single folder, like for example camel-amqp/ it needs to be segregated in a single folder. The error message seems to be related to this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751695162
Issue is resolved.... Thanks for your support.
This is my final configuation file.
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
name: camelamqpsourceconnector
labels:
strimzi.io/cluster: my-connect-cluster-new
spec:
class: org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
tasksMax: 1
config:
camel.component.amqp.includeAmqpAnnotations: true
camel.component.amqp.connectionFactory: '#class:org.apache.qpid.jms.JmsConnectionFactory'
camel.component.amqp.connectionFactory.remoteURI: amqp://10.106.88.243:15672
camel.component.amqp.username: consumer@HONO
camel.component.amqp.password: verysecret
camel.component.amqp.testConnectionOnStartup: true
camel.source.path.destinationType: queue( my usecase i am reading from hono)
camel.source.path.destinationName: telemetry/LTTS_TENANT( this is the place where hono writing the data)
topics: mytopic (kafka topics where you want to write payload data)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751213044
What do you see in log?
Il ven 25 dic 2020, 08:26 iamrsaravana <no...@github.com> ha
scritto:
> I am planning to pull the data from Hono using Strimzi kafka(Version
> 0.20.0) connector (Camel AMQP source connector). I have followed the below
> steps to read data from Hono.
>
> I downloaded Camel-amqp-kafka-connector, JMS Jar files from below link:
>
>
> https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-amqp-kafka-connector/0.7.0/camel-amqp-kafka-connector-0.7.0-package.tar.gz
>
> https://downloads.apache.org/qpid/jms/0.51.0/apache-qpid-jms-0.51.0-bin.tar.gz
>
> After downloaded the above tar and unzipped and created docker image file
> using below command
>
> cat <Dockerfile
> FROM strimzi/kafka:0.20.1-kafka-2.6.0
> USER root:root
> RUN mkdir -p /opt/kafka/plugins/camel
> COPY ./camel-activemq-kafka-connector/* /opt/kafka/plugins/camel/
> USER 1001
> EOF
>
> Docker build -f ./Dockerfile -t localhost:5000/my-connector-amqp_new .
> docker push localhost:5000/my-connector-amqp_new
>
> using below command i have created kafkaConnect( here used my local image
> created above)
>
> apiVersion: kafka.strimzi.io/v1beta1
> kind: KafkaConnect
> metadata:
> name: my-connect-cluster
> annotations:
> strimzi.io/use-connector-resources: "true"
> spec:
> image: 10.128.0.6:5000/my-connector-amqp_new
> replicas: 1
> bootstrapServers: my-cluster-kafka-bootstrap:9092
> config:
> group.id: connect-cluster
> offset.storage.topic: connect-cluster-offsets
> config.storage.topic: connect-cluster-configs
> status.storage.topic: connect-cluster-status
> config.storage.replication.factor: 1
> offset.storage.replication.factor: 1
> status.storage.replication.factor: 1
>
> using below command to pull the data from hono message queue:
>
> apiVersion: kafka.strimzi.io/v1alpha1
> kind: KafkaConnector
> metadata:
> name: camelamqpsourceconnector
> labels:
> strimzi.io/cluster: my-connect-cluster
> spec:
> class: org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
> tasksMax: 1
> config:
> camel.component.amqp.includeAmqpAnnotations: true
> camel.component.amqp.connectionFactory.class:
> org.apache.qpid.jms.JmsConnectionFactory
> camel.component.amqp.connectionFactory.remoteURI: $( kubectl get service
> eclipse-hono-dispatch-router-ext --output="jsonpath={.spec.clusterIP}" -n
> hono):15672
> camel.component.amqp.username: consumer@HONO
> camel.component.amqp.password: verysecret
> camel.component.amqp.testConnectionOnStartup: true
> camel.source.kafka.topic: mytopic
> camel.source.path.destinationType: queue
> camel.source.path.destinationName: test-queue
>
> i am not getting any data in my above kafka topics (mytopic)
>
> Could you check is there any i need to modify to get the data in my kafka
> topics. I have searched various forum but i could not get any clear
> direction.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/camel-kafka-connector/issues/817>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABG6XV7POLF3RW5GFJKG4ZDSWQ5BTANCNFSM4VI5DDCQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana closed issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana closed issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751686430
These two paramters are source side or target side? i am pulling data from Hono(AMQP Queue) and pushing into kafka topic.
These parameter name is confusing us. whether it is source or target?
camel.source.path.destinationType:
camel.source.path.destinationName:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751687445
I don't think they are confusing. Take a look at documentation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751598608
Thanks for your reply, I Gave direct IP address like below:
**camel.component.amqp.connectionFactory: org.apache.qpid.jms.JmsConnectionFactory
camel.component.amqp.connectionFactory.remoteURI: amqp://10.106.88.243:15671**
I referred the below link for configuration properties, but I am not finding **"camel.component.amqp.connectionFactory.remoteURI"**:
https://camel.apache.org/camel-kafka-connector/latest/connectors/camel-amqp-kafka-source-connector.html
I am getting below error messages now:
org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:144)
at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:232)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:235)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalArgumentException: Cannot find getter method: connectionFactory on bean: class org.apache.camel.component.amqp.AMQPComponent when binding property: connectionFactory.remoteURI
at org.apache.camel.support.PropertyBindingSupport.doBuildPropertyOgnlPath(PropertyBindingSupport.java:282)
at org.apache.camel.support.PropertyBindingSupport.doBindProperties(PropertyBindingSupport.java:210)
at org.apache.camel.support.PropertyBindingSupport.access$100(PropertyBindingSupport.java:88)
at org.apache.camel.support.PropertyBindingSupport$Builder.bind(PropertyBindingSupport.java:1785)
at org.apache.camel.main.MainHelper.setPropertiesOnTarget(MainHelper.java:163)
at org.apache.camel.main.BaseMainSupport.autoConfigurationFromProperties(BaseMainSupport.java:1133)
at org.apache.camel.main.BaseMainSupport.autoconfigure(BaseMainSupport.java:424)
at org.apache.camel.main.BaseMainSupport.postProcessCamelContext(BaseMainSupport.java:472)
at org.apache.camel.main.SimpleMain.doInit(SimpleMain.java:32)
at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:141)
Anybody gives a suggestion how to pass source AMQP IP address in the configuration:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751688881
Thanks for confirming it is source.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-752314128
Now I am able to send the data from Hono to Strimzi Kafka Connector using Camel AMQP Source Connector.
Follow the below steps to pull data from Hono and send to Strimzi Kafka Connector:
1) Creating Name space:
kubectl create namespace kafka
2) install latest Strimzi operator using the below command:
kubectl apply -f 'https://strimzi.io/install/latest?namespace=kafka' -n kafka
3) creating Kafka (1 pod) and zookeeper(2 pods ) using the below yaml configuration:
**kafka.yaml:**
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: my-cluster
spec:
kafka:
version: 2.6.0
replicas: 1
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
config:
offsets.topic.replication.factor: 1
transaction.state.log.replication.factor: 1
transaction.state.log.min.isr: 1
log.message.format.version: "2.6"
storage:
type: jbod
volumes:
- id: 0
type: persistent-claim
size: 10Gi
deleteClaim: false
zookeeper:
replicas: 2
storage:
type: persistent-claim
size: 10Gi
deleteClaim: false
entityOperator:
topicOperator: {}
userOperator: {}
Kuebctl apply -f kafka.yaml -n kafka.
_ps: Here I use glusterfs persistent storage._
4) Creating AMQP - Source Connector image file using below commands:
I downloaded Camel-amqp-kafka-connector, JMS Jar files from below link:
https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-amqp-kafka-connector/0.7.0/camel-amqp-kafka-connector-0.7.0-package.tar.gz
https://downloads.apache.org/qpid/jms/0.51.0/apache-qpid-jms-0.51.0-bin.tar.gz
After downloaded the above tar and unzipped and created docker image file using below command
**Dockerfile:**
FROM strimzi/kafka:0.20.1-kafka-2.6.0
USER root:root
RUN mkdir -p /opt/kafka/plugins/camel-kafka-connectors/camel-amqp-kafka-connector/
COPY ./plugin/apache-qpid-jms-0.55.0/* /opt/kafka/plugins/camel-kafka-connectors/camel-amqp-kafka-connector/
COPY ./camel-amqp-kafka-connector/* /opt/kafka/plugins/camel-kafka-connectors/camel-amqp-kafka-connector/
USER 1001
Docker build -f ./Dockerfile -t localhost:5000/my-connector-amqp_new .
docker push localhost:5000/my-connector-amqp_new
_ps: Here I have used local docker repository_
5) Creating Kafkaconnect using below Yaml configuration:
Kafkaconnect.yaml:
apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaConnect
metadata:
name: my-connect-cluster-new
annotations:
strimzi.io/use-connector-resources: "true"
spec:
image: 10.128.0.6:5000/my-connector-amqp_new5
replicas: 1
bootstrapServers: my-cluster-kafka-bootstrap:9092
config:
group.id: connect-cluster-new
offset.storage.topic: connect-cluster-offsets
config.storage.topic: connect-cluster-configs
status.storage.topic: connect-cluster-status
config.storage.replication.factor: 1
offset.storage.replication.factor: 1
status.storage.replication.factor: 1
kubectl apply -f Kafkaconnect.yaml -n kafka
6) Creating AMQP-KafkaConnector using below yaml file:
amqp_connector.yaml
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
name: camelamqpsourceconnector
labels:
strimzi.io/cluster: my-connect-cluster-new
spec:
class: org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
tasksMax: 1
config:
camel.component.amqp.includeAmqpAnnotations: true
camel.component.amqp.connectionFactory: '#class:org.apache.qpid.jms.JmsConnectionFactory'
camel.component.amqp.connectionFactory.remoteURI: amqp://10.106.88.243:15672
camel.component.amqp.username: consumer@HONO
camel.component.amqp.password: verysecret
camel.component.amqp.testConnectionOnStartup: true
camel.source.path.destinationType: queue
#( my usecase i am reading from hono)
camel.source.path.destinationName: telemetry/LTTS_TENANT
# ( this is the place where hono writing the data)
topics: mytopic
#(kafka topics where you want to write it)
kubectl apply -f amqp_connector.yaml -n kafka.
ps: IP address given here all are my pod ip address. it may vary according to your pod.
7 ) viewing data in strimzi kafka using the below command:
kubectl -n kafka exec my-cluster-kafka-0 -c kafka -i -t -- bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytopic
I Hope this may be useful for somebody who wants to pull the data directly using AMQP Camel Connector.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751325339
for this command **kubectl logs deployment.apps/my-connect-cluster-connect -n kafka**
i am getting below error message:
2020-12-26 06:26:31,979 ERROR Uncaught exception in REST call to /connectors/camelamqpsourceconnector/config (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper) [qtp1240328647-22]
org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector, available connectors are: PluginDesc{klass=class org.apache.camel.kafkaconnector.CamelSinkConnector, name='org.apache.camel.kafkaconnector.CamelSinkConnector', version='0.6.1', encodedVersion=0.6.1, type=sink, typeName='sink', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.camel.kafkaconnector.CamelSourceConnector, name='org.apache.camel.kafkaconnector.CamelSourceConnector', version='0.6.1', encodedVersion=0.6.1, type=source, typeName='source', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.camel.kafkaconnector.activemq.CamelActivemqSinkConnector, name='org.apache.camel.kafkaconnector.activemq.CamelActivemqSinkConnector', version='0.6.1', encodedVersion=0.6.1, type=sink, typeName='sink', location='file:/opt/kafka/plugins/camel/'}, Plugi
nDesc{klass=class org.apache.camel.kafkaconnector.activemq.CamelActivemqSourceConnector, name='org.apache.camel.kafkaconnector.activemq.CamelActivemqSourceConnector', version='0.6.1', encodedVersion=0.6.1, type=source, typeName='source', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{kl
ass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='2.6.0', encodedVersion=2.6.0, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tool
s.MockSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}
at org.apache.kafka.connect.runtime.isolation.Plugins.connectorClass(Plugins.java:208)
at org.apache.kafka.connect.runtime.isolation.Plugins.newConnector(Plugins.java:180)
at org.apache.kafka.connect.runtime.AbstractHerder.getConnector(AbstractHerder.java:572)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:342)
at org.apache.kafka.connect.runtime.AbstractHerder.lambda$validateConnectorConfig$1(AbstractHerder.java:326)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
2020-12-26 06:28:31,877 ERROR Uncaught exception in REST call to /connectors/camelamqpsourceconnector/config (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper) [qtp1240328647-22]
org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector, available connectors are: PluginDesc{klass=class org.apache.camel.kafkaconnector.CamelSinkConnector, name='org.apache.camel.kafkaconnector.CamelSinkConnector', version='0.6.1', encodedVersion=0.6.1, type=sink, typeName='sink', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.camel.kafkaconnector.CamelSourceConnector, name='org.apache.camel.kafkaconnector.CamelSourceConnector', version='0.6.1', encodedVersion=0.6.1, type=source, typeName='source', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.camel.kafkaconnector.activemq.CamelActivemqSinkConnector, name='org.apache.camel.kafkaconnector.activemq.CamelActivemqSinkConnector', version='0.6.1', encodedVersion=0.6.1, type=sink, typeName='sink', location='file:/opt/kafka/plugins/camel/'}, Plugi
nDesc{klass=class org.apache.camel.kafkaconnector.activemq.CamelActivemqSourceConnector, name='org.apache.camel.kafkaconnector.activemq.CamelActivemqSourceConnector', version='0.6.1', encodedVersion=0.6.1, type=source, typeName='source', location='file:/opt/kafka/plugins/camel/'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{kl
ass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='2.6.0', encodedVersion=2.6.0, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tool
s.MockSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='2.6.0', encodedVersion=2.6.0, type=source, typeName='source', location='classpath'}
at org.apache.kafka.connect.runtime.isolation.Plugins.connectorClass(Plugins.java:208)
at org.apache.kafka.connect.runtime.isolation.Plugins.newConnector(Plugins.java:180)
at org.apache.kafka.connect.runtime.AbstractHerder.getConnector(AbstractHerder.java:572)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:342)
at org.apache.kafka.connect.runtime.AbstractHerder.lambda$validateConnectorConfig$1(AbstractHerder.java:326)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751353362
fixed that issue thanks for your support, i am getting the below error message now:
2020-12-26 12:50:54,474 INFO Creating task camelamqpsourceconnector-0 (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,479 INFO ConnectorConfig values:
config.action.reload = restart
connector.class = org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = camelamqpsourceconnector
predicates = []
tasks.max = 1
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.ConnectorConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,480 INFO EnrichedConnectorConfig values:
config.action.reload = restart
connector.class = org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = camelamqpsourceconnector
predicates = []
tasks.max = 1
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,481 INFO TaskConfig values:
task.class = class org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceTask
(org.apache.kafka.connect.runtime.TaskConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,481 INFO Instantiated task camelamqpsourceconnector-0 with version 0.7.0 of type org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceTask (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,482 INFO JsonConverterConfig values:
converter.type = key
decimal.format = BASE64
schemas.cache.size = 1000
schemas.enable = true
(org.apache.kafka.connect.json.JsonConverterConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,483 INFO Set up the key converter class org.apache.kafka.connect.json.JsonConverter for task camelamqpsourceconnector-0 using the worker config (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,485 INFO JsonConverterConfig values:
converter.type = value
decimal.format = BASE64
schemas.cache.size = 1000
schemas.enable = true
(org.apache.kafka.connect.json.JsonConverterConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,486 INFO Set up the value converter class org.apache.kafka.connect.json.JsonConverter for task camelamqpsourceconnector-0 using the worker config (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,486 INFO Set up the header converter class org.apache.kafka.connect.storage.SimpleHeaderConverter for task camelamqpsourceconnector-0 using the worker config (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,492 INFO SourceConnectorConfig values:
config.action.reload = restart
connector.class = org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = camelamqpsourceconnector
predicates = []
tasks.max = 1
topic.creation.groups = []
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.SourceConnectorConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,493 INFO EnrichedConnectorConfig values:
config.action.reload = restart
connector.class = org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = camelamqpsourceconnector
predicates = []
tasks.max = 1
topic.creation.groups = []
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,497 INFO Initializing: org.apache.kafka.connect.runtime.TransformationChain{} (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,500 INFO ProducerConfig values:
acks = -1
batch.size = 16384
bootstrap.servers = [my-cluster-kafka-bootstrap:9092]
buffer.memory = 33554432
client.dns.lookup = use_all_dns_ips
client.id = connector-producer-camelamqpsourceconnector-0
compression.type = none
connections.max.idle.ms = 540000
delivery.timeout.ms = 2147483647
enable.idempotence = false
interceptor.classes = []
internal.auto.downgrade.txn.commit = false
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms = 0
max.block.ms = 9223372036854775807
max.in.flight.requests.per.connection = 1
max.request.size = 1048576
metadata.max.age.ms = 300000
metadata.max.idle.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 2147483647
retries = 2147483647
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
ssl.endpoint.identification.algorithm = https
ssl.engine.factory.class = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLSv1.3
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
(org.apache.kafka.clients.producer.ProducerConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,517 WARN The configuration 'metrics.context.connect.group.id' was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,518 WARN The configuration 'metrics.context.connect.kafka.cluster.id' was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,518 INFO Kafka version: 2.6.0 (org.apache.kafka.common.utils.AppInfoParser) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,518 INFO Kafka commitId: 62abe01bee039651 (org.apache.kafka.common.utils.AppInfoParser) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,518 INFO Kafka startTimeMs: 1608987054518 (org.apache.kafka.common.utils.AppInfoParser) [StartAndStopExecutor-connect-1-2]
2020-12-26 12:50:54,543 INFO [Worker clientId=connect-1, groupId=connect-cluster-new] Finished starting connectors and tasks (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [DistributedHerder-connect-1-1]
2020-12-26 12:50:54,600 INFO Starting CamelSourceTask connector task (org.apache.camel.kafkaconnector.CamelSourceTask) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:54,655 INFO [Producer clientId=connector-producer-camelamqpsourceconnector-0] Cluster ID: AxvFw7iiSDCEUx-RwUy0gw (org.apache.kafka.clients.Metadata) [kafka-producer-network-thread | connector-producer-camelamqpsourceconnector-0]
2020-12-26 12:50:54,660 INFO CamelAmqpSourceConnectorConfig values:
camel.aggregation.size = 10
camel.aggregation.timeout = 500
camel.beans.aggregate = null
camel.component.amqp.acceptMessagesWhileStopping = false
camel.component.amqp.acknowledgementModeName = AUTO_ACKNOWLEDGE
camel.component.amqp.allowAutoWiredConnectionFactory = true
camel.component.amqp.allowAutoWiredDestinationResolver = true
camel.component.amqp.allowReplyManagerQuickStop = false
camel.component.amqp.allowSerializedHeaders = false
camel.component.amqp.artemisStreamingEnabled = true
camel.component.amqp.asyncConsumer = false
camel.component.amqp.asyncStartListener = false
camel.component.amqp.asyncStopListener = false
camel.component.amqp.autoStartup = true
camel.component.amqp.autowiredEnabled = true
camel.component.amqp.cacheLevel = null
camel.component.amqp.cacheLevelName = CACHE_AUTO
camel.component.amqp.clientId = null
camel.component.amqp.concurrentConsumers = 1
camel.component.amqp.configuration = null
camel.component.amqp.connectionFactory = org.apache.qpid.jms.JmsConnectionFactory
camel.component.amqp.consumerType = Default
camel.component.amqp.defaultTaskExecutorType = null
camel.component.amqp.destinationResolver = null
camel.component.amqp.disableReplyTo = false
camel.component.amqp.durableSubscriptionName = null
camel.component.amqp.eagerLoadingOfProperties = false
camel.component.amqp.eagerPoisonBody = Poison JMS message due to ${exception.message}
camel.component.amqp.errorHandler = null
camel.component.amqp.errorHandlerLogStackTrace = true
camel.component.amqp.errorHandlerLoggingLevel = WARN
camel.component.amqp.exceptionListener = null
camel.component.amqp.exposeListenerSession = false
camel.component.amqp.headerFilterStrategy = null
camel.component.amqp.idleConsumerLimit = 1
camel.component.amqp.idleTaskExecutionLimit = 1
camel.component.amqp.includeAllJMSXProperties = false
camel.component.amqp.includeAmqpAnnotations = true
camel.component.amqp.jmsKeyFormatStrategy = null
camel.component.amqp.jmsMessageType = null
camel.component.amqp.lazyCreateTransactionManager = true
camel.component.amqp.mapJmsMessage = true
camel.component.amqp.maxConcurrentConsumers = null
camel.component.amqp.maxMessagesPerTask = -1
camel.component.amqp.messageConverter = null
camel.component.amqp.messageCreatedStrategy = null
camel.component.amqp.messageIdEnabled = true
camel.component.amqp.messageListenerContainerFactory = null
camel.component.amqp.messageTimestampEnabled = true
camel.component.amqp.password = verysecret
camel.component.amqp.pubSubNoLocal = false
camel.component.amqp.queueBrowseStrategy = null
camel.component.amqp.receiveTimeout = 1000
camel.component.amqp.recoveryInterval = 5000
camel.component.amqp.replyTo = null
camel.component.amqp.replyToDeliveryPersistent = true
camel.component.amqp.replyToSameDestinationAllowed = false
camel.component.amqp.requestTimeoutCheckerInterval = 1000
camel.component.amqp.selector = null
camel.component.amqp.subscriptionDurable = false
camel.component.amqp.subscriptionName = null
camel.component.amqp.subscriptionShared = false
camel.component.amqp.taskExecutor = null
camel.component.amqp.testConnectionOnStartup = true
camel.component.amqp.transacted = false
camel.component.amqp.transactedInOut = false
camel.component.amqp.transactionManager = null
camel.component.amqp.transactionName = null
camel.component.amqp.transactionTimeout = -1
camel.component.amqp.transferException = false
camel.component.amqp.transferExchange = false
camel.component.amqp.useMessageIDAsCorrelationID = false
camel.component.amqp.username = consumer@HONO
camel.component.amqp.waitForProvisionCorrelationToBeUpdatedCounter = 50
camel.component.amqp.waitForProvisionCorrelationToBeUpdatedThreadSleepingTime = 100
camel.error.handler = default
camel.error.handler.max.redeliveries = 0
camel.error.handler.redelivery.delay = 1000
camel.idempotency.enabled = false
camel.idempotency.expression.header = null
camel.idempotency.expression.type = body
camel.idempotency.kafka.bootstrap.servers = localhost:9092
camel.idempotency.kafka.max.cache.size = 1000
camel.idempotency.kafka.poll.duration.ms = 100
camel.idempotency.kafka.topic = kafka_idempotent_repository
camel.idempotency.memory.dimension = 100
camel.idempotency.repository.type = memory
camel.remove.headers.pattern =
camel.source.camelMessageHeaderKey = null
camel.source.component = amqp
camel.source.contentLogLevel = OFF
camel.source.endpoint.acceptMessagesWhileStopping = false
camel.source.endpoint.acknowledgementModeName = AUTO_ACKNOWLEDGE
camel.source.endpoint.allowReplyManagerQuickStop = false
camel.source.endpoint.allowSerializedHeaders = false
camel.source.endpoint.artemisStreamingEnabled = true
camel.source.endpoint.asyncConsumer = false
camel.source.endpoint.asyncStartListener = false
camel.source.endpoint.asyncStopListener = false
camel.source.endpoint.autoStartup = true
camel.source.endpoint.cacheLevel = null
camel.source.endpoint.cacheLevelName = CACHE_AUTO
camel.source.endpoint.clientId = null
camel.source.endpoint.concurrentConsumers = 1
camel.source.endpoint.connectionFactory = null
camel.source.endpoint.consumerType = Default
camel.source.endpoint.defaultTaskExecutorType = null
camel.source.endpoint.destinationResolver = null
camel.source.endpoint.disableReplyTo = false
camel.source.endpoint.durableSubscriptionName = null
camel.source.endpoint.eagerLoadingOfProperties = false
camel.source.endpoint.eagerPoisonBody = Poison JMS message due to ${exception.message}
camel.source.endpoint.errorHandler = null
camel.source.endpoint.errorHandlerLogStackTrace = true
camel.source.endpoint.errorHandlerLoggingLevel = WARN
camel.source.endpoint.exceptionHandler = null
camel.source.endpoint.exceptionListener = null
camel.source.endpoint.exchangePattern = null
camel.source.endpoint.exposeListenerSession = false
camel.source.endpoint.headerFilterStrategy = null
camel.source.endpoint.idleConsumerLimit = 1
camel.source.endpoint.idleTaskExecutionLimit = 1
camel.source.endpoint.includeAllJMSXProperties = false
camel.source.endpoint.jmsKeyFormatStrategy = null
camel.source.endpoint.jmsMessageType = null
camel.source.endpoint.lazyCreateTransactionManager = true
camel.source.endpoint.mapJmsMessage = true
camel.source.endpoint.maxConcurrentConsumers = null
camel.source.endpoint.maxMessagesPerTask = -1
camel.source.endpoint.messageConverter = null
camel.source.endpoint.messageCreatedStrategy = null
camel.source.endpoint.messageIdEnabled = true
camel.source.endpoint.messageListenerContainerFactory = null
camel.source.endpoint.messageTimestampEnabled = true
camel.source.endpoint.password = null
camel.source.endpoint.pubSubNoLocal = false
camel.source.endpoint.receiveTimeout = 1000
camel.source.endpoint.recoveryInterval = 5000
camel.source.endpoint.replyTo = null
camel.source.endpoint.replyToDeliveryPersistent = true
camel.source.endpoint.replyToSameDestinationAllowed = false
camel.source.endpoint.requestTimeoutCheckerInterval = 1000
camel.source.endpoint.selector = null
camel.source.endpoint.subscriptionDurable = false
camel.source.endpoint.subscriptionName = null
camel.source.endpoint.subscriptionShared = false
camel.source.endpoint.synchronous = false
camel.source.endpoint.taskExecutor = null
camel.source.endpoint.testConnectionOnStartup = false
camel.source.endpoint.transacted = false
camel.source.endpoint.transactedInOut = false
camel.source.endpoint.transactionManager = null
camel.source.endpoint.transactionName = null
camel.source.endpoint.transactionTimeout = -1
camel.source.endpoint.transferException = false
camel.source.endpoint.transferExchange = false
camel.source.endpoint.useMessageIDAsCorrelationID = false
camel.source.endpoint.username = null
camel.source.endpoint.waitForProvisionCorrelationToBeUpdatedCounter = 50
camel.source.endpoint.waitForProvisionCorrelationToBeUpdatedThreadSleepingTime = 100
camel.source.marshal = null
camel.source.maxBatchPollSize = 1000
camel.source.maxPollDuration = 1000
camel.source.path.destinationName = test-queue
camel.source.path.destinationType = queue
camel.source.pollingConsumerBlockTimeout = 0
camel.source.pollingConsumerBlockWhenFull = true
camel.source.pollingConsumerQueueSize = 1000
camel.source.unmarshal = null
camel.source.url = null
topics = test
(org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnectorConfig) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:54,905 INFO Setting initial properties in Camel context: [{camel.source.path.destinationName=test-queue, connector.class=org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector, camel.component.amqp.username=consumer@HONO, tasks.max=1, camel.component.amqp.connectionFactory=org.apache.qpid.jms.JmsConnectionFactory, camel.source.component=amqp, camel.component.amqp.testConnectionOnStartup=true, camel.component.amqp.connectionFactory.remoteURI=amqp://$( kubectl get service eclipse-hono-dispatch-router-ext --output="jsonpath={.spec.clusterIP}" -n hono):15671, camel.component.amqp.password=verysecret, camel.component.amqp.includeAmqpAnnotations=true, camel.source.kafka.topic=mytopic, camel.source.path.destinationType=queue, task.class=org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceTask, name=camelamqpsourceconnector}] (org.apache.camel.kafkaconnector.utils.CamelKafkaConnectMain) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:54,985 INFO Using properties from: classpath:application.properties;optional=true (org.apache.camel.main.BaseMainSupport) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:55,175 INFO WorkerSourceTask{id=camelamqpsourceconnector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:55,175 INFO WorkerSourceTask{id=camelamqpsourceconnector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:55,176 ERROR WorkerSourceTask{id=camelamqpsourceconnector-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-camelamqpsourceconnector-0]
org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:144)
at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:232)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:235)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalArgumentException: Cannot find getter method: connectionFactory on bean: class org.apache.camel.component.amqp.AMQPComponent when binding property: connectionFactory.remoteURI
at org.apache.camel.support.PropertyBindingSupport.doBuildPropertyOgnlPath(PropertyBindingSupport.java:282)
at org.apache.camel.support.PropertyBindingSupport.doBindProperties(PropertyBindingSupport.java:210)
at org.apache.camel.support.PropertyBindingSupport.access$100(PropertyBindingSupport.java:88)
at org.apache.camel.support.PropertyBindingSupport$Builder.bind(PropertyBindingSupport.java:1785)
at org.apache.camel.main.MainHelper.setPropertiesOnTarget(MainHelper.java:163)
at org.apache.camel.main.BaseMainSupport.autoConfigurationFromProperties(BaseMainSupport.java:1133)
at org.apache.camel.main.BaseMainSupport.autoconfigure(BaseMainSupport.java:424)
at org.apache.camel.main.BaseMainSupport.postProcessCamelContext(BaseMainSupport.java:472)
at org.apache.camel.main.SimpleMain.doInit(SimpleMain.java:32)
at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:141)
... 8 more
2020-12-26 12:50:55,176 ERROR WorkerSourceTask{id=camelamqpsourceconnector-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:55,177 INFO Stopping CamelSourceTask connector task (org.apache.camel.kafkaconnector.CamelSourceTask) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:55,177 INFO CamelSourceTask connector task stopped (org.apache.camel.kafkaconnector.CamelSourceTask) [task-thread-camelamqpsourceconnector-0]
2020-12-26 12:50:55,177 INFO [Producer clientId=connector-producer-camelamqpsourceconnector-0] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer) [task-thread-camelamqpsourceconnector-0]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751687061
If you read source, the options are related to source connector. With target do you mean sink? Are we talking about kafka connect framework or do you mean something else?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana closed issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana closed issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751356435
The property you're setting through kubectl is not resolved. So the connection factory cannot be instantiated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751336478
It seems you have no connector for amqp installed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751365617
I think you need to explicitly add the address of the hono in the amqp:// scheme. Kubetctl won't be resolved in the configuration.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana edited a comment on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana edited a comment on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751365170
I was referring the below link and gave my inputs
https://github.com/apache/camel-kafka-connector/blob/master/examples/CamelAmqpSourceConnector.properties
Values are in this file as mentioned it below:
name=CamelAmqpSourceConnector
topics=mytopic
tasks.max=1
connector.class=org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
camel.source.path.destinationType=queue
camel.source.path.destinationName=test-queue
camel.component.amqp.includeAmqpAnnotations=true
**camel.component.amqp.connectionFactory=#class:org.apache.qpid.jms.JmsConnectionFactory
camel.component.amqp.connectionFactory.remoteURI=amqp://localhost:5672**
camel.component.amqp.username=admin
camel.component.amqp.password=admin
camel.component.amqp.testConnectionOnStartup=true
I am using the below configurations. Could you suggest me what is the correct value for this property?
**camel.component.amqp.connectionFactory**
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
name: camelamqpsourceconnector
labels:
strimzi.io/cluster: my-connect-cluster-new
spec:
class: org.apache.camel.kafkaconnector.amqp.CamelAmqpSourceConnector
tasksMax: 1
config:
camel.component.amqp.includeAmqpAnnotations: true
**camel.component.amqp.connectionFactory: org.apache.qpid.jms.JmsConnectionFactory
camel.component.amqp.connectionFactory.remoteURI: amqp://$( kubectl get service eclipse-hono-dispatch-router-ext --output="jsonpath={.spec.clusterIP}" -n hono):15671**
camel.component.amqp.username: consumer@HONO
camel.component.amqp.password: verysecret
camel.component.amqp.testConnectionOnStartup: true
camel.source.kafka.topic: mytopic
camel.source.path.destinationType: queue
camel.source.path.destinationName: test-queue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751634533
It should be
https://github.com/apache/camel-kafka-connector-examples/blob/master/amqp/config/CamelAmqpSourceConnector.properties#L27
prepend #class
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751647891
I guess you need to include the string in ' xxx '
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] iamrsaravana commented on issue #817: Reading data from AMQP protocal using Strimzi kafka connector
Posted by GitBox <gi...@apache.org>.
iamrsaravana commented on issue #817:
URL: https://github.com/apache/camel-kafka-connector/issues/817#issuecomment-751687717
These parameter name is confusing us. whether it is source or target?
camel.source.path.destinationType:
camel.source.path.destinationName:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org