You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "Berryman, Eric" <be...@frib.msu.edu> on 2016/11/15 21:07:38 UTC
jms to kafka to jms
Hello!
I have two routes; one from jms(BytesMessage) to kafka, and one from kafka to jms(BytesMessage).
When I received my jms object at the end, I noticed it is a TextMessage instead of BytesMessage.
from("jms:topic:jmsTopic")
.to("kafka:kafkaserver:9092?topic=jpa-cache" + "&requestRequiredAcks=-1"
+ "&serializerClass=org.apache.kafka.common.serialization.ByteArraySerializer&"
+ "keySerializerClass=org.apache.kafka.common.serialization.ByteArraySerializer");
from("kafka:kafkaserver:9092?topic=jpa-cache&groupId=cache&autoOffsetReset=earliest&consumersCount=1")
.to("jms:topic:jmsTopic");
Could someone help me preserve the BytesMessage?
Thank you!
Eric
Re: jms to kafka to jms
Posted by Tomohisa Igarashi <tm...@gmail.com>.
Hi,
I played with this one. So it looks you need to specify ByteArrayDeserializer at kafka consumer side
https://github.com/igarashitm/issues/blob/master/camel/misc/src/test/java/CamelJmsKafkaBytesMessageTest.java
https://github.com/igarashitm/issues/blob/master/camel/misc/src/test/java/CamelJmsKafkaBytesMessageTest.java#L111-L112
Otherwise StringDeserializer is used anyway.
Thanks,
Tomo
On 11/16/2016 06:07 AM, Berryman, Eric wrote:
> Hello!
>
> I have two routes; one from jms(BytesMessage) to kafka, and one from kafka to jms(BytesMessage).
> When I received my jms object at the end, I noticed it is a TextMessage instead of BytesMessage.
>
> from("jms:topic:jmsTopic")
> .to("kafka:kafkaserver:9092?topic=jpa-cache" + "&requestRequiredAcks=-1"
> + "&serializerClass=org.apache.kafka.common.serialization.ByteArraySerializer&"
> + "keySerializerClass=org.apache.kafka.common.serialization.ByteArraySerializer");
>
> from("kafka:kafkaserver:9092?topic=jpa-cache&groupId=cache&autoOffsetReset=earliest&consumersCount=1")
> .to("jms:topic:jmsTopic");
>
> Could someone help me preserve the BytesMessage?
>
> Thank you!
> Eric
>
>
Re: jms to kafka to jms
Posted by Willem Jiang <wi...@gmail.com>.
Hi,
I'm not sure if it relates to the deserializerClass option setting of kafka
consumer, as you didn't set the option on the kafka endpoint of the second.
BTW, You can use the *jmsMessageType* option on the jms endpoint URL to
force a specific message type for all messages.
Willem Jiang
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Wed, Nov 16, 2016 at 5:07 AM, Berryman, Eric <be...@frib.msu.edu>
wrote:
> Hello!
>
> I have two routes; one from jms(BytesMessage) to kafka, and one from kafka
> to jms(BytesMessage).
> When I received my jms object at the end, I noticed it is a TextMessage
> instead of BytesMessage.
>
> from("jms:topic:jmsTopic")
> .to("kafka:kafkaserver:9092?topic=jpa-cache" +
> "&requestRequiredAcks=-1"
> + "&serializerClass=org.apache.
> kafka.common.serialization.ByteArraySerializer&"
> + "keySerializerClass=org.
> apache.kafka.common.serialization.ByteArraySerializer");
>
> from("kafka:kafkaserver:9092?topic=jpa-cache&groupId=cache&
> autoOffsetReset=earliest&consumersCount=1")
> .to("jms:topic:jmsTopic");
>
> Could someone help me preserve the BytesMessage?
>
> Thank you!
> Eric
>
>
RE: jms to kafka to jms
Posted by "Berryman, Eric" <be...@frib.msu.edu>.
The ByteArrayDeserializer at the consumer worked!
Thank you!
-----Original Message-----
From: Willem Jiang [mailto:willem.jiang@gmail.com]
Sent: Tuesday, November 15, 2016 11:46 PM
To: users@camel.apache.org
Subject: Re: jms to kafka to jms
Hi,
I'm not sure if it relates to the deserializerClass option setting of kafka consumer, as you didn't set the option on the kafka endpoint of the second.
BTW, You can use the *jmsMessageType* option on the jms endpoint URL to force a specific message type for all messages.
Willem Jiang
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Wed, Nov 16, 2016 at 5:07 AM, Berryman, Eric <be...@frib.msu.edu>
wrote:
> Hello!
>
> I have two routes; one from jms(BytesMessage) to kafka, and one from
> kafka to jms(BytesMessage).
> When I received my jms object at the end, I noticed it is a
> TextMessage instead of BytesMessage.
>
> from("jms:topic:jmsTopic")
> .to("kafka:kafkaserver:9092?topic=jpa-cache" +
> "&requestRequiredAcks=-1"
> + "&serializerClass=org.apache.
> kafka.common.serialization.ByteArraySerializer&"
> + "keySerializerClass=org.
> apache.kafka.common.serialization.ByteArraySerializer");
>
> from("kafka:kafkaserver:9092?topic=jpa-cache&groupId=cache&
> autoOffsetReset=earliest&consumersCount=1")
> .to("jms:topic:jmsTopic");
>
> Could someone help me preserve the BytesMessage?
>
> Thank you!
> Eric
>
>
RE: jms to kafka to jms
Posted by "Berryman, Eric" <be...@frib.msu.edu>.
Ah!
Thank you! This worked.
-----Original Message-----
From: Tomohisa Igarashi [mailto:tm.igarashi@gmail.com]
Sent: Wednesday, November 16, 2016 7:47 AM
To: users@camel.apache.org
Subject: Re: jms to kafka to jms
Hi,
I played with this one. So it looks you need to specify ByteArrayDeserializer at kafka consumer side https://github.com/igarashitm/issues/blob/master/camel/misc/src/test/java/CamelJmsKafkaBytesMessageTest.java
https://github.com/igarashitm/issues/blob/master/camel/misc/src/test/java/CamelJmsKafkaBytesMessageTest.java#L111-L112
Otherwise StringDeserializer is used anyway.
Thanks,
Tomo
On 11/16/2016 06:07 AM, Berryman, Eric wrote:
> Hello!
>
> I have two routes; one from jms(BytesMessage) to kafka, and one from kafka to jms(BytesMessage).
> When I received my jms object at the end, I noticed it is a TextMessage instead of BytesMessage.
>
> from("jms:topic:jmsTopic")
> .to("kafka:kafkaserver:9092?topic=jpa-cache" + "&requestRequiredAcks=-1"
> + "&serializerClass=org.apache.kafka.common.serialization.ByteArraySerializer&"
> +
> "keySerializerClass=org.apache.kafka.common.serialization.ByteArraySer
> ializer");
>
> from("kafka:kafkaserver:9092?topic=jpa-cache&groupId=cache&autoOffsetReset=earliest&consumersCount=1")
> .to("jms:topic:jmsTopic");
>
> Could someone help me preserve the BytesMessage?
>
> Thank you!
> Eric
>
>