You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Alan Robson <al...@viasat.com> on 2015/04/01 21:13:00 UTC
kafka producer - multiple brokers
Please excuse my lack of experience with Kafka, but I understood that for
redundancy purposes, a cluster would generally contain multiple brokers and
that a producer woudl try to connect to any of a list of them.
If that's true, how might I tell Camel about the availability of multiple
brokers. I only see an opportunity to put a single broker in the URI
(192.168.0.1:9092 in this case) ?
For example if myblueprint contains...
<to
uri="kafka:192.168.0.1:9092?producerType=sync&zookeeperHost=192.168.0.2&zookeeperPort=2181&topic=test&partitioner=camel.myPartitioner&serializerClass=kafka.serializer.DefaultEncoder&keySerializerClass=kafka.serializer.StringEncoder"
/>
Many thanks
Alan
--
View this message in context: http://camel.465427.n5.nabble.com/kafka-producer-multiple-brokers-tp5765174.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: kafka producer - multiple brokers
Posted by Alan Robson <al...@viasat.com>.
My current thinking is to try leaving the code as-is, create multiple kafka
endpoints (one per broker) and then use the load-balancer EIP processor to
determine which endpoint(s) to send traffic through.
--
View this message in context: http://camel.465427.n5.nabble.com/kafka-producer-multiple-brokers-tp5765174p5765330.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: kafka producer - multiple brokers
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Ah good, yeah maybe that can be used the same on the producer side.
The documentation really need a bit of love.
Anyway wanna help?
http://camel.apache.org/how-do-i-edit-the-website.html
On Sun, Apr 5, 2015 at 6:10 AM, Derek Bassett
<de...@thistech.com> wrote:
> So we use the Kafka Consumer with multiple brokers
>
> X.X.X.X:9092,X.X.X.X:9092,X.X.XX.X:9092
>
> without any issue.
>
> Derek
>
>> On Apr 4, 2015, at 8:23 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>
>> Hi
>>
>> I am not sure if the current camel-kafka supports this. You are
>> welcome to dive into the source code.
>> And if its not then welcome to log a JIRA and work on a patch.
>> Also the docs could use some love.
>>
>> You can read here about how to help
>> http://camel.apache.org/contributing.html
>>
>> On Wed, Apr 1, 2015 at 9:13 PM, Alan Robson <al...@viasat.com> wrote:
>>> Please excuse my lack of experience with Kafka, but I understood that for
>>> redundancy purposes, a cluster would generally contain multiple brokers and
>>> that a producer woudl try to connect to any of a list of them.
>>>
>>> If that's true, how might I tell Camel about the availability of multiple
>>> brokers. I only see an opportunity to put a single broker in the URI
>>> (192.168.0.1:9092 in this case) ?
>>>
>>> For example if myblueprint contains...
>>>
>>> <to
>>> uri="kafka:192.168.0.1:9092?producerType=sync&zookeeperHost=192.168.0.2&zookeeperPort=2181&topic=test&partitioner=camel.myPartitioner&serializerClass=kafka.serializer.DefaultEncoder&keySerializerClass=kafka.serializer.StringEncoder"
>>> />
>>>
>>>
>>> Many thanks
>>>
>>> Alan
>>>
>>>
>>>
>>> --
>>> View this message in context: http://camel.465427.n5.nabble.com/kafka-producer-multiple-brokers-tp5765174.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> Red Hat, Inc.
>> Email: cibsen@redhat.com
>> Twitter: davsclaus
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
>> hawtio: http://hawt.io/
>> fabric8: http://fabric8.io/
>
> The information contained in this e-mail message is the confidential information of THIS TECHNOLOGY, Inc. and is intended for use only by the individual or entity above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or duplication of this message is strictly prohibited. If you have received this message in error, please immediately notify us by telephone and return the original message to us. Thank you.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
Re: kafka producer - multiple brokers
Posted by Derek Bassett <de...@thistech.com>.
So we use the Kafka Consumer with multiple brokers
X.X.X.X:9092,X.X.X.X:9092,X.X.XX.X:9092
without any issue.
Derek
> On Apr 4, 2015, at 8:23 AM, Claus Ibsen <cl...@gmail.com> wrote:
>
> Hi
>
> I am not sure if the current camel-kafka supports this. You are
> welcome to dive into the source code.
> And if its not then welcome to log a JIRA and work on a patch.
> Also the docs could use some love.
>
> You can read here about how to help
> http://camel.apache.org/contributing.html
>
> On Wed, Apr 1, 2015 at 9:13 PM, Alan Robson <al...@viasat.com> wrote:
>> Please excuse my lack of experience with Kafka, but I understood that for
>> redundancy purposes, a cluster would generally contain multiple brokers and
>> that a producer woudl try to connect to any of a list of them.
>>
>> If that's true, how might I tell Camel about the availability of multiple
>> brokers. I only see an opportunity to put a single broker in the URI
>> (192.168.0.1:9092 in this case) ?
>>
>> For example if myblueprint contains...
>>
>> <to
>> uri="kafka:192.168.0.1:9092?producerType=sync&zookeeperHost=192.168.0.2&zookeeperPort=2181&topic=test&partitioner=camel.myPartitioner&serializerClass=kafka.serializer.DefaultEncoder&keySerializerClass=kafka.serializer.StringEncoder"
>> />
>>
>>
>> Many thanks
>>
>> Alan
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/kafka-producer-multiple-brokers-tp5765174.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: cibsen@redhat.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
> hawtio: http://hawt.io/
> fabric8: http://fabric8.io/
The information contained in this e-mail message is the confidential information of THIS TECHNOLOGY, Inc. and is intended for use only by the individual or entity above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or duplication of this message is strictly prohibited. If you have received this message in error, please immediately notify us by telephone and return the original message to us. Thank you.
Re: kafka producer - multiple brokers
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I am not sure if the current camel-kafka supports this. You are
welcome to dive into the source code.
And if its not then welcome to log a JIRA and work on a patch.
Also the docs could use some love.
You can read here about how to help
http://camel.apache.org/contributing.html
On Wed, Apr 1, 2015 at 9:13 PM, Alan Robson <al...@viasat.com> wrote:
> Please excuse my lack of experience with Kafka, but I understood that for
> redundancy purposes, a cluster would generally contain multiple brokers and
> that a producer woudl try to connect to any of a list of them.
>
> If that's true, how might I tell Camel about the availability of multiple
> brokers. I only see an opportunity to put a single broker in the URI
> (192.168.0.1:9092 in this case) ?
>
> For example if myblueprint contains...
>
> <to
> uri="kafka:192.168.0.1:9092?producerType=sync&zookeeperHost=192.168.0.2&zookeeperPort=2181&topic=test&partitioner=camel.myPartitioner&serializerClass=kafka.serializer.DefaultEncoder&keySerializerClass=kafka.serializer.StringEncoder"
> />
>
>
> Many thanks
>
> Alan
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/kafka-producer-multiple-brokers-tp5765174.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/