You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Chris Snow <ch...@gmail.com> on 2017/08/19 03:43:59 UTC
Camel Kafka SASL plain with broker list rather than zookeeper
IBM MessageHub is a kafka 0.10.2.1 service that uses SASL plain to connect.
The connection parameters require passing a list of bootstrap servers and a
user name and password:
"kafka_brokers_sasl": [
"kafka04-prod01.messagehub.services.eu-de.bluemix.net:9093",
"kafka05-prod01.messagehub.services.eu-de.bluemix.net:9093",
"kafka01-prod01.messagehub.services.eu-de.bluemix.net:9093",
"kafka03-prod01.messagehub.services.eu-de.bluemix.net:9093",
"kafka02-prod01.messagehub.services.eu-de.bluemix.net:9093"
],
"user": "xxxxx",
"password": "xxxxx"
Note that the connection uses the broker list because zookeeper is not
exposed.
I couldn't see from the camel documentation how to connect with only the
above information - is this possible?
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Kafka-SASL-plain-with-broker-list-rather-than-zookeeper-tp5811191.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Kafka SASL plain with broker list rather than zookeeper
Posted by Chris Snow <ch...@gmail.com>.
I was wondering whether camel should also support consumer.properties and
producer.properties files as a configuration mechanism? This would allow
developers to easily reuse their existing assets for connecting to kafka.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Kafka-SASL-plain-with-broker-list-rather-than-zookeeper-tp5811191p5811347.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Kafka SASL plain with broker list rather than zookeeper
Posted by Chris Snow <ch...@gmail.com>.
I've raised a JIRA to track this:
https://issues.apache.org/jira/browse/CAMEL-11682
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Kafka-SASL-plain-with-broker-list-rather-than-zookeeper-tp5811191p5811365.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Kafka SASL plain with broker list rather than zookeeper
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Chris,
yeah, they are comma separated (see the source[1]), for the
`sasl.mechanism` and `sasl.jaas.config` I don't see support for them
in the source code, you're welcome to file an issue or better yet
contribute via pull request support for those, perhaps we could also
have a generic way of adding any number of properties in the
KafkaConfiguration[2],
zoran
[1] https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/KafkaComponent.java#L93-L98
[2] https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/KafkaConfiguration.java
On Sun, Aug 20, 2017 at 10:33 AM, Chris Snow <ch...@gmail.com> wrote:
> Thanks Zoran, from the documentation it appears that I need to pass the
> parameters:
>
> brokers=kafka01-prod01.messagehub.services.eu-de.bluemix.net:9093kafka02-prod01.messagehub.services.eu-de.bluemix.net:9093
> ...
> saslMechanism=PLAIN
> securityProtocol=SASL_SSL
> sslProtocol=TLSv1.2
> sslEnabledProtocols=TLSv1.2
> sslEndpointAlgorithm=HTTPS
>
> Strangely, there isn't a comma separating the brokers in the documentation.
> I'm not sure if that is a documentation bug?
>
> Also, I'm still not clear how to set the username and password. In my
> producer properties, I would use:
>
> sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
> required username="USERNAME" password="PASSWORD";
>
> See here for a producer.properties sample:
> https://github.com/ibm-messaging/message-hub-samples/blob/master/kafka-java-console-sample/resources/producer.properties
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Kafka-SASL-plain-with-broker-list-rather-than-zookeeper-tp5811191p5811345.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Zoran Regvart
Re: Camel Kafka SASL plain with broker list rather than zookeeper
Posted by Chris Snow <ch...@gmail.com>.
Thanks Zoran, from the documentation it appears that I need to pass the
parameters:
brokers=kafka01-prod01.messagehub.services.eu-de.bluemix.net:9093kafka02-prod01.messagehub.services.eu-de.bluemix.net:9093
...
saslMechanism=PLAIN
securityProtocol=SASL_SSL
sslProtocol=TLSv1.2
sslEnabledProtocols=TLSv1.2
sslEndpointAlgorithm=HTTPS
Strangely, there isn't a comma separating the brokers in the documentation.
I'm not sure if that is a documentation bug?
Also, I'm still not clear how to set the username and password. In my
producer properties, I would use:
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="USERNAME" password="PASSWORD";
See here for a producer.properties sample:
https://github.com/ibm-messaging/message-hub-samples/blob/master/kafka-java-console-sample/resources/producer.properties
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Kafka-SASL-plain-with-broker-list-rather-than-zookeeper-tp5811191p5811345.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Kafka SASL plain with broker list rather than zookeeper
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Chris,
the documentation on the website (wiki) is slightly out of date, you
can find the latest documentation in the source code[1]. The Kafka
component has support for `brokers` component/endpoint property see
CAMEL-10832[2].
zoran
[1] https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
[2] https://issues.apache.org/jira/browse/CAMEL-10832
On Sat, Aug 19, 2017 at 5:43 AM, Chris Snow <ch...@gmail.com> wrote:
> IBM MessageHub is a kafka 0.10.2.1 service that uses SASL plain to connect.
> The connection parameters require passing a list of bootstrap servers and a
> user name and password:
>
> "kafka_brokers_sasl": [
> "kafka04-prod01.messagehub.services.eu-de.bluemix.net:9093",
> "kafka05-prod01.messagehub.services.eu-de.bluemix.net:9093",
> "kafka01-prod01.messagehub.services.eu-de.bluemix.net:9093",
> "kafka03-prod01.messagehub.services.eu-de.bluemix.net:9093",
> "kafka02-prod01.messagehub.services.eu-de.bluemix.net:9093"
> ],
> "user": "xxxxx",
> "password": "xxxxx"
>
> Note that the connection uses the broker list because zookeeper is not
> exposed.
>
> I couldn't see from the camel documentation how to connect with only the
> above information - is this possible?
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Kafka-SASL-plain-with-broker-list-rather-than-zookeeper-tp5811191.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Zoran Regvart