You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Nixon Rodrigues <ni...@freestoneinfotech.com> on 2017/05/01 13:04:30 UTC
Support for Kafka Consumer over SASL_SSL
Hi Kafka dev team,
I am using kafka_2.11-0.10.0.0.jar api for kafka consumer,
i am facing issue while consuming from topics over SASL_SSL,
getting below exception
Consumer.createJavaConsumerConnector(new
kafka.consumer.ConsumerConfig(consumerProperties))
As per documentation found on
http://docs.confluent.io/2.0.0/kafka/sasl.html , it is said that
"SASL authentication is only supported for the new Kafka producer
and consumer, the older API is not supported."
Can anybody confirm me which kafka version onward SASl_SSL is
supported, any input on this is appreciated.
2017-04-26 16:06:36,655 WARN -
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread:] ~
Fetching topic metadata with correlation id 18 for topics [Set(TEST)]
from broker [BrokerEndPoint(1001,node11.openstacklocal,6667)] failed
(Logging$class:89)
java.io.EOFException
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:140)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:131)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:84)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:81)
at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
2017-04-26 16:06:36,655 WARN -
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread:] ~
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread],
Failed to find leader for Set([TEST,0]) (Logging$class:89)
kafka.common.KafkaException: fetching topic metadata for topics
[Set(TEST)] from broker
[ArrayBuffer(BrokerEndPoint(1001,node11.openstacklocal,6667))] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
Caused by: java.io.EOFException
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:140)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:131)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:84)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:81)
at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
Nixon
Re: Support for Kafka Consumer over SASL_SSL
Posted by Michal Borowiecki <mi...@openbet.com>.
Hi Nixon,
kafka.consumer.Consumer is the old consumer.
When the documentation is referring to the new producer and consumer, it
means those in the org.apache.kafka.clients.producer and
org.apache.kafka.clients.consumer packages respectively.
They are packaged in the kafka-clients-0.10.x.y.jar
Hope that helps,
Micha\u0142
On 01/05/17 14:04, Nixon Rodrigues wrote:
> Hi Kafka dev team,
>
>
> I am using kafka_2.11-0.10.0.0.jar api for kafka consumer,
>
> i am facing issue while consuming from topics over SASL_SSL,
> getting below exception
>
> Consumer.createJavaConsumerConnector(new
> kafka.consumer.ConsumerConfig(consumerProperties))
>
>
> As per documentation found on
> http://docs.confluent.io/2.0.0/kafka/sasl.html , it is said that
>
> "SASL authentication is only supported for the new Kafka producer
> and consumer, the older API is not supported."
>
> Can anybody confirm me which kafka version onward SASl_SSL is
> supported, any input on this is appreciated.
>
>
> 2017-04-26 16:06:36,655 WARN -
> [node11.openstacklocal-1493222790718-b862352a-leader-finder-thread:] ~
> Fetching topic metadata with correlation id 18 for topics [Set(TEST)]
> from broker [BrokerEndPoint(1001,node11.openstacklocal,6667)] failed
> (Logging$class:89)
> java.io.EOFException
> at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
> at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:140)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:131)
> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:84)
> at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:81)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
> at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
> 2017-04-26 16:06:36,655 WARN -
> [node11.openstacklocal-1493222790718-b862352a-leader-finder-thread:] ~
> [node11.openstacklocal-1493222790718-b862352a-leader-finder-thread],
> Failed to find leader for Set([TEST,0]) (Logging$class:89)
> kafka.common.KafkaException: fetching topic metadata for topics
> [Set(TEST)] from broker
> [ArrayBuffer(BrokerEndPoint(1001,node11.openstacklocal,6667))] failed
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73)
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
> at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
> Caused by: java.io.EOFException
> at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
> at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:140)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:131)
> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:84)
> at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:81)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
>
>
> Nixon
>
--
Signature
<http://www.openbet.com/> Michal Borowiecki
Senior Software Engineer L4
T: +44 208 742 1600
+44 203 249 8448
E: michal.borowiecki@openbet.com
W: www.openbet.com <http://www.openbet.com/>
OpenBet Ltd
Chiswick Park Building 9
566 Chiswick High Rd
London
W4 5XT
UK
<https://www.openbet.com/email_promo>
This message is confidential and intended only for the addressee. If you
have received this message in error, please immediately notify the
postmaster@openbet.com <ma...@openbet.com> and delete it
from your system as well as any copies. The content of e-mails as well
as traffic data may be monitored by OpenBet for employment and security
purposes. To protect the environment please do not print this e-mail
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company
registered in England and Wales. Registered no. 3134634. VAT no.
GB927523612