You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Vineet Kumar <vi...@gmail.com> on 2016/07/07 23:49:05 UTC
SSL: Broker works but consumer/producer fail
Hi
I followed Apache Kafka SSL instructions verbatim but my producer and
consumer both hang or error out as follows.
openssl s_client BTW does work fine with the server below yielding
certificates etc thereby confirming that the server can talk back SSL.
*Producer and Consumer*
=====================
Config changes (client-ssl.properties)
---------------
security.protocol=SSL
% bin/kafka-console-*consumer*.sh --bootstrap-server 192.168.1.XXX:9093
--topic test --new-consumer --consumer.config config/client-ssl.properties
*<Hang?.....No output.....>*
% bin/kafka-console-*producer*.sh --broker-list 192.168.1.XXX:9093 --topic
test --producer.config config/client-ssl.properties
aaaaaaaaaaaaaaaaaaaaaaaaaaaaa
*<After more than a minute of silence...>*
[2016-07-07 16:35:57,670] ERROR Error when sending message to topic test
with key: null, value: 29 bytes with error:
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata
after 60000 ms.
*Broker*
======
Config changes (server.properties)
---------------
listeners=SSL://192.168.1.XXX:9093
security.inter.broker.protocol=SSL
advertised.listeners=SSL://192.168.1.XXX:9093
ssl.keystore.location=/<......>/server.keystore.jks
ssl.keystore.password=<password>
ssl.key.password=<password>
% bin/kafka-*server*-start.sh config/server.properties
[2016-07-07 16:14:00,805] INFO Registered broker 0 at path /brokers/ids/0
with addresses: *SSL -> EndPoint(192.168.1.XXX,9093,SSL)*
(kafka.utils.ZkUtils)
[2016-07-07 16:14:00,820] INFO New leader is 0
(kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2016-07-07 16:14:00,825] INFO Kafka version : 0.10.0.0
(org.apache.kafka.common.utils.AppInfoParser)
[2016-07-07 16:14:00,825] INFO Kafka commitId : b8642491e78c5a13
(org.apache.kafka.common.utils.AppInfoParser)
[2016-07-07 16:14:00,827] INFO [Kafka Server 0], started
(kafka.server.KafkaServer)
*Zookeeper*
=========
Config changes
---------------
Nothing
% bin/zookeeper-server-start.sh config/zookeeper.properties
....
....
[2016-07-07 16:13:18,002] INFO binding to port 0.0.0.0/0.0.0.0:2181
(org.apache.zookeeper.server.NIOServerCnxnFactory)
....
....
[2016-07-07 16:14:00,131] INFO Accepted socket connection from /
127.0.0.1:41188 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2016-07-07 16:14:00,189] INFO Client attempting to establish new session
at /127.0.0.1:41188 (org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-07 16:14:00,199] INFO Established session 0x155c7a306dc0000 with
negotiated timeout 6000 for client /127.0.0.1:41188
(org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-07 16:14:00,652] INFO Got user-level KeeperException when
processing sessionid:0x155c7a306dc0000 type:delete cxid:0x22 zxid:0xd6
txntype:-1 reqpath:n/a Error Path:/admin/preferred_replica_election
Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election
(org.apache.zookeeper.server.PrepRequestProcessor)
[2016-07-07 16:14:00,778] INFO Got user-level KeeperException when
processing sessionid:0x155c7a306dc0000 type:create cxid:0x29 zxid:0xd7
txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode =
NodeExists for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
[2016-07-07 16:14:00,778] INFO Got user-level KeeperException when
processing sessionid:0x155c7a306dc0000 type:create cxid:0x2a zxid:0xd8
txntype:-1 reqpath:n/a Error Path:/brokers/ids Error:KeeperErrorCode =
NodeExists for /brokers/ids
(org.apache.zookeeper.server.PrepRequestProcessor)
Re: SSL: Broker works but consumer/producer fail
Posted by Harsha Chintalapani <ka...@harsha.io>.
Which version of kafka are you using.
-Harsha
On Fri, Jul 8, 2016 at 1:09 AM Narendra Bidari <na...@gmail.com>
wrote:
> Hi Vineet,
>
> The setup of ssl Kafka requires to make one too many steps precise
> correct. I have listed some below . Hope it helps
> https://github.com/Symantec/kafka-security-0.9
>
> Sent from my iPhone
>
> Regards
>
> > On Jul 7, 2016, at 4:49 PM, Vineet Kumar <vi...@gmail.com> wrote:
> >
> > Hi
> > I followed Apache Kafka SSL instructions verbatim but my producer and
> > consumer both hang or error out as follows.
> > openssl s_client BTW does work fine with the server below yielding
> > certificates etc thereby confirming that the server can talk back SSL.
> >
> >
> > *Producer and Consumer*
> > =====================
> >
> > Config changes (client-ssl.properties)
> > ---------------
> > security.protocol=SSL
> >
> > % bin/kafka-console-*consumer*.sh --bootstrap-server 192.168.1.XXX:9093
> > --topic test --new-consumer --consumer.config
> config/client-ssl.properties
> > *<Hang?.....No output.....>*
> >
> > % bin/kafka-console-*producer*.sh --broker-list 192.168.1.XXX:9093
> --topic
> > test --producer.config config/client-ssl.properties
> > aaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> >
> > *<After more than a minute of silence...>*
> >
> > [2016-07-07 16:35:57,670] ERROR Error when sending message to topic test
> > with key: null, value: 29 bytes with error:
> > (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
> > org.apache.kafka.common.errors.TimeoutException: Failed to update
> metadata
> > after 60000 ms.
> >
> > *Broker*
> > ======
> > Config changes (server.properties)
> > ---------------
> > listeners=SSL://192.168.1.XXX:9093
> > security.inter.broker.protocol=SSL
> > advertised.listeners=SSL://192.168.1.XXX:9093
> > ssl.keystore.location=/<......>/server.keystore.jks
> > ssl.keystore.password=<password>
> > ssl.key.password=<password>
> >
> > % bin/kafka-*server*-start.sh config/server.properties
> > [2016-07-07 16:14:00,805] INFO Registered broker 0 at path /brokers/ids/0
> > with addresses: *SSL -> EndPoint(192.168.1.XXX,9093,SSL)*
> > (kafka.utils.ZkUtils)
> > [2016-07-07 16:14:00,820] INFO New leader is 0
> > (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> > [2016-07-07 16:14:00,825] INFO Kafka version : 0.10.0.0
> > (org.apache.kafka.common.utils.AppInfoParser)
> > [2016-07-07 16:14:00,825] INFO Kafka commitId : b8642491e78c5a13
> > (org.apache.kafka.common.utils.AppInfoParser)
> > [2016-07-07 16:14:00,827] INFO [Kafka Server 0], started
> > (kafka.server.KafkaServer)
> >
> > *Zookeeper*
> > =========
> > Config changes
> > ---------------
> > Nothing
> >
> > % bin/zookeeper-server-start.sh config/zookeeper.properties
> > ....
> > ....
> > [2016-07-07 16:13:18,002] INFO binding to port 0.0.0.0/0.0.0.0:2181
> > (org.apache.zookeeper.server.NIOServerCnxnFactory)
> > ....
> > ....
> > [2016-07-07 16:14:00,131] INFO Accepted socket connection from /
> > 127.0.0.1:41188 (org.apache.zookeeper.server.NIOServerCnxnFactory)
> > [2016-07-07 16:14:00,189] INFO Client attempting to establish new session
> > at /127.0.0.1:41188 (org.apache.zookeeper.server.ZooKeeperServer)
> > [2016-07-07 16:14:00,199] INFO Established session 0x155c7a306dc0000 with
> > negotiated timeout 6000 for client /127.0.0.1:41188
> > (org.apache.zookeeper.server.ZooKeeperServer)
> > [2016-07-07 16:14:00,652] INFO Got user-level KeeperException when
> > processing sessionid:0x155c7a306dc0000 type:delete cxid:0x22 zxid:0xd6
> > txntype:-1 reqpath:n/a Error Path:/admin/preferred_replica_election
> > Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election
> > (org.apache.zookeeper.server.PrepRequestProcessor)
> > [2016-07-07 16:14:00,778] INFO Got user-level KeeperException when
> > processing sessionid:0x155c7a306dc0000 type:create cxid:0x29 zxid:0xd7
> > txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode =
> > NodeExists for /brokers
> (org.apache.zookeeper.server.PrepRequestProcessor)
> > [2016-07-07 16:14:00,778] INFO Got user-level KeeperException when
> > processing sessionid:0x155c7a306dc0000 type:create cxid:0x2a zxid:0xd8
> > txntype:-1 reqpath:n/a Error Path:/brokers/ids Error:KeeperErrorCode =
> > NodeExists for /brokers/ids
> > (org.apache.zookeeper.server.PrepRequestProcessor)
>
Re: SSL: Broker works but consumer/producer fail
Posted by Narendra Bidari <na...@gmail.com>.
Hi Vineet,
The setup of ssl Kafka requires to make one too many steps precise correct. I have listed some below . Hope it helps
https://github.com/Symantec/kafka-security-0.9
Sent from my iPhone
Regards
> On Jul 7, 2016, at 4:49 PM, Vineet Kumar <vi...@gmail.com> wrote:
>
> Hi
> I followed Apache Kafka SSL instructions verbatim but my producer and
> consumer both hang or error out as follows.
> openssl s_client BTW does work fine with the server below yielding
> certificates etc thereby confirming that the server can talk back SSL.
>
>
> *Producer and Consumer*
> =====================
>
> Config changes (client-ssl.properties)
> ---------------
> security.protocol=SSL
>
> % bin/kafka-console-*consumer*.sh --bootstrap-server 192.168.1.XXX:9093
> --topic test --new-consumer --consumer.config config/client-ssl.properties
> *<Hang?.....No output.....>*
>
> % bin/kafka-console-*producer*.sh --broker-list 192.168.1.XXX:9093 --topic
> test --producer.config config/client-ssl.properties
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>
> *<After more than a minute of silence...>*
>
> [2016-07-07 16:35:57,670] ERROR Error when sending message to topic test
> with key: null, value: 29 bytes with error:
> (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
> org.apache.kafka.common.errors.TimeoutException: Failed to update metadata
> after 60000 ms.
>
> *Broker*
> ======
> Config changes (server.properties)
> ---------------
> listeners=SSL://192.168.1.XXX:9093
> security.inter.broker.protocol=SSL
> advertised.listeners=SSL://192.168.1.XXX:9093
> ssl.keystore.location=/<......>/server.keystore.jks
> ssl.keystore.password=<password>
> ssl.key.password=<password>
>
> % bin/kafka-*server*-start.sh config/server.properties
> [2016-07-07 16:14:00,805] INFO Registered broker 0 at path /brokers/ids/0
> with addresses: *SSL -> EndPoint(192.168.1.XXX,9093,SSL)*
> (kafka.utils.ZkUtils)
> [2016-07-07 16:14:00,820] INFO New leader is 0
> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> [2016-07-07 16:14:00,825] INFO Kafka version : 0.10.0.0
> (org.apache.kafka.common.utils.AppInfoParser)
> [2016-07-07 16:14:00,825] INFO Kafka commitId : b8642491e78c5a13
> (org.apache.kafka.common.utils.AppInfoParser)
> [2016-07-07 16:14:00,827] INFO [Kafka Server 0], started
> (kafka.server.KafkaServer)
>
> *Zookeeper*
> =========
> Config changes
> ---------------
> Nothing
>
> % bin/zookeeper-server-start.sh config/zookeeper.properties
> ....
> ....
> [2016-07-07 16:13:18,002] INFO binding to port 0.0.0.0/0.0.0.0:2181
> (org.apache.zookeeper.server.NIOServerCnxnFactory)
> ....
> ....
> [2016-07-07 16:14:00,131] INFO Accepted socket connection from /
> 127.0.0.1:41188 (org.apache.zookeeper.server.NIOServerCnxnFactory)
> [2016-07-07 16:14:00,189] INFO Client attempting to establish new session
> at /127.0.0.1:41188 (org.apache.zookeeper.server.ZooKeeperServer)
> [2016-07-07 16:14:00,199] INFO Established session 0x155c7a306dc0000 with
> negotiated timeout 6000 for client /127.0.0.1:41188
> (org.apache.zookeeper.server.ZooKeeperServer)
> [2016-07-07 16:14:00,652] INFO Got user-level KeeperException when
> processing sessionid:0x155c7a306dc0000 type:delete cxid:0x22 zxid:0xd6
> txntype:-1 reqpath:n/a Error Path:/admin/preferred_replica_election
> Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election
> (org.apache.zookeeper.server.PrepRequestProcessor)
> [2016-07-07 16:14:00,778] INFO Got user-level KeeperException when
> processing sessionid:0x155c7a306dc0000 type:create cxid:0x29 zxid:0xd7
> txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode =
> NodeExists for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
> [2016-07-07 16:14:00,778] INFO Got user-level KeeperException when
> processing sessionid:0x155c7a306dc0000 type:create cxid:0x2a zxid:0xd8
> txntype:-1 reqpath:n/a Error Path:/brokers/ids Error:KeeperErrorCode =
> NodeExists for /brokers/ids
> (org.apache.zookeeper.server.PrepRequestProcessor)