You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by 深大李展发 <xl...@foxmail.com> on 2020/05/19 11:56:17 UTC
回复： Cannot access to kafka by server domain and port.
Hi, Moffatt, I had read this article many times, I am horrible and know few in Computer Network, it indeed explains how to configure Kafka to work in AWS and Docker, I will read again. Thank you.
------------------ 原始邮件 ------------------
发件人: "Robin Moffatt"<robin@confluent.io>;
发送时间: 2020年5月19日(星期二) 晚上7:31
收件人: "users"<users@kafka.apache.org>;
主题: Re: Cannot access to kafka by server domain and port.
This should help your understanding:
https://rmoff.net/2018/08/02/kafka-listeners-explained/
--
Robin Moffatt | Senior Developer Advocate | robin@confluent.io | @rmoff
On Tue, 19 May 2020 at 11:48, 深大李展发 <xlcyun@foxmail.com> wrote:
> Thank you, Ixy, thanks for your reply, it's working now!
> I will only need to try to set it up in Docker. May I ask why this is
> different? I had also tried "OUTSIDE://:9092" before, is there any
> difference between these config?&nbsp;
> I am always confused by this config, I thought it only means Kafka will
> bind to 9092 port, and accept all connections.&nbsp;
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:&nbsp;"lxy"<lxyscls@163.com&gt;;
> 发送时间:&nbsp;2020年5月19日(星期二) 晚上6:15
> 收件人:&nbsp;"users"<users@kafka.apache.org&gt;;
>
> 主题:&nbsp;Re:Cannot access to kafka by server domain and port.
>
>
>
>
>
>
> It seems that your OUTSIDE listener is wrong. "OUTSIDE://localhost:9092"
> means 127.0.0.1:9092. Try "OUTSIDE://0.0.0.0:9092"
>
>
>
>
>
>
>
>
>
>
> At 2020-05-19 17:57:41, "深大李展发" <xlcyun@foxmail.com&gt; wrote:
>
> Hi, I have been struggling for this connection problem for a whole week.
> I run Kafka&nbsp; on my server machine which is on Cloud. And I cannot
> manage to connect to Kafka in anyway.
> It always print out `Connection to node -1
> (myServerDomain.ltd/myServerIp:9092) could not be established. Broker may
> not be available.`
>
>
>
>
> Here is what I had done to keep it simple so I can find out why:
> - I stopped using Docker.
> - I configure the Cloud firewall, so it will not block port 9092.
> - I start Kafka standalone(1 broker).
> - I start zookeeper standalone(1 node).
> - Zookeeper and Kafka use JAAS to connect.
> - I configure Kafka to log in TRACE level.
> - SASL and SSL is all turn off.
>
>
> I know advertised.listeners is usually the key of these situation, so here
> is what I had tried:
> SITUATION 1:
> - set advertised.listener to `localhost` on propose
> - use `kafka-topic --list --bootstrap-server localhost:9092`:
> &nbsp;&nbsp;&nbsp; 1. Kafka print out the metadata request
> &nbsp;&nbsp;&nbsp; 2. Kafka print out the returned metadata
> &nbsp;&nbsp;&nbsp; 3. Client connect to the advertised listener(which is
> localhost) successfully
> &nbsp;&nbsp;&nbsp; 4. Topics list is returned, client print out topic
> list. All well.
> - use `kafka-topic --list --bootstrap-server xxx.ltd:9092`
> &nbsp;&nbsp;&nbsp; 1. Kafka even not print out the first metadata request.
> &nbsp;&nbsp;&nbsp; 2. In client, it print out(Notice, it is node -1, not
> node 1): `Connection to node -1 (myServerDomain.ltd/myServerIp:9092) could
> not be established. Broker may not be available.`
> &nbsp;&nbsp;&nbsp; 3. Stop Kafka, start a WWW service on port 9092, can
> access the WWW service by port 9092.
>
>
> SITUATION 2:
> - set advertised.listener to `xxx.ltd`
> - use `kafka-topic --list --bootstrap-server localhost:9092`:
> &nbsp;&nbsp;&nbsp; 1. Kafka print out the metadata request
> &nbsp;&nbsp;&nbsp; 2. Kafka print out the returned metadata
> &nbsp;&nbsp;&nbsp; 3. Client try to connect to the advertised
> listener(which is xxx.ltd)
> &nbsp;&nbsp;&nbsp; 4. Failed, it print out(Notice,it is node 1, not node
> -1 like above, that means, client is try to connect by the metadata
> returned by kafka): `Connection to node 1
> (myServerDomain.ltd/myServerIp:9092) could not be established. Broker may
> not be available.`
> - use `kafka-topic --list --bootstrap-server xxx.ltd:9092`:
> &nbsp;&nbsp;&nbsp; 1. Kafka even not print out the first metadat request.
> &nbsp;&nbsp;&nbsp; 2. In client, it print out(Notice is node -1, not node
> 1): `Connection to node -1 (myServerDomain.ltd/myServerIp:9092) could not
> be established. Broker may not be available.`
>
>
> So, maybe, I think, maybe there is not any TCP request, all request to
> myServerDomain.ltd/myServerIp:9092 is blocked somehow. So I use `tcpdump -p
> 9092` to capture packets, here is what I get:
> ---------------------------------------------
> ...
> 2020-05-19 17:34:41.428139 IP 172.18.118.28.9092 &gt; 61.140.182.143.5826:
> Flags [R.], seq 0, ack 4281665850, win 0, length 0
> 2020-05-19 17:34:41.842286 IP 61.140.182.143.5828 &gt; 172.18.118.28.9092:
> Flags [S], seq 3141006320, win 64240, options [mss 1400,sackOK,TS val
> 1788286298 ecr 0,nop,wscale 1], length 0
> 2020-05-19 17:34:41.842360 IP 172.18.118.28.9092 &gt; 61.140.182.143.5828:
> Flags [R.], seq 0, ack 3141006321, win 0, length 0
> 2020-05-19 17:34:42.657551 IP 61.140.182.143.5833 &gt; 172.18.118.28.9092:
> Flags [S], seq 44626980, win 64240, options [mss 1400,sackOK,TS val
> 1788287114 ecr 0,nop,wscale 1], length 0
> 2020-05-19 17:34:42.657604 IP 172.18.118.28.9092 &gt; 61.140.182.143.5833:
> Flags [R.], seq 0, ack 44626981, win 0, length
> ...
> 61.140.182.143 is my local laptop. It seems they were communicating.
>
>
> This is my `server.properties` config:
> --------------------------------------------------------
> broker.id=1
> num.network.threads=3
> num.io.threads=8
> socket.send.buffer.bytes=102400
> socket.receive.buffer.bytes=102400
> socket.request.max.bytes=104857600
> log.dirs=/var/log/kafka-1
> num.partitions=3
> num.recovery.threads.per.data.dir=3
> offset.topic.replication.factor=3
> transaction.state.log.replication.factor=3
> transaction.state.log.min.isr=3
>
>
> zookeeper.connect=zookeeper1.myServerDomain.ltd:2181(Original set for
> Docker network, now I use /etc/hosts map to localhost)
> zookeeper.connection.timeout.ms=18000
>
>
> listener.security.protocol.map=INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
> listeners=INSIDE://kafka1.myServerDomain.ltd:19092,OUTSIDE://localhost:9092
>
> advertised.listeners=INSIDE://kafka1.myServerDomain.ltd:19092,OUTSIDE://myServerDomain.ltd:9092
> inter.broker.listener.name=INSIDE
> --------------------------------------------------------
>
>
> I am almost desperate, ANY SOLUTION OR DIRECTION WOULD BE VERY
> APPRECIATED!!!