You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/07/07 09:46:25 UTC

[GitHub] [pulsar] yakir-Yang opened a new issue #11246: Pulsar Handshake was not completed within timeout, closing connection

yakir-Yang opened a new issue #11246:
URL: https://github.com/apache/pulsar/issues/11246


   我有三个 pulsar broker 节点，但是经常遇到部分 broker 节点 6650 端口服务不可用。节点故障情况下， 从 broker 的日志可以看出 healthcheck 一直处于失败的状态。healthcheck 失败的原因，我判断就是 handshake timeout，下面会贴具体日志。
   
   另外我发现即使 broker 节点不可用，但是其 8080 端口依旧能正常提供服务，可以执行 ./bin/pulsar-admin persistent get-partitioned-topic-metadata persistent://pulsar/pulsar-cluster/10.2.1.26:8080/healthcheck 命令，也能获取到正确的结果，所以只有 6650 端口服务异常。
   
   我通过在故障 broker 节点，通过 tcpdump 抓取 6650 端口报文，发现 pulsar client 和 pulsar broker 可以正常建立 tcp 连接，但是 pulsar client 发送了一笔 tcp data 后，pulsar broker 只会返回  tcp ack ，接下来 pulsar broker 没有任何 tcp 报文。我在 pulsar client 里面加日志，也如期的发现 pulsar client 连接一直处于 SentConnectFrame 状态。等上 30s 后，连接都超时异常。
   
   pulsar broker 日志：
   ![image](https://user-images.githubusercontent.com/9940360/124737222-51ed1d00-df4a-11eb-8347-272831ec1cc7.png)
   
   pulsar client 日志：
   ![image](https://user-images.githubusercontent.com/9940360/124738110-2fa7cf00-df4b-11eb-98b5-4bdf89c650b3.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] yakir-Yang commented on issue #11246: Pulsar Handshake was not completed within timeout, closing connection

Posted by GitBox <gi...@apache.org>.

yakir-Yang commented on issue #11246:
URL: https://github.com/apache/pulsar/issues/11246#issuecomment-875528709


   下面是 jstack & jmap 的信息
   
   =============== jstack ================
   [jstack.txt](https://github.com/apache/pulsar/files/6776768/jstack.txt)
   
   
   =============== jmap ===================
   [jmap.txt](https://github.com/apache/pulsar/files/6776767/jmap.txt)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codelipenghui commented on issue #11246: Pulsar Handshake was not completed within timeout, closing connection

Posted by GitBox <gi...@apache.org>.

codelipenghui commented on issue #11246:
URL: https://github.com/apache/pulsar/issues/11246#issuecomment-1058888550


   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org