You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "yapxue (via GitHub)" <gi...@apache.org> on 2023/04/26 11:46:56 UTC
[GitHub] [pulsar] yapxue opened a new issue, #20186: [Bug] broker cannot connect to bookie suddenly and after restarting this issue persists
yapxue opened a new issue, #20186:
URL: https://github.com/apache/pulsar/issues/20186
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
pulsar 2.10.X
### Minimal reproduce step
checkout out to pulsar branch-2.10.X, make a docker image and deploy a pulsar cluster by pulsar helm chart.
Then broker complain cannot connect to bookie. Restart cannot sovle it.
### What did you expect to see?
broker can connect to bookie.
### What did you see instead?
```
2023-04-26T11:22:59,494+0000 [pulsar-registration-client-33-1] WARN org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy - Failed to resolve network location for pulsar-yapxue-bookie-7.pulsar-yapxue-bookie.rheos-streaming-pp.svc.137.tess.io, using default rack for it : /default-rack.
2023-04-26T11:22:59,494+0000 [pulsar-registration-client-33-1] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/pulsar-yapxue-bookie-7.pulsar-yapxue-bookie.rheos-streaming-pp.svc.137.tess.io:3181
2023-04-26T11:22:59,494+0000 [pulsar-registration-client-33-1] WARN org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy - Failed to resolve network location for pulsar-yapxue-bookie-3.pulsar-yapxue-bookie.rheos-streaming-pp.svc.137.tess.io, using default rack for it : /default-rack.
2023-04-26T11:22:59,494+0000 [pulsar-registration-client-33-1] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/pulsar-yapxue-bookie-3.pulsar-yapxue-bookie.rheos-streaming-pp.svc.137.tess.io:3181
2023-04-26T11:22:59,494+0000 [pulsar-registration-client-33-1] WARN org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy - Failed to resolve network location for pulsar-yapxue-bookie-8.pulsar-yapxue-bookie.rheos-streaming-pp.svc.137.tess.io, using default rack for it : /default-rack.
2023-04-26T11:22:59,495+0000 [pulsar-registration-client-33-1] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/pulsar-yapxue-bookie-8.pulsar-yapxue-bookie.rheos-streaming-pp.svc.137.tess.io:3181
2023-04-26T11:22:59,495+0000 [pulsar-registration-client-33-1] WARN org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy - Failed to resolve network location for pulsar-yapxue-bookie-9.pulsar-yapxue-bookie.rheos-streaming-pp.svc.137.tess.io, using default rack for it : /default-rack.
2023-04-26T11:22:59,495+0000 [pulsar-registration-client-33-1] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/pulsar-yapxue-bookie-9.pulsar-yapxue-bookie.rheos-streaming-pp.svc.137.tess.io:3181
```
### Anything else?
[broker log](https://gist.github.com/yapxue/9b225b78ddcd4bee3426718ac91131f4)
[bookie log](https://gist.github.com/yapxue/5a2b91856e0e5f530dd1f335bf6575c5)
I login a broker node and try to connect one bookie, it succeeded.
<img width="1708" alt="Screenshot 2023-04-26 at 19 44 50" src="https://user-images.githubusercontent.com/20301740/234565267-73d9af0a-920c-4e4e-a5e0-77137f2d714f.png">
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yapxue commented on issue #20186: [Bug] broker cannot connect to bookie suddenly and after restarting this issue persists
Posted by "yapxue (via GitHub)" <gi...@apache.org>.
yapxue commented on issue #20186:
URL: https://github.com/apache/pulsar/issues/20186#issuecomment-1524646912
> I can see several bookie id patterns, Bookie L2032 disappeared, Bookie ID-0000003234 disappeared, Bookie 0x00000000_0xffffffff disappeared, log is https://gist.github.com/yapxue/febcf0396038bc6ce9638e4ac535b4dc#file-broker_log_2023-04-26-2
v2.10.3 has this problem brought by [https://github.com/apache/pulsar/pull/17922](https://github.com/apache/pulsar/pull/17922)
v2.10.4 fixed this https://github.com/apache/pulsar/pull/18133/files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yapxue closed issue #20186: [Bug] broker cannot connect to bookie suddenly and after restarting this issue persists
Posted by "yapxue (via GitHub)" <gi...@apache.org>.
yapxue closed issue #20186: [Bug] broker cannot connect to bookie suddenly and after restarting this issue persists
URL: https://github.com/apache/pulsar/issues/20186
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yapxue closed issue #20186: [Bug] broker cannot connect to bookie suddenly and after restarting this issue persists
Posted by "yapxue (via GitHub)" <gi...@apache.org>.
yapxue closed issue #20186: [Bug] broker cannot connect to bookie suddenly and after restarting this issue persists
URL: https://github.com/apache/pulsar/issues/20186
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yapxue commented on issue #20186: [Bug] broker cannot connect to bookie suddenly and after restarting this issue persists
Posted by "yapxue (via GitHub)" <gi...@apache.org>.
yapxue commented on issue #20186:
URL: https://github.com/apache/pulsar/issues/20186#issuecomment-1523307887
I can see the log "Bookie urL0000002422 disappeared"
In what case can bookie id be deserialized as "urL0000002422"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yapxue commented on issue #20186: [Bug] broker cannot connect to bookie suddenly and after restarting this issue persists
Posted by "yapxue (via GitHub)" <gi...@apache.org>.
yapxue commented on issue #20186:
URL: https://github.com/apache/pulsar/issues/20186#issuecomment-1523321845
I can see several bookie id patterns, Bookie L2032 disappeared, Bookie ID-0000003234 disappeared, Bookie 0x00000000_0xffffffff disappeared
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org