You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "wallacepeng (via GitHub)" <gi...@apache.org> on 2024/04/22 20:24:26 UTC

[I] [Bug] broker healthcheck ran into loop after decommissioned a cluster of bookies [pulsar]

wallacepeng opened a new issue, #22559:
URL: https://github.com/apache/pulsar/issues/22559

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Read release policy
   
   - [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
   
   
   ### Version
   
   puslar 2.10.5
   
   ### Minimal reproduce step
   
   1. setup two bookkeeper clusters using helm charts 
      bookkeeper and bookkeeper1
   2. make bookkeeper as readonly 
   3. decommission bookkeeper till zero replica (as we are using kubernetes, scale down one node, autorecovery replicates the ledgers) 
   4. restart brokers. 
   5. broker ran into loop on health check 
   
   ### What did you expect to see?
   
   broker health check should continue to work
   
   ### What did you see instead?
   
   broker health check ran into loop 
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] broker healthcheck ran into loop after decommissioned a cluster of bookies [pulsar]

Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
wallacepeng commented on issue #22559:
URL: https://github.com/apache/pulsar/issues/22559#issuecomment-2070891379

   <img width="1782" alt="image" src="https://github.com/apache/pulsar/assets/894641/f8879642-e3e2-40b1-b063-308e957e693a">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] broker healthcheck ran into loop after decommissioned a cluster of bookies [pulsar]

Posted by "hpvd (via GitHub)" <gi...@apache.org>.
hpvd commented on issue #22559:
URL: https://github.com/apache/pulsar/issues/22559#issuecomment-2071657216

   since 2.10 is not supported anymore, cna you plz check if this also appears in newer versions?
   For details see
   - supported versions: https://pulsar.apache.org/contribute/release-policy/
   - latest pulsar helm chart: https://github.com/apache/pulsar-helm-chart/releases


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] broker healthcheck ran into loop after decommissioned a cluster of bookies [pulsar]

Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
wallacepeng closed issue #22559: [Bug] broker healthcheck ran into loop after decommissioned a cluster of bookies
URL: https://github.com/apache/pulsar/issues/22559


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] broker healthcheck ran into loop after decommissioned a cluster of bookies [pulsar]

Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
wallacepeng commented on issue #22559:
URL: https://github.com/apache/pulsar/issues/22559#issuecomment-2078544904

   I finally fixed it . I have to set up another broker cluster, then did some clean up for namespace and managed-ledgers , schemas , then restore the old broker cluster, it fixed the issue . 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] broker healthcheck ran into loop after decommissioned a cluster of bookies [pulsar]

Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
wallacepeng commented on issue #22559:
URL: https://github.com/apache/pulsar/issues/22559#issuecomment-2072404304

   @hpvd we are downgrading the storage so provisioned the bookkeeper cluster. we will upgrade the cluster a bit later. is there any way to clean the healthcheck topic it looks like the ledger cached old bookies ? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org