You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "wallacepeng (via GitHub)" <gi...@apache.org> on 2024/04/21 03:19:36 UTC
[D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless [pulsar]
GitHub user wallacepeng created a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless
```
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
````
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng edited a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
this is caused by that internal healthcheck cannot create producer for topic (it seems have null producerName)
persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck .
it reports No such ledger exists on Metadata Server . All the others work except the healthcheck .
i have bookkeeper clusters, i decommission the default one then it got this error . does it still try to create ledger on decommisisoned bookie ?
```
bin/pulsar-admin topics stats-internal persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck
No such ledger exists on Metadata Server
Reason: No such ledger exists on Metadata Server
````
<img width="1754" alt="image" src="https://github.com/apache/pulsar/assets/894641/9eb33a0f-48f4-4ff4-8f80-88ea5c7cc1d7">
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng edited a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
this is caused by that internal healthcheck cannot create producer for topic
persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck .
it reports No such ledger exists on Metadata Server . All the others work except the healthcheck .
i have bookkeeper clusters, i decommission the default one then it got this error . does it still try to create ledger on decommisisoned bookie ?
```
bin/pulsar-admin topics stats-internal persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck
No such ledger exists on Metadata Server
Reason: No such ledger exists on Metadata Server
````
<img width="1754" alt="image" src="https://github.com/apache/pulsar/assets/894641/9eb33a0f-48f4-4ff4-8f80-88ea5c7cc1d7">
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng closed a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
this is caused by that internal healthcheck cannot create producer for topic (it seems have null producerName)
persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck .
it reports No such ledger exists on Metadata Server . All the others work except the healthcheck .
i have bookkeeper clusters, i decommission the default one then it got this error . does it still try to create ledger on decommisisoned bookie ?
```
bin/pulsar-admin topics stats-internal persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck
No such ledger exists on Metadata Server
Reason: No such ledger exists on Metadata Server
````
<img width="1754" alt="image" src="https://github.com/apache/pulsar/assets/894641/9eb33a0f-48f4-4ff4-8f80-88ea5c7cc1d7">
correct log from other pulsar cluster looks like
<img width="1908" alt="image" src="https://github.com/apache/pulsar/assets/894641/90c104d9-b351-4212-b26b-41dc170a18e4">
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng closed the discussion with a comment: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly
I ended up with setting up another broker cluster, then i shutdown old broker cluster, clean up the namespaces , managed-ledgers then store old cluster , decommission new broker cluster. it magically fixed .
GitHub link: https://github.com/apache/pulsar/discussions/22545#discussioncomment-9231670
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng added a comment to the discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless
Reason: No such ledger exists on Metadata Server
GitHub link: https://github.com/apache/pulsar/discussions/22545#discussioncomment-9177650
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng edited a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
this is caused by that internal healthcheck cannot create producer for topic
persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck .
it reports No such ledger exists on Metadata Server .
i have bookkeeper clusters, i decommission the default one then it got this error . does it still try to create ledger on decommisisoned bookie ?
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng edited a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
this is caused by that internal healthcheck cannot create producer for topic (it seems have null producerName)
persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck .
it reports No such ledger exists on Metadata Server . All the others work except the healthcheck .
i have bookkeeper clusters, i decommission the default one then it got this error . does it still try to create ledger on decommisisoned bookie ?
```
bin/pulsar-admin topics stats-internal persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck
No such ledger exists on Metadata Server
Reason: No such ledger exists on Metadata Server
````
<img width="1754" alt="image" src="https://github.com/apache/pulsar/assets/894641/9eb33a0f-48f4-4ff4-8f80-88ea5c7cc1d7">
correct log from other pulsar cluster looks like
<img width="1908" alt="image" src="https://github.com/apache/pulsar/assets/894641/90c104d9-b351-4212-b26b-41dc170a18e4">
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng edited a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
this is caused by that internal healthcheck cannot create producer for topic
persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck .
it reports No such ledger exists on Metadata Server .
i have bookkeeper clusters, i decommission the default one then it got this error . does it still try to create ledger on decommisisoned bookie ?
```
bin/pulsar-admin topics stats-internal persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck
No such ledger exists on Metadata Server
Reason: No such ledger exists on Metadata Server
````
<img width="1754" alt="image" src="https://github.com/apache/pulsar/assets/894641/9eb33a0f-48f4-4ff4-8f80-88ea5c7cc1d7">
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng edited a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endlessly
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
this is caused by that internal healthcheck cannot create producer for topic
persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck .
it reports No such ledger exists on Metadata Server .
i have bookkeeper clusters, i decommission the default one then it got this error . does it still try to create ledger on decommisisoned bookie ?
```
bin/pulsar-admin topics stats-internal persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck
No such ledger exists on Metadata Server
Reason: No such ledger exists on Metadata Server
````
<img width="1754" alt="image" src="https://github.com/apache/pulsar/assets/894641/9eb33a0f-48f4-4ff4-8f80-88ea5c7cc1d7">
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng deleted a comment on the discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless
Reason: No such ledger exists on Metadata Server
GitHub link: https://github.com/apache/pulsar/discussions/22545#discussioncomment-9177650
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless [pulsar]
Posted by "wallacepeng (via GitHub)" <gi...@apache.org>.
GitHub user wallacepeng edited a discussion: broker healthcheck endpoint reports No such ledger exists on Metadata Server - ledger endless
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck] [null] Failed to create producer: {"errorMsg":"org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger","reqId":3493733038286036253, "remote":"pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650", "local":"/172.20.203.179:58124"}
2024-04-21 03:14:20.474
2024-04-21T03:14:20,474+0000 [pulsar-io-4-7] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xdce5010a, L:/172.20.203.179:58124 - R:pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local/172.20.203.179:6650] Received error from server: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger caused by org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server - ledger=3596046 - operation=Failed to open ledger
this is caused by that internal healthcheck cannot create producer for topic
persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck .
it reports No such ledger exists on Metadata Server .
i have bookkeeper clusters, i decommission the default one then it got this error . does it still try to create ledger on decommisisoned bookie ?
```
bin/pulsar-admin topics stats-internal persistent://pulsar/pulsar/pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:8080/healthcheck
No such ledger exists on Metadata Server
Reason: No such ledger exists on Metadata Server
````
GitHub link: https://github.com/apache/pulsar/discussions/22545
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org