You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/09/16 22:25:07 UTC
[GitHub] [pulsar] dlg99 opened a new issue #12070: Debezium connector hung and failed to restart, "Failed to get batch size for entry .. No such ledger exists on Metadata Server" logged for the offset topic
dlg99 opened a new issue #12070:
URL: https://github.com/apache/pulsar/issues/12070
**Describe the bug**
Debezium connector hung (was not processing messages) and was not working after restarts until the offset storage topic was deleted and recreated.
**To Reproduce**
I do not have a repro, this happened in prod and so far I cannot repro it.
Interesting log excerpts:
```
[tenant/namespace/debezium-postgres-source-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - Source open produced uncaught exception:
java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription reader-c791f27169 of the topic persistent://tenant/namespace/debezium-postgres-source-debezium-offset-topic gets the last message id was failed
Failed to get batch size for entry org.apache.bookkeeper.mledger.ManagedLedgerException$NonRecoverableLedgerException: No such ledger exists on Metadata Server
```
```
[bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [replicated-tenant/gns/persistent/debezium-postgres-source-debezium-offset-topic] Removing ledger 375120 - size: 1049452
[bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [replicated-tenant/gns/persistent/debezium-postgres-source-debezium-offset-topic] End TrimConsumedLedgers. ledgers=1 totalSize=0
[bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [replicated-tenant/gns/persistent/debezium-postgres-source-debezium-offset-topic] Ledger 375120 contains the current last confirmed entry 375120:3160, and it is going to be deleted
[bookkeeper-ml-workers-OrderedExecutor-1-0] WARN org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [replicated-tenant/gns/persistent/debezium-postgres-source-debezium-offset-topic] Failed to mark delete while trimming data ledgers: Invalid mark deleted position
```
retention:
```
bin/pulsar-admin namespaces get-retention tenant/namespace
{
"retentionTimeInMinutes" : -1,
"retentionSizeInMB" : 1
}
```
Past issues that look similar: https://github.com/apache/pulsar/issues/8677 PR: https://github.com/apache/pulsar/pull/8725
**Expected behavior**
connector not hanging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #12070: Debezium connector hung and failed to restart, "Failed to get batch size for entry .. No such ledger exists on Metadata Server" logged for the offset topic
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #12070:
URL: https://github.com/apache/pulsar/issues/12070#issuecomment-929689346
@dlg99 I think it might be a bug fixed by @codelipenghui a couple of days ago. https://github.com/apache/pulsar/pull/12161
@codelipenghui Can you take a look at this to see if this is the same issue or not?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #12070: Debezium connector hung and failed to restart, "Failed to get batch size for entry .. No such ledger exists on Metadata Server" logged for the offset topic
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12070:
URL: https://github.com/apache/pulsar/issues/12070#issuecomment-1054903133
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #12070: Debezium connector hung and failed to restart, "Failed to get batch size for entry .. No such ledger exists on Metadata Server" logged for the offset topic
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #12070:
URL: https://github.com/apache/pulsar/issues/12070#issuecomment-929689346
@dlg99 I think it might be a bug fixed by @codelipenghui a couple of days ago. https://github.com/apache/pulsar/pull/12161
@codelipenghui Can you take a look at this to see if this is the same issue or not?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org