You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/03/07 07:51:16 UTC
[GitHub] [pulsar] JackrayWang opened a new issue #14575: broker restart failed
JackrayWang opened a new issue #14575:
URL: https://github.com/apache/pulsar/issues/14575
```
ERROR org.apache.pulsar.functions.worker.FunctionMetaDataManager - Failed to initialize meta data store
org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription c-pulsar-cluster-zk-fw-192.168.228.59-8080-function-metadata-tailer-reader-55d9f6694a of the topic persistent://public/functions/metadata gets the last message id was failed
{"errorMsg":"Failed to read last entry of the compacted Ledger Error while reading ledger","reqId":240264000667504132, "remote":"192.168.228.107/192.168.228.107:6650", "local":"/192.168.228.59:46798"}
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1034) ~[org.apache.pulsar-pulsar-client-api-2.9.1.jar:2.9.1]
at org.apache.pulsar.client.impl.ConsumerImpl.hasMessageAvailable(ConsumerImpl.java:1992) ~[org.apache.pulsar-pulsar-client-original-2.9.1.jar:2.9.1]
at org.apache.pulsar.client.impl.ReaderImpl.hasMessageAvailable(ReaderImpl.java:181) ~[org.apache.pulsar-pulsar-client-original-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:109) [org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:504) [org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1538) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:750) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:273) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:350) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
2022-03-07T15:47:11,871+0800 [main] ERROR org.apache.pulsar.functions.worker.PulsarWorkerService - Error Starting up in worker
java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:115) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:504) [org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1538) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:750) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:273) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:350) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
Caused by: org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription c-pulsar-cluster-zk-fw-192.168.228.59-8080-function-metadata-tailer-reader-55d9f6694a of the topic persistent://public/functions/metadata gets the last message id was failed
{"errorMsg":"Failed to read last entry of the compacted Ledger Error while reading ledger","reqId":240264000667504132, "remote":"192.168.228.107/192.168.228.107:6650", "local":"/192.168.228.59:46798"}
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1034) ~[org.apache.pulsar-pulsar-client-api-2.9.1.jar:2.9.1]
at org.apache.pulsar.client.impl.ConsumerImpl.hasMessageAvailable(ConsumerImpl.java:1992) ~[org.apache.pulsar-pulsar-client-original-2.9.1.jar:2.9.1]
at org.apache.pulsar.client.impl.ReaderImpl.hasMessageAvailable(ReaderImpl.java:181) ~[org.apache.pulsar-pulsar-client-original-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:109) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
... 5 more
2022-03-07T15:47:11,872+0800 [main] ERROR org.apache.pulsar.broker.PulsarService - Failed to start Pulsar service: java.lang.RuntimeException: Failed to initialize Metadata Manager
java.lang.RuntimeException: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:587) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1538) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:750) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:273) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:350) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
Caused by: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:115) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:504) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
... 4 more
Caused by: org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription c-pulsar-cluster-zk-fw-192.168.228.59-8080-function-metadata-tailer-reader-55d9f6694a of the topic persistent://public/functions/metadata gets the last message id was failed
{"errorMsg":"Failed to read last entry of the compacted Ledger Error while reading ledger","reqId":240264000667504132, "remote":"192.168.228.107/192.168.228.107:6650", "local":"/192.168.228.59:46798"}
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1034) ~[org.apache.pulsar-pulsar-client-api-2.9.1.jar:2.9.1]
at org.apache.pulsar.client.impl.ConsumerImpl.hasMessageAvailable(ConsumerImpl.java:1992) ~[org.apache.pulsar-pulsar-client-original-2.9.1.jar:2.9.1]
at org.apache.pulsar.client.impl.ReaderImpl.hasMessageAvailable(ReaderImpl.java:181) ~[org.apache.pulsar-pulsar-client-original-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:109) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:504) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
... 4 more
2022-03-07T15:47:11,873+0800 [main] ERROR org.apache.pulsar.PulsarBrokerStarter - Failed to start pulsar service.
org.apache.pulsar.broker.PulsarServerException: java.lang.RuntimeException: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:782) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:273) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:350) [org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:587) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1538) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:750) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
... 2 more
Caused by: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:115) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:504) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1538) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:750) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
... 2 more
Caused by: org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription c-pulsar-cluster-zk-fw-192.168.228.59-8080-function-metadata-tailer-reader-55d9f6694a of the topic persistent://public/functions/metadata gets the last message id was failed
{"errorMsg":"Failed to read last entry of the compacted Ledger Error while reading ledger","reqId":240264000667504132, "remote":"192.168.228.107/192.168.228.107:6650", "local":"/192.168.228.59:46798"}
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1034) ~[org.apache.pulsar-pulsar-client-api-2.9.1.jar:2.9.1]
at org.apache.pulsar.client.impl.ConsumerImpl.hasMessageAvailable(ConsumerImpl.java:1992) ~[org.apache.pulsar-pulsar-client-original-2.9.1.jar:2.9.1]
at org.apache.pulsar.client.impl.ReaderImpl.hasMessageAvailable(ReaderImpl.java:181) ~[org.apache.pulsar-pulsar-client-original-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:109) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:504) ~[org.apache.pulsar-pulsar-functions-worker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1538) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:750) ~[org.apache.pulsar-pulsar-broker-2.9.1.jar:2.9.1]
... 2 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #14575: broker restart failed
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #14575:
URL: https://github.com/apache/pulsar/issues/14575#issuecomment-1060774890
@JackrayWang It's a standalone or a cluster? could you please share the complete logs which will help us to see what happens during the broker restart.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] JackrayWang commented on issue #14575: broker restart failed
Posted by GitBox <gi...@apache.org>.
JackrayWang commented on issue #14575:
URL: https://github.com/apache/pulsar/issues/14575#issuecomment-1061333950
cluster @codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #14575: broker restart failed
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #14575:
URL: https://github.com/apache/pulsar/issues/14575#issuecomment-1067573483
Ok, is it able to reproduce? or could you please help confirm if the latest branch-2.9 also has the problem? there are some topic compaction fixes in branch-2.9
https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Acomponent%2Fcompaction+is%3Aclosed+label%3Arelease%2F2.9.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org