You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/25 08:28:57 UTC
[GitHub] [pulsar] Richard-Yi opened a new issue #11774: Fail to restart broker Due to org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException
Richard-Yi opened a new issue #11774:
URL: https://github.com/apache/pulsar/issues/11774
**Describe the bug**
When I restart broker after installing kafka-source plugin into the running broker, the broker restart failed.
the error log as belows:
```log
15:56:47.533 [pulsar-client-io-65-1] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xa042a253, L:/10.70.20.106:46632 - R:10.70.20.106/10.70.20.106:6650] Received error from server: Failed to get batch size for entry java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6
15:56:47.534 [pulsar-client-io-65-1] ERROR org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771] Failed getLastMessageId command
15:56:47.535 [pulsar-client-io-65-1] ERROR org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata][c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771] Failed getLastMessageId command
15:56:47.540 [pulsar-io-4-1] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.70.20.106:46632] Closing consumer: consumerId=1
15:56:47.540 [pulsar-io-4-1] INFO org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer - Removing consumer Consumer{subscription=PersistentSubscription{topic=persistent://public/functions/metadata, name=c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771}, consumerId=1, consumerName=c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer, address=/10.70.20.106:46632}
15:56:47.541 [pulsar-io-4-1] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/metadata][c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771] Successfully closed subscription [NonDurableCursorImpl{ledger=public/functions/persistent/metadata, ackPos=182:-1, readPos=182:1}]
15:56:47.542 [pulsar-io-4-1] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/functions/metadata][c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771] Successfully closed dispatcher for reader
15:56:47.543 [pulsar-io-4-1] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.70.20.106:46632] Closed consumer, consumerId=1
15:56:47.545 [pulsar-client-io-65-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://public/functions/metadata] [c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771] Closed consumer
15:56:47.545 [main] ERROR org.apache.pulsar.functions.worker.FunctionMetaDataManager - Failed to initialize meta data store
org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771 of the topic persistent://public/functions/metadata gets the last message id was failed
Failed to get batch size for entry java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:987) ~[org.apache.pulsar-pulsar-client-api-2.8.0.jar:2.8.0]
at org.apache.pulsar.client.impl.ConsumerImpl.hasMessageAvailable(ConsumerImpl.java:1876) ~[org.apache.pulsar-pulsar-client-original-2.8.0.jar:2.8.0]
at org.apache.pulsar.client.impl.ReaderImpl.hasMessageAvailable(ReaderImpl.java:168) ~[org.apache.pulsar-pulsar-client-original-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:110) [org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:496) [org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1490) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:793) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:259) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:331) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
15:56:47.545 [main] ERROR org.apache.pulsar.functions.worker.PulsarWorkerService - Error Starting up in worker
java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:116) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:496) [org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1490) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:793) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:259) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:331) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
Caused by: org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771 of the topic persistent://public/functions/metadata gets the last message id was failed
Failed to get batch size for entry java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:987) ~[org.apache.pulsar-pulsar-client-api-2.8.0.jar:2.8.0]
at org.apache.pulsar.client.impl.ConsumerImpl.hasMessageAvailable(ConsumerImpl.java:1876) ~[org.apache.pulsar-pulsar-client-original-2.8.0.jar:2.8.0]
at org.apache.pulsar.client.impl.ReaderImpl.hasMessageAvailable(ReaderImpl.java:168) ~[org.apache.pulsar-pulsar-client-original-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:110) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
... 5 more
15:56:47.545 [main] ERROR org.apache.pulsar.broker.PulsarService - Failed to start Pulsar service: java.lang.RuntimeException: Failed to initialize Metadata Manager
java.lang.RuntimeException: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:571) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1490) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:793) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:259) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:331) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
Caused by: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:116) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:496) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
... 4 more
Caused by: org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771 of the topic persistent://public/functions/metadata gets the last message id was failed
Failed to get batch size for entry java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:987) ~[org.apache.pulsar-pulsar-client-api-2.8.0.jar:2.8.0]
at org.apache.pulsar.client.impl.ConsumerImpl.hasMessageAvailable(ConsumerImpl.java:1876) ~[org.apache.pulsar-pulsar-client-original-2.8.0.jar:2.8.0]
at org.apache.pulsar.client.impl.ReaderImpl.hasMessageAvailable(ReaderImpl.java:168) ~[org.apache.pulsar-pulsar-client-original-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:110) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:496) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
... 4 more
15:56:47.545 [main] ERROR org.apache.pulsar.PulsarBrokerStarter - Failed to start pulsar service.
org.apache.pulsar.broker.PulsarServerException: java.lang.RuntimeException: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:821) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:259) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:331) [org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:571) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1490) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:793) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
... 2 more
Caused by: java.lang.RuntimeException: Failed to initialize Metadata Manager
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:116) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:496) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1490) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:793) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
... 2 more
Caused by: org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The subscription c-pulsar-cluster-zk-1-fw-10.70.20.106-8080-function-metadata-tailer-reader-1b67bf0771 of the topic persistent://public/functions/metadata gets the last message id was failed
Failed to get batch size for entry java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:987) ~[org.apache.pulsar-pulsar-client-api-2.8.0.jar:2.8.0]
at org.apache.pulsar.client.impl.ConsumerImpl.hasMessageAvailable(ConsumerImpl.java:1876) ~[org.apache.pulsar-pulsar-client-original-2.8.0.jar:2.8.0]
at org.apache.pulsar.client.impl.ReaderImpl.hasMessageAvailable(ReaderImpl.java:168) ~[org.apache.pulsar-pulsar-client-original-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.FunctionMetaDataManager.initialize(FunctionMetaDataManager.java:110) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:496) ~[org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1490) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:793) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
... 2 more
```
**Additional context**
- pulsar version: 2.8.0
- pulsar mode: cluster
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #11774: Fail to restart broker Due to org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #11774:
URL: https://github.com/apache/pulsar/issues/11774#issuecomment-905341313
This problem is most likely fixed by #10968 which will be included in Pulsar 2.8.1 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari closed issue #11774: Fail to restart broker Due to org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException
Posted by GitBox <gi...@apache.org>.
lhotari closed issue #11774:
URL: https://github.com/apache/pulsar/issues/11774
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on issue #11774: Fail to restart broker Due to org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on issue #11774:
URL: https://github.com/apache/pulsar/issues/11774#issuecomment-905341313
This problem is most likely fixed by #10968 which will be included in Pulsar 2.8.1 . Cherry-picked commit in branch-2.8 is 9c3e904a .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on issue #11774: Fail to restart broker Due to org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on issue #11774:
URL: https://github.com/apache/pulsar/issues/11774#issuecomment-905343447
this issue duplicates #10967 . Closing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #11774: Fail to restart broker Due to org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #11774:
URL: https://github.com/apache/pulsar/issues/11774#issuecomment-905343447
this issue duplicate #10967 . Closing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org