You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2019/02/07 00:43:31 UTC
[GitHub] nkurihar opened a new issue #3535: Broker stopped with a lot of
LedgerFencedException
nkurihar opened a new issue #3535: Broker stopped with a lot of LedgerFencedException
URL: https://github.com/apache/pulsar/issues/3535
**Describe the bug**
We have seen unexpected Broker shutdown.
1. There were LedgerFencedExceptions for a lot of ledgers:
```
01:47:09.430 [BookKeeperClientWorker-OrderedExecutor-43-0] WARN o.a.bookkeeper.client.PendingAddOp - Fencing exception on write: L9104171 E28233 on xxx.xx.xx.xx:3181
01:47:09.430 [BookKeeperClientWorker-OrderedExecutor-43-0] ERROR o.a.bookkeeper.client.LedgerHandle - Closing ledger 9104171 due to LedgerFencedException: Ledger has been fenced off. Some other client must have opened it to read
01:47:09.430 [BookKeeperClientWorker-OrderedExecutor-43-0] WARN o.a.bookkeeper.client.PendingAddOp - Fencing exception on write: L9104171 E28234 on xxx.xx.xx.xx:3181
01:47:09.430 [BookKeeperClientWorker-OrderedExecutor-43-0] ERROR o.a.bookkeeper.client.LedgerHandle - Closing ledger 9104171 due to LedgerFencedException: Ledger has been fenced off. Some other client must have opened it to read
01:47:09.430 [BookKeeperClientWorker-OrderedExecutor-43-0] WARN o.a.bookkeeper.client.PendingAddOp - Fencing exception on write: L9104171 E28235 on xxx.xx.xx.xx:3181
01:47:09.430 [BookKeeperClientWorker-OrderedExecutor-43-0] ERROR o.a.bookkeeper.client.LedgerHandle - Closing ledger 9104171 due to LedgerFencedException: Ledger has been fenced off. Some other client must have opened it to read
...
```
2. There were a lot of "Failed to create producer: Producer with name <geo prelicator>".
Moreover, the strange thing is that this broker is in cluster**1** and normally geo-replicates to cluster**2**, so the replicator name is expected as `pulsar.repl.<cluster2>`.
```
01:47:09.907 [pulsar-io-21-31] ERROR o.a.pulsar.client.impl.ProducerImpl - [persistent://<topicname>] [pulsar.repl.<cluster1>] Failed to create producer: Producer with name 'pulsar.repl.<cluster1>' is already connected to topic
```
3. Finally, Broker suddenly stopped with
```
01:47:09.963 [pulsar-ordered-OrderedExecutor-4-0-EventThread] ERROR o.a.p.z.ZooKeeperSessionWatcher - ZooKeeper session already expired, invoking shutdown
```
**Additional context**
Broker OS: CentOS Linux release 7.6.1810
Broker version: 2.1.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services