You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Demogorgon314 (via GitHub)" <gi...@apache.org> on 2023/10/08 01:22:51 UTC
[PR] [branch-3.0][fix][broker] Fix lookup heartbeat and sla namespace bundle when using extensible load manager (#21213) [pulsar]
Demogorgon314 opened a new pull request, #21313:
URL: https://github.com/apache/pulsar/pull/21313
(cherry picked from commit f85e0dc085d47587037cbc3f7d5443745384d350)
### Motivation
Currently, if the cluster has multiple brokers, and the cluster is doing rolling restart, the heartbeat namespace topic's lookup result might be wrong, because the `ExtensibleLoadManagerImpl` does not check the heartbeat and SLA namespace bundle lookup candidate broker to let them own by the specified broker.
This ownership selection is wrong:
```
2023-09-20T08:27:32,188+0000 [ForkJoinPool.commonPool-worker-1] INFO org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl - Selected new owner broker: broker-2:8080 for bundle: pulsar/broker-0:8080/0x00000000_0xffffffff.
```
After the ownership assignment, the broker-0 will fail to start.
```
pulsar-broker 2023-09-20T08:27:54,707+0000 [main] INFO org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl - Try acquiring ownership for bundle: pulsar/broker-0:8080/0x00000000_0xffffffff - broker-0:8080.
pulsar-broker 2023-09-20T08:27:54,707+0000 [main] ERROR org.apache.pulsar.broker.namespace.NamespaceService - namespace already owned by other broker : ns=pulsar/broker-0:8080 expected=pulsar://broker-0:6650 actual=pulsar://broker-2:6650
pulsar-broker java.lang.IllegalStateException: namespace already owned by other broker : ns=pulsar/broker-0:8080 expected=pulsar://broker-0:6650 actual=pulsar://broker-2:6650
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerNamespace(NamespaceService.java:400) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerBootstrapNamespaces(NamespaceService.java:343) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:863) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:276) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:356) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker 2023-09-20T08:27:54,710+0000 [main] ERROR org.apache.pulsar.broker.PulsarService - Failed to start Pulsar service: java.lang.IllegalStateException: namespace already owned by other broker : ns=pulsar/broker-0:8080 expected=pulsar://broker-0:6650 actual=pulsar://broker-2:6650
pulsar-broker org.apache.pulsar.broker.PulsarServerException: java.lang.IllegalStateException: namespace already owned by other broker : ns=pulsar/broker-0:8080 expected=pulsar://broker-0:6650 actual=pulsar://broker-2:6650
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerNamespace(NamespaceService.java:403) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerBootstrapNamespaces(NamespaceService.java:343) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:863) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:276) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:356) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker Caused by: java.lang.IllegalStateException: namespace already owned by other broker : ns=pulsar/broker-0:8080 expected=pulsar://broker-0:6650 actual=pulsar://broker-2:6650
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerNamespace(NamespaceService.java:400) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker ... 4 more
pulsar-broker 2023-09-20T08:27:54,710+0000 [main] ERROR org.apache.pulsar.PulsarBrokerStarter - Failed to start pulsar service.
pulsar-broker org.apache.pulsar.broker.PulsarServerException: org.apache.pulsar.broker.PulsarServerException: java.lang.IllegalStateException: namespace already owned by other broker : ns=pulsar/broker-0:8080 expected=pulsar://broker-0:6650 actual=pulsar://broker-2:6650
pulsar-broker at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:938) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:276) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:356) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker Caused by: org.apache.pulsar.broker.PulsarServerException: java.lang.IllegalStateException: namespace already owned by other broker : ns=pulsar/broker-0:8080 expected=pulsar://broker-0:6650 actual=pulsar://broker-2:6650
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerNamespace(NamespaceService.java:403) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerBootstrapNamespaces(NamespaceService.java:343) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:863) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker ... 2 more
pulsar-broker Caused by: java.lang.IllegalStateException: namespace already owned by other broker : ns=pulsar/broker-0:8080 expected=pulsar://broker-0:6650 actual=pulsar://broker-2:6650
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerNamespace(NamespaceService.java:400) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.broker.namespace.NamespaceService.registerBootstrapNamespaces(NamespaceService.java:343) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:863) ~[io.streamnative-pulsar-broker-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
pulsar-broker ... 2 more
pulsar-broker 2023-09-20T08:27:54,711+0000 [main] WARN org.apache.pulsar.common.util.ShutdownUtil - Triggering immediate shutdown of current process with status 1
```
### Modifications
* Fix lookup heartbeat and sla namespace bundle when using extensible load manager
* When updating the topK bundles, filter out the heartbeat and SLA namespace bundles.
* Skip override orphan heartbeat namespace bundle
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][broker][branch-3.0] Fix lookup heartbeat and sla namespace bundle when using extensible load manager (#21213) [pulsar]
Posted by "Technoboy- (via GitHub)" <gi...@apache.org>.
Technoboy- merged PR #21313:
URL: https://github.com/apache/pulsar/pull/21313
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][broker][branch-3.0] Fix lookup heartbeat and sla namespace bundle when using extensible load manager (#21213) [pulsar]
Posted by "Demogorgon314 (via GitHub)" <gi...@apache.org>.
Demogorgon314 closed pull request #21313: [fix][broker][branch-3.0] Fix lookup heartbeat and sla namespace bundle when using extensible load manager (#21213)
URL: https://github.com/apache/pulsar/pull/21313
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org