You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/09/07 10:49:58 UTC
[GitHub] [pulsar] gaozhangmin opened a new pull request #11947: fix-npe-ZkBookieRackAffinityMapping
gaozhangmin opened a new pull request #11947:
URL: https://github.com/apache/pulsar/pull/11947
### Motivation
NPE when `ZkBookieRackAffinityMapping`
```
17:24:03.724 [main-SendThread(bigdata-pulsar-bookie004.ys:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server bigdata-pulsar-bookie004.ys/10.89.146.53:2181, session id = 0x57ba9d6a8d800a8, negotiated timeout = 30000
17:24:03.725 [main-EventThread] INFO org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now.
17:24:03.792 [main] ERROR org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping - Error creating zookeeper client
java.lang.IllegalArgumentException: Path must not end with / character
at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:55) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.client.ConnectStringParser.<init>(ConnectStringParser.java:61) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.ZooKeeper.createDefaultHostProvider(ZooKeeper.java:1515) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:1108) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.bookkeeper.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.java:286) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.zookeeper.ZooKeeperClient$Builder.build(ZooKeeperClient.java:247) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.getAndSetZkCache(ZkBookieRackAffinityMapping.java:138) [org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0]
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.setConf(ZkBookieRackAffinityMapping.java:75) [org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0]
at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.initialize(RackawareEnsemblePlacementPolicyImpl.java:264) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.initialize(RackawareEnsemblePlacementPolicyImpl.java:80) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.client.BookKeeper.initializeEnsemblePlacementPolicy(BookKeeper.java:582) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:506) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.client.BookKeeper$Builder.build(BookKeeper.java:307) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.replication.Auditor.createBookKeeperClient(Auditor.java:270) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.replication.AutoRecoveryMain.<init>(AutoRecoveryMain.java:92) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.server.service.AutoRecoveryService.<init>(AutoRecoveryService.java:41) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:320) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.server.Main.doMain(Main.java:226) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
at org.apache.bookkeeper.server.Main.main(Main.java:208) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
```
*Explain here the context, and why you're making that change. What is the problem you're trying to solve.*
### Modifications
1、change some deprecated method
2、add a judgment.
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (yes / no) no
- The public API: (yes / no) no
- The schema: (yes / no / don't know) no
- The default values of configurations: (yes / no) no
- The wire protocol: (yes / no)no
- The rest endpoints: (yes / no)no
- The admin cli options: (yes / no)no
- Anything that affects deployment: (yes / no / don't know)no
### Documentation
no need doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower merged pull request #11947: fix-npe-ZkBookieRackAffinityMapping
Posted by GitBox <gi...@apache.org>.
BewareMyPower merged pull request #11947:
URL: https://github.com/apache/pulsar/pull/11947
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on a change in pull request #11947: fix-npe-ZkBookieRackAffinityMapping
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on a change in pull request #11947:
URL: https://github.com/apache/pulsar/pull/11947#discussion_r704402141
##########
File path: pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
##########
@@ -123,18 +126,20 @@ private void updateRacksWithHost(BookiesRackConfiguration racks) {
if (conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE) != null) {
zkCache = (ZooKeeperCache) conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE);
} else {
- int zkTimeout;
- String zkServers;
if (conf instanceof ClientConfiguration) {
- zkTimeout = ((ClientConfiguration) conf).getZkTimeout();
- zkServers = ((ClientConfiguration) conf).getZkServers();
- String zkLedgersRootPath = ((ClientConfiguration) conf).getZkLedgersRootPath();
+ int zkTimeout = ((ClientConfiguration) conf).getZkTimeout();
try {
- int zkLedgerRootIndex = zkLedgersRootPath.contains("/") ?
- zkLedgersRootPath.lastIndexOf("/") : 0;
- String zkChangeRoot = zkLedgersRootPath.substring(0, zkLedgerRootIndex);
- zkChangeRoot = zkChangeRoot.startsWith("/") ? zkChangeRoot : "/" + zkChangeRoot;
- ZooKeeper zkClient = ZooKeeperClient.newBuilder().connectString(zkServers + zkChangeRoot)
+ final String metadataServiceUriStr = ((ClientConfiguration) conf).getMetadataServiceUri();
+ URI metadataServiceUri = URI.create(metadataServiceUriStr);
+ String zkServers;
+ String ledgersRootPath = metadataServiceUri.getPath();
+ if (ZK_LEDGERS_DEFAULT_ROOT_PATH.equals(ledgersRootPath)) {
+ zkServers = getZKServersFromServiceUri(metadataServiceUri);
Review comment:
I see, TY
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaozhangmin commented on pull request #11947: fix-npe-ZkBookieRackAffinityMapping
Posted by GitBox <gi...@apache.org>.
gaozhangmin commented on pull request #11947:
URL: https://github.com/apache/pulsar/pull/11947#issuecomment-914202621
@hanbo1990 @BewareMyPower PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on a change in pull request #11947: fix-npe-ZkBookieRackAffinityMapping
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on a change in pull request #11947:
URL: https://github.com/apache/pulsar/pull/11947#discussion_r704105273
##########
File path: pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
##########
@@ -123,18 +126,20 @@ private void updateRacksWithHost(BookiesRackConfiguration racks) {
if (conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE) != null) {
zkCache = (ZooKeeperCache) conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE);
} else {
- int zkTimeout;
- String zkServers;
if (conf instanceof ClientConfiguration) {
- zkTimeout = ((ClientConfiguration) conf).getZkTimeout();
- zkServers = ((ClientConfiguration) conf).getZkServers();
- String zkLedgersRootPath = ((ClientConfiguration) conf).getZkLedgersRootPath();
+ int zkTimeout = ((ClientConfiguration) conf).getZkTimeout();
try {
- int zkLedgerRootIndex = zkLedgersRootPath.contains("/") ?
- zkLedgersRootPath.lastIndexOf("/") : 0;
- String zkChangeRoot = zkLedgersRootPath.substring(0, zkLedgerRootIndex);
- zkChangeRoot = zkChangeRoot.startsWith("/") ? zkChangeRoot : "/" + zkChangeRoot;
- ZooKeeper zkClient = ZooKeeperClient.newBuilder().connectString(zkServers + zkChangeRoot)
+ final String metadataServiceUriStr = ((ClientConfiguration) conf).getMetadataServiceUri();
+ URI metadataServiceUri = URI.create(metadataServiceUriStr);
+ String zkServers;
+ String ledgersRootPath = metadataServiceUri.getPath();
+ if (ZK_LEDGERS_DEFAULT_ROOT_PATH.equals(ledgersRootPath)) {
+ zkServers = getZKServersFromServiceUri(metadataServiceUri);
Review comment:
do we need to add /ledgers as uri path?
##########
File path: pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
##########
@@ -56,6 +58,7 @@
public static final String BOOKIE_INFO_ROOT_PATH = "/bookies";
public static final String ZK_DATA_CACHE_BK_RACK_CONF_INSTANCE = "zk_data_cache_bk_rack_conf_instance";
+ public static final String ZK_LEDGERS_DEFAULT_ROOT_PATH = "/ledgers";
Review comment:
private?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaozhangmin commented on a change in pull request #11947: fix-npe-ZkBookieRackAffinityMapping
Posted by GitBox <gi...@apache.org>.
gaozhangmin commented on a change in pull request #11947:
URL: https://github.com/apache/pulsar/pull/11947#discussion_r704137897
##########
File path: pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
##########
@@ -123,18 +126,20 @@ private void updateRacksWithHost(BookiesRackConfiguration racks) {
if (conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE) != null) {
zkCache = (ZooKeeperCache) conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE);
} else {
- int zkTimeout;
- String zkServers;
if (conf instanceof ClientConfiguration) {
- zkTimeout = ((ClientConfiguration) conf).getZkTimeout();
- zkServers = ((ClientConfiguration) conf).getZkServers();
- String zkLedgersRootPath = ((ClientConfiguration) conf).getZkLedgersRootPath();
+ int zkTimeout = ((ClientConfiguration) conf).getZkTimeout();
try {
- int zkLedgerRootIndex = zkLedgersRootPath.contains("/") ?
- zkLedgersRootPath.lastIndexOf("/") : 0;
- String zkChangeRoot = zkLedgersRootPath.substring(0, zkLedgerRootIndex);
- zkChangeRoot = zkChangeRoot.startsWith("/") ? zkChangeRoot : "/" + zkChangeRoot;
- ZooKeeper zkClient = ZooKeeperClient.newBuilder().connectString(zkServers + zkChangeRoot)
+ final String metadataServiceUriStr = ((ClientConfiguration) conf).getMetadataServiceUri();
+ URI metadataServiceUri = URI.create(metadataServiceUriStr);
+ String zkServers;
+ String ledgersRootPath = metadataServiceUri.getPath();
+ if (ZK_LEDGERS_DEFAULT_ROOT_PATH.equals(ledgersRootPath)) {
+ zkServers = getZKServersFromServiceUri(metadataServiceUri);
Review comment:
from the previous logic, it was no.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaozhangmin commented on a change in pull request #11947: fix-npe-ZkBookieRackAffinityMapping
Posted by GitBox <gi...@apache.org>.
gaozhangmin commented on a change in pull request #11947:
URL: https://github.com/apache/pulsar/pull/11947#discussion_r704143371
##########
File path: pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
##########
@@ -123,18 +126,20 @@ private void updateRacksWithHost(BookiesRackConfiguration racks) {
if (conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE) != null) {
zkCache = (ZooKeeperCache) conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE);
} else {
- int zkTimeout;
- String zkServers;
if (conf instanceof ClientConfiguration) {
- zkTimeout = ((ClientConfiguration) conf).getZkTimeout();
- zkServers = ((ClientConfiguration) conf).getZkServers();
- String zkLedgersRootPath = ((ClientConfiguration) conf).getZkLedgersRootPath();
+ int zkTimeout = ((ClientConfiguration) conf).getZkTimeout();
try {
- int zkLedgerRootIndex = zkLedgersRootPath.contains("/") ?
- zkLedgersRootPath.lastIndexOf("/") : 0;
- String zkChangeRoot = zkLedgersRootPath.substring(0, zkLedgerRootIndex);
- zkChangeRoot = zkChangeRoot.startsWith("/") ? zkChangeRoot : "/" + zkChangeRoot;
- ZooKeeper zkClient = ZooKeeperClient.newBuilder().connectString(zkServers + zkChangeRoot)
+ final String metadataServiceUriStr = ((ClientConfiguration) conf).getMetadataServiceUri();
+ URI metadataServiceUri = URI.create(metadataServiceUriStr);
+ String zkServers;
+ String ledgersRootPath = metadataServiceUri.getPath();
+ if (ZK_LEDGERS_DEFAULT_ROOT_PATH.equals(ledgersRootPath)) {
+ zkServers = getZKServersFromServiceUri(metadataServiceUri);
Review comment:
https://github.com/apache/pulsar/pull/9894
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org