You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/09/10 14:11:57 UTC
[GitHub] [pulsar] savearray2 commented on issue #8027: Issue applying region/rack information (ZkBookieRackAffinityMapping)
savearray2 commented on issue #8027:
URL: https://github.com/apache/pulsar/issues/8027#issuecomment-690316104
Hi there, @mkozioro
I've rewritten my configuration:
```
# Enable rack-aware bookie selection policy. BK will chose bookies from different racks when
# forming a new bookie ensemble
bookkeeperClientRackawarePolicyEnabled=true
# Enable region-aware bookie selection policy. BK will chose bookies from
# different regions and racks when forming a new bookie ensemble
# If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
bookkeeperClientRegionawarePolicyEnabled=false
# Enable/disable reordering read sequence on reading entries.
bookkeeperClientReorderReadSequenceEnabled=true
```
Here's the output from ```racks-placement```:
```
# ./pulsar-admin bookies racks-placement
{
"default" : {
"koto-bk-aen-0:3181" : {
"rack" : "/aen"
},
"koto-bk-aen-1:3181" : {
"rack" : "/aen"
},
"koto-bk-aen-0" : {
"rack" : "/aen"
},
"koto-bk-aen-1" : {
"rack" : "/aen"
}
}
}
```
I still get the same error (on the broker):
```
14:00:28.490 [main] ERROR org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl - Failed to initialize DNS Resolver org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping, used default subnet resolver : java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.net.UnknownHostException: koto-bk-aen-0
14:00:28.489 [ForkJoinPool.commonPool-worker-1] WARN com.github.benmanes.caffeine.cache.LocalAsyncLoadingCache - Exception thrown during asynchronous load
java.lang.RuntimeException: java.net.UnknownHostException: koto-bk-aen-0
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.lambda$1(ZkBookieRackAffinityMapping.java:103) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.6.1.jar:2.6.1]
at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684) ~[?:1.8.0_252]
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.lambda$0(ZkBookieRackAffinityMapping.java:85) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.6.1.jar:2.6.1]
at java.util.TreeMap.forEach(TreeMap.java:1005) ~[?:1.8.0_252]
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.updateRacksWithHost(ZkBookieRackAffinityMapping.java:84) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.6.1.jar:2.6.1]
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.access$3(ZkBookieRackAffinityMapping.java:78) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.6.1.jar:2.6.1]
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping$2.deserialize(ZkBookieRackAffinityMapping.java:154) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.6.1.jar:2.6.1]
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping$2.deserialize(ZkBookieRackAffinityMapping.java:1) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.6.1.jar:2.6.1]
at org.apache.pulsar.zookeeper.ZooKeeperCache.lambda$13(ZooKeeperCache.java:362) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.6.1.jar:2.6.1]
at org.apache.bookkeeper.zookeeper.ZooKeeperClient$19$1.processResult(ZooKeeperClient.java:994) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:598) ~[org.apache.pulsar-pulsar-zookeeper-2.6.1.jar:2.6.1]
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) ~[org.apache.pulsar-pulsar-zookeeper-2.6.1.jar:2.6.1]
Caused by: java.net.UnknownHostException: koto-bk-aen-0
at org.apache.bookkeeper.net.BookieSocketAddress.<init>(BookieSocketAddress.java:68) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0]
at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.lambda$1(ZkBookieRackAffinityMapping.java:87) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.6.1.jar:2.6.1]
... 11 more
```
Which still results in the following:
```
14:00:28.536 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/koto-bk-aen-0:3181
14:00:28.536 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/koto-bk-aen-1:3181
```
Could I ask you a question?
I'm currently using the following as part of my ```bookkeeper.conf```:
```
# Port that bookie server listen on
bookiePort=3181
# Configure a specific hostname or IP address that the bookie should use to advertise itself to
# clients. If not set, bookie will advertised its own IP address or hostname, depending on the
# listeningInterface and useHostNameAsBookieID settings.
advertisedAddress=koto-bk-aen-0
# Whether the bookie should use its hostname to register with the
# co-ordination service(eg: Zookeeper service).
# When false, bookie will use its ip address for the registration.
# Defaults to false.
useHostNameAsBookieID=true
```
Do you include the port (```:3181```) on the ```advertisedAddress``` value (e.g. ```advertisedAddress=koto-bk-aen-0:3181```)? I'm wondering if this has something to do with the problem...
As an example my bookkeeper node starts up with the following log:
```
13:51:30.989 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=koto-bk-aen-0
```
Also included in the bookkeeper node log is this:
```
13:51:34.855 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/koto-bk-aen-0:3181
13:51:34.855 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/koto-bk-aen-1:3181
```
But I'm not sure if this matters for bookkeeper...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org