You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/08/07 09:11:02 UTC
Slack digest for #general - 2018-08-07
2018-08-06 09:53:50 UTC - wuhaoyang: @wuhaoyang has joined the channel
----
2018-08-06 09:57:25 UTC - lipeng: @lipeng has joined the channel
----
2018-08-06 09:59:47 UTC - EvenGui: @EvenGui has joined the channel
----
2018-08-06 17:28:08 UTC - Vasily Yanov: Hi! Guys, trying to start bookie but on 2 of 4 nodes and getting:
`java.net.UnknownHostException: Unable to resolve default hostname: <http://static.88-99-145-39.clients.your-server.de|static.88-99-145-39.clients.your-server.de>. for interface: eth0`
any ideas why? another 2 nodes have same config and no issues on start
----
2018-08-06 17:28:22 UTC - Vasily Yanov: pulsar 2.1
----
2018-08-06 18:02:46 UTC - Ali Ahmed: is this in mesos ?
----
2018-08-06 18:05:42 UTC - Sijie Guo: @Vasily Yanov it seems that the host can’t resolve the hostname `<http://static.88-99-145-39.clients.your-server.de|static.88-99-145-39.clients.your-server.de>`. on that host, can you run `host <http://static.88-99-145-39.clients.your-server.de|static.88-99-145-39.clients.your-server.de>` and `hostname`?
----
2018-08-06 18:21:46 UTC - Vasily Yanov: yes
----
2018-08-06 18:22:29 UTC - Vasily Yanov: reverse dns points to `<http://static.88-99-145-39.clients.your-server.de|static.88-99-145-39.clients.your-server.de>` but there is no DNS record for `<http://static.88-99-145-39.clients.your-server.de|static.88-99-145-39.clients.your-server.de>`
----
2018-08-06 18:22:50 UTC - Vasily Yanov: but why it so important to bind to interface?
----
2018-08-06 18:25:09 UTC - Sijie Guo: so the bookie has to advertise its address so that brokers can discover and connect to it. so it doesn’t has to be eth0, it can be any interfaces that allows your brokers to connect.
how many nic interfaces does your host have? I am wondering if it is possible it attempts to bind to a different interface.
----
2018-08-06 18:29:15 UTC - Vasily Yanov: 1 eth intercace and 1 lo interface. so no possibility to choose another one. is it possible to advertise bookie with IP address instead of FQDN?
----
2018-08-06 18:29:30 UTC - Sijie Guo: yes
----
2018-08-06 18:29:36 UTC - Sijie Guo: you can set useHostname to false
----
2018-08-06 18:30:01 UTC - Sijie Guo: <https://github.com/apache/incubator-pulsar/blob/master/conf/bookkeeper.conf#L97>
----
2018-08-06 18:33:57 UTC - Vasily Yanov: it is already set to false:
`useHostNameAsBookieID=false`
----
2018-08-06 18:34:35 UTC - Sijie Guo: interesting, do you use advertisedAddress?
----
2018-08-06 18:35:09 UTC - Vasily Yanov: yes
----
2018-08-06 18:36:15 UTC - Vasily Yanov: ```
listeningInterface=eth0
advertisedAddress=88.99.148.111
allowLoopback=true
```
----
2018-08-06 18:39:39 UTC - Sijie Guo: okay give me one second
----
2018-08-06 18:43:18 UTC - Sijie Guo: hmm if you set `advertisedAddress`, it will take the ip address to construct an `InetSocketAddress`. so it seems that exception is thrown from java `InetSocketAddress`.
----
2018-08-06 18:44:52 UTC - Vasily Yanov: ok. When I set advAddr:
----
2018-08-06 18:45:19 UTC - Vasily Yanov: ```
org.apache.bookkeeper.bookie.BookieException$UnknownBookieIdException: java.net.UnknownHostException: Unable to resolve default hostname: <http://static.88-99-148-235.clients.your-server.de|static.88-99-148-235.clients.your-server.de>. for interface: eth0
```
----
2018-08-06 18:45:24 UTC - Vasily Yanov: when no:
----
2018-08-06 18:45:44 UTC - Vasily Yanov: ```
java.net.UnknownHostException: Unable to resolve default hostname: <http://static.88-99-148-235.clients.your-server.de|static.88-99-148-235.clients.your-server.de>. for interface: eth0
```
----
2018-08-06 18:49:55 UTC - Sijie Guo: can you paste to full exception stack traces? so I can find which line the exception was thrown.
----
2018-08-06 18:51:32 UTC - Vasily Yanov: yes. sure:
```
18:51:17.258 [main] ERROR org.apache.bookkeeper.server.Main - Failed to build bookie server
org.apache.bookkeeper.bookie.BookieException$UnknownBookieIdException: java.net.UnknownHostException: Unable to resolve default hostname: <http://static.88-99-148-235.clients.your-server.de|static.88-99-148-235.clients.your-server.de>. for interface: eth0
at org.apache.bookkeeper.bookie.Bookie.possibleBookieIds(Bookie.java:325) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.bookie.Bookie.checkEnvironmentWithStorageExpansion(Bookie.java:415) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.bookie.Bookie.checkEnvironment(Bookie.java:256) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.bookie.Bookie.<init>(Bookie.java:627) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:115) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.proto.BookieServer.<init>(BookieServer.java:96) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.server.service.BookieService.<init>(BookieService.java:42) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:299) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.server.Main.doMain(Main.java:219) [org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.server.Main.main(Main.java:201) [org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.proto.BookieServer.main(BookieServer.java:252) [org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
Caused by: java.net.UnknownHostException: Unable to resolve default hostname: <http://static.88-99-148-235.clients.your-server.de|static.88-99-148-235.clients.your-server.de>. for interface: eth0
at org.apache.bookkeeper.bookie.Bookie.getBookieAddress(Bookie.java:542) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
at org.apache.bookkeeper.bookie.Bookie.possibleBookieIds(Bookie.java:315) ~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
... 10 more
```
----
2018-08-06 18:51:53 UTC - Vasily Yanov: with advAddr set
----
2018-08-06 18:56:40 UTC - Sijie Guo: the stack trace doesn’t look right to me. if `advertisedAddress` is set, it shouldn’t be executing line 542: <https://github.com/apache/bookkeeper/blob/release-4.7.1/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java#L542>
it should be using the `advertisedAddress` : <https://github.com/apache/bookkeeper/blob/release-4.7.1/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java#L529>
are you sure you set `advertisedAddress` and the bookie is using the right configuration file?
----
2018-08-06 18:58:22 UTC - Vasily Yanov: bookie starts with "--conf /opt/pulsar/conf/bookkeeper.conf"
----
2018-08-06 18:58:59 UTC - Vasily Yanov: and when I try to change something in mentioned config I see result during bookie start
----
2018-08-06 19:00:16 UTC - Vasily Yanov: config file
```
bookiePort=3181
journalDirectory=data/bookkeeper/journal
minUsableSizeForIndexFileCreation=1073741824
listeningInterface=eth0
advertisedAddress=88.99.148.235
allowLoopback=true
bookieDeathWatchInterval=1000
flushInterval=60000
useHostNameAsBookieID=false
gcWaitTime=900000
gcOverreplicatedLedgerWaitTime=86400000
numAddWorkerThreads=0
numReadWorkerThreads=8
numHighPriorityWorkerThreads=8
maxPendingReadRequestsPerThread=2500
maxPendingAddRequestsPerThread=10000
auditorPeriodicBookieCheckInterval=86400
rereplicationEntryBatchSize=5000
serverTcpNoDelay=true
journalMaxSizeMB=2048
journalMaxBackups=5
journalPreAllocSizeMB=16
journalWriteBufferSizeKB=64
journalRemoveFromPageCache=true
journalSyncData=false
journalAdaptiveGroupWrites=true
journalMaxGroupWaitMSec=1
journalBufferedWritesThreshold=524288
numJournalCallbackThreads=8
journalAlignmentSize=4096
journalFlushWhenQueueEmpty=false
ledgerStorageClass=org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage
ledgerDirectories=data/bookkeeper/ledgers
auditorPeriodicCheckInterval=604800
openFileLimit=0
pageLimit=0
ledgerManagerFactoryClass=org.apache.bookkeeper.meta.HierarchicalLedgerManagerFactory
zkLedgersRootPath=/ledgers
logSizeLimit=1073741824
entryLogFilePreallocationEnabled=true
readBufferSizeBytes=4096
writeBufferSizeBytes=65536
compactionRate=1000
minorCompactionThreshold=0.2
minorCompactionInterval=3600
compactionMaxOutstandingRequests=100000
majorCompactionThreshold=0.5
majorCompactionInterval=86400
isThrottleByBytes=false
compactionRateByEntries=1000
compactionRateByBytes=1000000
statsProviderClass=org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider
prometheusStatsHttpPort=8005
readOnlyModeEnabled=true
diskUsageThreshold=0.95
diskCheckInterval=10000
zkServers=<http://dev-pulsar21-01.onde.in:2181,dev-pulsar21-02.onde.in:2181,dev-pulsar21-03.onde.in:2181|dev-pulsar21-01.onde.in:2181,dev-pulsar21-02.onde.in:2181,dev-pulsar21-03.onde.in:2181>
zkTimeout=30000
zkEnableSecurity=false
dbStorage_writeCacheMaxSizeMb=512
dbStorage_readAheadCacheMaxSizeMb=256
dbStorage_readAheadCacheBatchSize=1000
dbStorage_rocksDB_blockCacheSize=268435456
dbStorage_rocksDB_writeBufferSizeMB=64
dbStorage_rocksDB_sstSizeInMB=64
dbStorage_rocksDB_blockSize=65536
dbStorage_rocksDB_bloomFilterBitsPerKey=10
dbStorage_rocksDB_numLevels=-1
dbStorage_rocksDB_numFilesInLevel0=4
dbStorage_rocksDB_maxSizeInLevel1MB=256
```
----
2018-08-06 19:17:01 UTC - Vasily Yanov: looks like the issue is solved. I just commented out `listeningInterface=eth0`
----
2018-08-06 19:29:47 UTC - Sijie Guo: Cool. Glad to see it is resolved
----
2018-08-06 21:34:03 UTC - Sijie Guo: sorry I miss this thread. from the topic stats, it seems that all publishers are sending messages. I didn’t see any msgThroughputOut being 0. can anyone give more context around the areas that I can look into?
----
2018-08-07 01:24:25 UTC - Abhishek Sharma: @Abhishek Sharma has joined the channel
----