You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2018/08/03 01:52:24 UTC
[GitHub] hrsakai opened a new issue #1578: Auditor run Periodic check only
once
hrsakai opened a new issue #1578: Auditor run Periodic check only once
URL: https://github.com/apache/bookkeeper/issues/1578
**BUG REPORT**
**What did you do?**
In our cluster, Auditor run periodic check only once. If interval expires after first periodic check, auditor will not run periodic check.
If we want to run periodic check again, we have to restart auditor bookie.
Auditor's thread dump
It seems that `AuditorBookie` thread stop by `CountDownLatch` with some reason.
https://gist.github.com/hrsakai/d65e8e2cd511173232b1010a9bbdf126
```
"AuditorBookie-XXXXX:3181" #40 daemon prio=5 os_prio=0 tid=0x00007f049c117830 nid=0x5da4 waiting on condition [0x00007f0477dfc000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e04e54f8> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at org.apache.bookkeeper.replication.Auditor.checkAllLedgers(Auditor.java:696)
at org.apache.bookkeeper.replication.Auditor$5.run(Auditor.java:359)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
I saw many timed-out logs in Auditor's log file.
```
23:14:56.769 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO o.a.b.proto.PerChannelBookieClient - Timed-out 2 operations to channel [id: 0xa17320e1, L:/AAAA:38234 - R:XXXX.co.jp/BBBB:3181] for BBBB:3181
23:14:56.921 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO o.a.b.proto.PerChannelBookieClient - Timed-out 48 operations to channel [id: 0x359c20bd, L:/AAAA:38222 - R:BBBB/BBBB:3181] for BBBB:3181
23:15:37.768 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO o.a.b.proto.PerChannelBookieClient - Timed-out 2 operations to channel [id: 0xa17320e1, L:/AAAA:38234 - R:XXXX.co.jp/BBBB:3181] for BBBB:3181
23:15:37.921 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO o.a.b.proto.PerChannelBookieClient - Timed-out 91 operations to channel [id: 0x359c20bd, L:/AAAA:38222 - R:BBBB/BBBB:3181] for BBBB:3181
・
・
```
**What did you expect to see?**
Auditor run periodic check after every interval expires.
**What did you see instead?**
Auditor run periodic check only once.
**System configuration**
BookKeeper version : 4.7.0
Number of Bookies: 5
Ensemble size: 2
Write quorum size: 2
Ack quorum size: 2
Priodic check interval: 1day
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services