You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2021/12/16 06:57:09 UTC

[GitHub] [bookkeeper] dragonls edited a comment on issue #2945: Bookkeeper "READ_ENTRY (Inprogress)" is not 0

dragonls edited a comment on issue #2945:
URL: https://github.com/apache/bookkeeper/issues/2945#issuecomment-995493274


   The log shows that lots of `No ledger found while trying to read last entry: xxx` first:
   ```
   22:25:19.742 [db-storage-14-1] INFO  org.apache.bookkeeper.bookie.Journal - garbage collected journal 17ce633929b.txn
   22:25:20.292 [bookie-io-1-9] INFO  org.apache.bookkeeper.proto.AuthHandler - Authentication success on server side
   22:25:20.292 [bookie-io-1-9] INFO  org.apache.bookkeeper.proto.BookieRequestHandler - Channel connected  [id: 0x793e94c8, L:/9.146.225.131:3181 - R:/9.146.225.179:40152]
   22:25:20.440 [BookieReadThreadPool-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.proto.ReadLacProcessorV3 - No ledger found while performing readLac from ledger: 930007
   org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 930007 not found
           at org.apache.bookkeeper.bookie.LedgerDescriptor.createReadOnly(LedgerDescriptor.java:52) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.bookie.HandleFactoryImpl.getReadOnlyHandle(HandleFactoryImpl.java:61) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.bookie.Bookie.getExplicitLac(Bookie.java:1364) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.ReadLacProcessorV3.getReadLacResponse(ReadLacProcessorV3.java:71) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.ReadLacProcessorV3.safeRun(ReadLacProcessorV3.java:118) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_302]
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_302]
           at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.60.Final.jar:4.1.60.Final]
           at java.lang.Thread.run(Thread.java:748) [?:1.8.0_302]
   22:25:20.441 [BookieReadThreadPool-OrderedExecutor-7-0] INFO  org.apache.bookkeeper.proto.ReadEntryProcessorV3 - No ledger found while reading entry: 0 from ledger: 930007
   22:25:20.441 [BookieReadThreadPool-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.proto.ReadLacProcessorV3 - No ledger found while trying to read last entry: 930007
   org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 930007 not found
           at org.apache.bookkeeper.bookie.LedgerDescriptor.createReadOnly(LedgerDescriptor.java:52) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.bookie.HandleFactoryImpl.getReadOnlyHandle(HandleFactoryImpl.java:61) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.bookie.Bookie.readEntry(Bookie.java:1441) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.ReadLacProcessorV3.getReadLacResponse(ReadLacProcessorV3.java:86) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.ReadLacProcessorV3.safeRun(ReadLacProcessorV3.java:118) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_302]
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_302]
           at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.60.Final.jar:4.1.60.Final]
           at java.lang.Thread.run(Thread.java:748) [?:1.8.0_302]
   22:25:20.441 [BookieReadThreadPool-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.proto.ReadLacProcessorV3 - No ledger found while performing readLac from ledger: 930018
   org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 930018 not found
           at org.apache.bookkeeper.bookie.LedgerDescriptor.createReadOnly(LedgerDescriptor.java:52) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.bookie.HandleFactoryImpl.getReadOnlyHandle(HandleFactoryImpl.java:61) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.bookie.Bookie.getExplicitLac(Bookie.java:1364) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.ReadLacProcessorV3.getReadLacResponse(ReadLacProcessorV3.java:71) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.ReadLacProcessorV3.safeRun(ReadLacProcessorV3.java:118) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_302]
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_302]
           at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.60.Final.jar:4.1.60.Final]
           at java.lang.Thread.run(Thread.java:748) [?:1.8.0_302]
   ```
   
   And then lots of `Bookie operation timeout`:
   ```
   22:25:32.828 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - Timed-out 2 operations to channel [id: 0xbc5f70b3, L:/9.146.225.131:33454 - R:9.146.225.169/9.146.225.169:3181] for 9.146.225.169:3181
   22:25:32.829 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - Timed-out 2 operations to channel [id: 0x0191e854, L:/9.146.225.131:55480 - R:9.146.225.110/9.146.225.110:3181] for 9.146.225.110:3181
   22:25:32.829 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - Timed-out 2 operations to channel [id: 0x2feb6cb8, L:/9.146.225.131:40600 - R:9.146.225.131/9.146.225.131:3181] for 9.146.225.131:3181
   22:25:32.829 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - Timed-out 2 operations to channel [id: 0x9ce6a3c1, L:/9.146.225.131:36128 - R:9.146.225.128/9.146.225.128:3181] for 9.146.225.128:3181
   22:25:32.829 [ReplicationWorker] INFO  org.apache.bookkeeper.client.LedgerFragmentReplicator - Replicating fragment Fragment(LedgerID: 1212480, FirstEntryID: 0[0], LastKnownEntryID: 49999[49999], Host: [9.146.225.169:3181, 9.146.225.110:3181, 9.146.225.131:3181, 9.146.225.128:3181], Closed: true) in 50 sub fragments.
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Bookie operation timeout while reading L1212480 E36190 from bookie: 9.146.225.131:3181
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.client.PendingReadOp - Received error from a host which is not in the ensemble 9.146.225.131:3181 [9.146.225.110:3181, 9.146.225.169:3181, 9.146.225.131:3181, 9.146.225.128:3181].
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Bookie operation timeout while reading L1212480 E36298 from bookie: 9.146.225.131:3181
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Bookie operation timeout while reading L1212480 E36050 from bookie: 9.146.225.131:3181
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Bookie operation timeout while reading L1212480 E36006 from bookie: 9.146.225.131:3181
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Bookie operation timeout while reading L1212480 E36022 from bookie: 9.146.225.131:3181
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Bookie operation timeout while reading L1212480 E36326 from bookie: 9.146.225.131:3181
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.client.PendingReadOp - Received error from a host which is not in the ensemble 9.146.225.131:3181 [9.146.225.110:3181, 9.146.225.169:3181, 9.146.225.131:3181, 9.146.225.128:3181].
   22:25:40.828 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Bookie operation timeout while reading L1212480 E36762 from bookie: 9.146.225.131:3181
   
   ```
   
   Also some `LedgerFencedException`:
   ```
   22:26:17.715 [BookieHighPriorityThread-3181-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.proto.ReadEntryProcessorV3 - Ledger fence request received for ledger: 1212888 from address: /9.146.225.170:56668
   22:26:17.719 [bookie-io-1-7] ERROR org.apache.bookkeeper.proto.WriteEntryProcessor - Attempt to write to fenced ledger
   org.apache.bookkeeper.bookie.BookieException$LedgerFencedException: null
           at org.apache.bookkeeper.bookie.BookieException.create(BookieException.java:57) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1399) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.WriteEntryProcessor.processPacket(WriteEntryProcessor.java:80) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.PacketProcessorBase.safeRun(PacketProcessorBase.java:85) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.BookieRequestProcessor.processAddRequest(BookieRequestProcessor.java:625) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.BookieRequestProcessor.processRequest(BookieRequestProcessor.java:365) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.BookieRequestHandler.channelRead(BookieRequestHandler.java:80) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at org.apache.bookkeeper.proto.AuthHandler$ServerSideHandler.channelRead(AuthHandler.java:95) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at org.apache.bookkeeper.proto.BookieProtoEncoding$RequestDecoder.channelRead(BookieProtoEncoding.java:450) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [io.netty-netty-codec-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [io.netty-netty-codec-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
   
   ```
   
   And `Too many requests to the same Bookie`:
   ```
   22:26:23.356 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Too many requests to the same Bookie while reading L1212480 E562 from bookie: 9.146.225.131:3181
   22:26:23.356 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Too many requests to the same Bookie while reading L1212480 E566 from bookie: 9.146.225.131:3181
   22:26:23.356 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Too many requests to the same Bookie while reading L1212480 E570 from bookie: 9.146.225.131:3181
   22:26:23.356 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Too many requests to the same Bookie while reading L1212480 E574 from bookie: 9.146.225.131:3181
   22:26:23.356 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.PendingReadOp - Error: Too many requests to the same Bookie while reading L1212480 E578 from bookie: 9.146.225.131:3181
   
   ```
   
   Also `Queue at limit of 5000 items`:
   ```
   22:26:31.915 [bookie-io-1-11] ERROR org.apache.bookkeeper.proto.BookieRequestHandler - Unhandled exception occurred in I/O thread or handler on [id: 0xffe37d0c, L:/9.146.225.131:3181 - R:/9.146.225.131:40600]
   java.util.concurrent.RejectedExecutionException: Queue at limit of 5000 items
           at org.apache.bookkeeper.common.util.BoundedExecutorService.checkQueue(BoundedExecutorService.java:56) ~[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.common.util.BoundedExecutorService.execute(BoundedExecutorService.java:88) ~[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
           at com.google.common.util.concurrent.ForwardingExecutorService.execute(ForwardingExecutorService.java:99) ~[com.google.guava-guava-30.1-jre.jar:?]
           at org.apache.bookkeeper.common.util.OrderedExecutor$1.execute(OrderedExecutor.java:341) ~[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.common.util.OrderedExecutor.executeOrdered(OrderedExecutor.java:493) ~[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.BookieRequestProcessor.processReadLacRequestV3(BookieRequestProcessor.java:416) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.BookieRequestProcessor.processRequest(BookieRequestProcessor.java:334) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at org.apache.bookkeeper.proto.BookieRequestHandler.channelRead(BookieRequestHandler.java:80) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at org.apache.bookkeeper.proto.AuthHandler$ServerSideHandler.channelRead(AuthHandler.java:95) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at org.apache.bookkeeper.proto.BookieProtoEncoding$RequestDecoder.channelRead(BookieProtoEncoding.java:450) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [io.netty-netty-codec-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [io.netty-netty-codec-4.1.60.Final.jar:4.1.60.Final]
           at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.60.Final.jar:4.1.60.Final]
   ```
   
   Finally, the log seems back to normal:
   ```
   22:26:42.893 [bookie-io-1-12] INFO  org.apache.bookkeeper.proto.BookieRequestHandler - Channels disconnected: [id: 0xd543c588, L:/9.146.225.131:3181 ! R:/9.146.225.124:35706]
   22:26:47.911 [main-EventThread] INFO  org.apache.bookkeeper.meta.ZkLedgerUnderreplicationManager - Latch countdown due to ZK event: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/ledgers/underreplication/locks
   22:26:47.913 [main-EventThread] INFO  org.apache.bookkeeper.meta.ZkLedgerUnderreplicationManager - Latch countdown due to ZK event: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/ledgers/underreplication/locks
   22:26:48.793 [bookie-io-1-10] INFO  org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage - Write cache is full, triggering flush
   22:26:49.881 [bookie-io-1-6] INFO  org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage - Write cache is full, triggering flush
   22:26:50.027 [db-storage-14-1] INFO  org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true
   22:26:50.028 [db-storage-14-1] INFO  org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 134253 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=134253, logFile=/data3/mmdcpulsarbookie/ledgers/current/20c6d.log, ledgerIdAssigned=-1}].
   22:26:50.029 [pool-6-thread-1] INFO  org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file /data3/mmdcpulsarbookie/ledgers/current/20c6f.log for logId 134255.
   22:26:50.217 [db-storage-11-1] INFO  org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true
   22:26:50.217 [db-storage-11-1] INFO  org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 134761 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=134761, logFile=/data2/mmdcpulsarbookie/ledgers/current/20e69.log, ledgerIdAssigned=-1}].
   22:26:50.218 [pool-5-thread-1] INFO  org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file /data2/mmdcpulsarbookie/ledgers/current/20e6b.log for logId 134763.
   22:26:51.646 [db-storage-14-1] INFO  org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true
   22:26:51.647 [db-storage-14-1] INFO  org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 134254 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=134253, logFile=/data3/mmdcpulsarbookie/ledgers/current/20c6d.log, ledgerIdAssigned=-1}, BufferedChannel{logId=134254, logFile=/data3/mmdcpulsarbookie/ledgers/current/20c6e.log, ledgerIdAssigned=-1}].
   22:26:51.648 [pool-6-thread-1] INFO  org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file /data3/mmdcpulsarbookie/ledgers/current/20c70.log for logId 134256.
   
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org