[GitHub] [bookkeeper] OlegKashtanov commented on issue #3258: Bookkeeper 4.15 shows a write throughput drop of ~20% compared to Bookkeeper 4.14

[GitBox <gi...@apache.org>]

OlegKashtanov commented on issue #3258:
URL: https://github.com/apache/bookkeeper/issues/3258#issuecomment-1118235944

   > Can you include the throughput charts on the bookies too?
   
   I've added the Bookies write throughput charts. We can observe the same throughput difference as on Pravega side.
   ![bk-write-tp](https://user-images.githubusercontent.com/8101276/166875897-7dd86224-568c-4ab4-845e-1b3214c151a2.png)
   
   > Are you using a docker image build from Bk repo? r are you using some BK package built from Pravega?
   
   We are using the BK code from apache/bookkeeper repo for building the BK package. After that we build the BK docker image using our own Dockerfile with different base image. In our case it's Alpine.
   
   >  Can you share the bk configuration? Is is exactly the same with 4.14 and 4.15? 
   
   We are using the https://github.com/pravega/bookkeeper-operator for BK deploying. We overwrite some of the options via operator, but for that level there is no difference for deployment between 4.14 and 4.15 - only the BK docker image. We've checked these BK configuration options and all of them were applied by both BK servers (checked it in the logs).
   But today we've compared the bk_server.conf file for 4.14 and 4.15 and observed several new added options in 4.15 which could impact in our result:
   ```
   +# Set the Channel Provider for journal.
   +# The default value is
   +# journalChannelProvider=org.apache.bookkeeper.bookie.DefaultFileChannelProvider
   ...
   +# Max number of concurrent requests in garbage collection of overreplicated ledgers.
   +# gcOverreplicatedLedgerMaxConcurrentRequests=1000
   ...
   +# True if bookie should persist entrylog file metadata and avoid in-memory object allocation
   +gcEntryLogMetadataCacheEnabled=false
   +# Directory to persist Entrylog metadata if gcPersistentEntrylogMetadataMapEnabled is true
   +# [Default: it creates a sub-directory under a first available base ledger directory with
   +# name "entrylogIndexCache"]
   +# gcEntryLogMetadataCachePath=
   ...
   +# Semaphore limit of getting ledger from zookeeper. Used to throttle the zookeeper client request operation
   +# sending to Zookeeper server. Default value is 500
   +# auditorMaxNumberOfConcurrentOpenLedgerOperations=500
   ...
   +# Semaphore limit of getting ledger from zookeeper. Used to throttle the zookeeper client request operation
   +# sending to Zookeeper server. Default value is 500
   +# auditorMaxNumberOfConcurrentOpenLedgerOperations=500
   ```
   We'll add the flamegraphs, which could help to identify the root cause IMO.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org