You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by zh...@apache.org on 2020/08/26 23:46:39 UTC
[bookkeeper] branch master updated: Trigger flush if write cache is
full while waiting (#2402)
This is an automated email from the ASF dual-hosted git repository.
zhaijia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
The following commit(s) were added to refs/heads/master by this push:
new a0d49fc Trigger flush if write cache is full while waiting (#2402)
a0d49fc is described below
commit a0d49fcb65a40b2b49fbfb6c50f60c7a9f66f67f
Author: Matteo Merli <mm...@apache.org>
AuthorDate: Wed Aug 26 16:46:28 2020 -0700
Trigger flush if write cache is full while waiting (#2402)
Under heavy load, the write cache can fill up faster than it can be
flushed. When this occurs, something needs to trigger the
flush. Previously we've triggered the flush and then gone into a loop
waiting to be able to add to the write cache. However, if the
currently active write cache fills while we are waiting for the
flushing write cache to be flushed, we can end up in a situation where
nothing is able to flush because all I/O threads are waiting for the
write cache to empty.
The solution is to move the flush into the loop so that if we fail to
add to the cache, we flush the write cache if it is needed.
Co-authored-by: Ivan Kelly <ik...@splunk.com>
---
.../ldb/SingleDirectoryDbLedgerStorage.java | 30 +++++++++++-----------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/SingleDirectoryDbLedgerStorage.java b/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/SingleDirectoryDbLedgerStorage.java
index 931c3e0..9ff35bf 100644
--- a/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/SingleDirectoryDbLedgerStorage.java
+++ b/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/SingleDirectoryDbLedgerStorage.java
@@ -338,25 +338,25 @@ public class SingleDirectoryDbLedgerStorage implements CompactableLedgerStorage
private void triggerFlushAndAddEntry(long ledgerId, long entryId, ByteBuf entry)
throws IOException, BookieException {
- // Write cache is full, we need to trigger a flush so that it gets rotated
- // If the flush has already been triggered or flush has already switched the
- // cache, we don't need to trigger another flush
- if (!isFlushOngoing.get() && hasFlushBeenTriggered.compareAndSet(false, true)) {
- // Trigger an early flush in background
- log.info("Write cache is full, triggering flush");
- executor.execute(() -> {
- try {
- flush();
- } catch (IOException e) {
- log.error("Error during flush", e);
- }
- });
- }
-
dbLedgerStorageStats.getThrottledWriteRequests().inc();
long absoluteTimeoutNanos = System.nanoTime() + maxThrottleTimeNanos;
while (System.nanoTime() < absoluteTimeoutNanos) {
+ // Write cache is full, we need to trigger a flush so that it gets rotated
+ // If the flush has already been triggered or flush has already switched the
+ // cache, we don't need to trigger another flush
+ if (!isFlushOngoing.get() && hasFlushBeenTriggered.compareAndSet(false, true)) {
+ // Trigger an early flush in background
+ log.info("Write cache is full, triggering flush");
+ executor.execute(() -> {
+ try {
+ flush();
+ } catch (IOException e) {
+ log.error("Error during flush", e);
+ }
+ });
+ }
+
long stamp = writeCacheRotationLock.readLock();
try {
if (writeCache.put(ledgerId, entryId, entry)) {