You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by "thetumbled (via GitHub)" <gi...@apache.org> on 2023/12/21 08:47:10 UTC

[PR] [Improvement] skip read rebundunct content when scanning entry log. [bookkeeper]

thetumbled opened a new pull request, #4161:
URL: https://github.com/apache/bookkeeper/pull/4161

   Descriptions of the changes in this PR:
   Remove the rebundunt disk read when scannig the entry log file, which could reduce the pressure of disk a lot.
   
   
   ### Motivation
   We have `org.apache.bookkeeper.bookie.storage.EntryLogScanner` to process the entry data when scanning the entry log file, in which `process` method is used to process the entry data.
   We have a lot of implementations for `EntryLogScanner` for various reason.
   <img width="977" alt="image" src="https://github.com/apache/bookkeeper/assets/52550727/c096d601-1483-4944-ac10-d98828c9ed0f">
   But some of them do not need to access all data in the entry log file.
   For example:
   - scanner implemented in method `extractEntryLogMetadataByScanning` only need to know the `entrySize` field, in such case we do not need to read any data in the entry.
   - scanner implemented in method `org.apache.bookkeeper.bookie.InterleavedStorageRegenerateIndexOp#initiate` only need to know the `entryId` field, in such case we only need to read the first 16 bytes(ledgerId+entryId).
   - many more simillar situations...
   
   So, we **do not need to read the entry data when scanning the entry log in some cases.**
   
   
   ### Changes
   
   Add method `getLengthToRead` in `EntryLogScanner` to indicate how much data do we need to read, and read this mount of data only.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improvement] trim redundant read when scanning entry log. [bookkeeper]

Posted by "eolivelli (via GitHub)" <gi...@apache.org>.
eolivelli commented on code in PR #4161:
URL: https://github.com/apache/bookkeeper/pull/4161#discussion_r1433807650


##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/EntryLogScanner.java:
##########
@@ -27,6 +27,10 @@
  * Scan entries in a entry log file.
  */
 public interface EntryLogScanner {
+    public static final int READ_ALL = Integer.MAX_VALUE;
+    public static final int READ_NOTHING = 0;
+    public static final int READ_LEDGER_ENTRY_ID = 16;

Review Comment:
   what about `READ_LEDGER_ENTRY_ID_LENGTH` ?
   
   but as we basically expect only these 3 values we can use an "enum" and ensure that the code is not called with other combinations (like negative numbers or non sense numbers caused by bugs)



##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/DefaultEntryLogger.java:
##########
@@ -1036,15 +1036,20 @@ public void scanEntryLog(long entryLogId, EntryLogScanner scanner) throws IOExce
                 }
                 // read the entry
                 data.clear();
-                data.capacity(entrySize);
-                int rc = readFromLogChannel(entryLogId, bc, data, pos);
-                if (rc != entrySize) {
-                    LOG.warn("Short read for ledger entry from entryLog {}@{} ({} != {})",
-                            entryLogId, pos, rc, entrySize);
-                    return;
+                int capacity = Math.min(scanner.getLengthToRead(), entrySize);

Review Comment:
   if getLengthToRead returns READ_NOTHING then maybe "data" can be passed as "null"
   if getLengthToRead returns READ_LEDGER_ENTRY_ID then the caller is interested only in the initial bytes.
   
   Please consider changing the interface and have 3 different "process" methods
   
   READ_NOTHING:
   process(ledgerId, entrySize)
   
   READ_LEDGER_ENTRY_ID;
   process(ledgerId, entryId, entrySize)
   in this case the entryId is extracted by this code and not by the code downstream
   
   READ_ALL
   process(ledgerId, offset, data)
   
   This way it is clearer what is happening, and the code is safer, as the implementation of the interface doesn't need to understand the internal format of the "data" buffer when it doesn't need it.
   Using the enum (as suggested below) does not introduce extra costs at runtime



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improvement] trim redundant read when scanning entry log. [bookkeeper]

Posted by "eolivelli (via GitHub)" <gi...@apache.org>.
eolivelli commented on code in PR #4161:
URL: https://github.com/apache/bookkeeper/pull/4161#discussion_r1435149593


##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/DefaultEntryLogger.java:
##########
@@ -1036,16 +1037,33 @@ public void scanEntryLog(long entryLogId, EntryLogScanner scanner) throws IOExce
                 }
                 // read the entry
                 data.clear();
-                data.capacity(entrySize);
-                int rc = readFromLogChannel(entryLogId, bc, data, pos);
-                if (rc != entrySize) {
-                    LOG.warn("Short read for ledger entry from entryLog {}@{} ({} != {})",
-                            entryLogId, pos, rc, entrySize);
-                    return;
+                int rc;
+                // process the entry based on the read length type
+                switch (scanner.getReadLengthType()) {
+                    case READ_NOTHING:
+                        // skip read
+                        scanner.process(ledgerId, offset, entrySize);
+                        break;
+                    case READ_LEDGER_ENTRY_ID:
+                        data.capacity(READ_LEDGER_ENTRY_ID.getLengthToRead());
+                        rc = readFromLogChannel(entryLogId, bc, data, pos);
+                        if (rc != READ_LEDGER_ENTRY_ID.getLengthToRead()) {
+                            LOG.warn("Short read for ledger entry id from entrylog {}", entryLogId);
+                            return;
+                        }
+                        long entryId = data.getLong(Long.BYTES);
+                        scanner.process(ledgerId, offset, entrySize, entryId);
+                        break;
+                    case READ_ALL:
+                        data.capacity(entrySize);
+                        rc = readFromLogChannel(entryLogId, bc, data, pos);
+                        if (rc != entrySize) {
+                            LOG.warn("Short read for ledger entry id from entrylog {}", entryLogId);
+                            return;
+                        }
+                        scanner.process(ledgerId, offset, data);
+                        break;

Review Comment:
   please add the "default" case



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improvement] trim redundant read when scanning entry log. [bookkeeper]

Posted by "thetumbled (via GitHub)" <gi...@apache.org>.
thetumbled commented on code in PR #4161:
URL: https://github.com/apache/bookkeeper/pull/4161#discussion_r1433837408


##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/EntryLogScanner.java:
##########
@@ -27,6 +27,10 @@
  * Scan entries in a entry log file.
  */
 public interface EntryLogScanner {
+    public static final int READ_ALL = Integer.MAX_VALUE;
+    public static final int READ_NOTHING = 0;
+    public static final int READ_LEDGER_ENTRY_ID = 16;

Review Comment:
   good idea, i have introduced the enum.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improvement] trim redundant read when scanning entry log. [bookkeeper]

Posted by "thetumbled (via GitHub)" <gi...@apache.org>.
thetumbled commented on PR #4161:
URL: https://github.com/apache/bookkeeper/pull/4161#issuecomment-1868853950

   > All the changes in this PR are specific to uncommon cases, occurring only when the entrylog file's index is missing.
   
   We have encountered cases that the ledger map in entry log is missed, maybe because the bookie crashed before flushing entry log. As long as the corrupted entry exists, bookie will scan the entry log by `extractEntryLogMetadataByScanning` to generate `EntryLogMetadataMap` when doing gc every `gcWaitTime` milliseconds (default 15min). 
   And other cases is related to index rebuilding, which may be rarely used.
   
   > The readFromLogChannel function utilizes BufferedLogChannel, which is a RandomAccessFile channel. When reading data from the file channel, the data is pre-fetched into the OS PageCache, with a default pre-fetch size of 4KB. Even though we only retrieve the entryId, the actual data read from the disk is a minimum of 4KB.
   
   It is common that the size of one entry reach 4MB (  we set the max batch size of Pulsar Client to be 4MB), so we can decrease 90% of the disk read with this enhancement.
   
   > There is a potential risk associated with this PR:
   > We solely read the entryID without validating the entry data. If the file is corrupted, the scan operation will be unable to detect it and will incorrectly populate the index with the wrong ledger size. This introduces a high level of risk.
   
   The fault detecting logic is not changed, in the old logic we just read `entrySize` amount of data and check if we have read such amount of data. In the new read type, `READ_NOTHING` or `READ_LEDGER_ENTRY_ID`, we still move the read position in the loop and check if we have read expected amount of data. I don't think it is a break change.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improvement] trim redundant read when scanning entry log. [bookkeeper]

Posted by "thetumbled (via GitHub)" <gi...@apache.org>.
thetumbled commented on code in PR #4161:
URL: https://github.com/apache/bookkeeper/pull/4161#discussion_r1434027175


##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/DefaultEntryLogger.java:
##########
@@ -1036,15 +1036,20 @@ public void scanEntryLog(long entryLogId, EntryLogScanner scanner) throws IOExce
                 }
                 // read the entry
                 data.clear();
-                data.capacity(entrySize);
-                int rc = readFromLogChannel(entryLogId, bc, data, pos);
-                if (rc != entrySize) {
-                    LOG.warn("Short read for ledger entry from entryLog {}@{} ({} != {})",
-                            entryLogId, pos, rc, entrySize);
-                    return;
+                int capacity = Math.min(scanner.getLengthToRead(), entrySize);

Review Comment:
   Good idea, i have refactored the code, please help to review it again, thanks. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improvement] skip read rebundunct content when scanning entry log. [bookkeeper]

Posted by "thetumbled (via GitHub)" <gi...@apache.org>.
thetumbled commented on PR #4161:
URL: https://github.com/apache/bookkeeper/pull/4161#issuecomment-1865894749

   Could you help to reviw this PR? thanks!
   @eolivelli @dlg99 @hangc0276 @nicoloboschi @shoothzj @zymap @wenbingshen


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improvement] trim redundant read when scanning entry log. [bookkeeper]

Posted by "thetumbled (via GitHub)" <gi...@apache.org>.
thetumbled commented on code in PR #4161:
URL: https://github.com/apache/bookkeeper/pull/4161#discussion_r1435467648


##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/DefaultEntryLogger.java:
##########
@@ -1036,16 +1037,33 @@ public void scanEntryLog(long entryLogId, EntryLogScanner scanner) throws IOExce
                 }
                 // read the entry
                 data.clear();
-                data.capacity(entrySize);
-                int rc = readFromLogChannel(entryLogId, bc, data, pos);
-                if (rc != entrySize) {
-                    LOG.warn("Short read for ledger entry from entryLog {}@{} ({} != {})",
-                            entryLogId, pos, rc, entrySize);
-                    return;
+                int rc;
+                // process the entry based on the read length type
+                switch (scanner.getReadLengthType()) {
+                    case READ_NOTHING:
+                        // skip read
+                        scanner.process(ledgerId, offset, entrySize);
+                        break;
+                    case READ_LEDGER_ENTRY_ID:
+                        data.capacity(READ_LEDGER_ENTRY_ID.getLengthToRead());
+                        rc = readFromLogChannel(entryLogId, bc, data, pos);
+                        if (rc != READ_LEDGER_ENTRY_ID.getLengthToRead()) {
+                            LOG.warn("Short read for ledger entry id from entrylog {}", entryLogId);
+                            return;
+                        }
+                        long entryId = data.getLong(Long.BYTES);
+                        scanner.process(ledgerId, offset, entrySize, entryId);
+                        break;
+                    case READ_ALL:
+                        data.capacity(entrySize);
+                        rc = readFromLogChannel(entryLogId, bc, data, pos);
+                        if (rc != entrySize) {
+                            LOG.warn("Short read for ledger entry id from entrylog {}", entryLogId);
+                            return;
+                        }
+                        scanner.process(ledgerId, offset, data);
+                        break;

Review Comment:
   Done, PTAL.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org