[GitHub] [bookkeeper] Vanlightly commented on a change in pull request #2779: Issue 2728: Entry Log GC may get blocked when using entryLogPerLedgerEnabled option

[GitBox <gi...@apache.org>]

Vanlightly commented on a change in pull request #2779:
URL: https://github.com/apache/bookkeeper/pull/2779#discussion_r699290224



##########
File path: bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java
##########
@@ -583,12 +584,15 @@ protected void compactEntryLog(EntryLogMetadata entryLogMeta) {
      * @throws IOException
      */
     protected Map<Long, EntryLogMetadata> extractMetaFromEntryLogs(Map<Long, EntryLogMetadata> entryLogMetaMap) {
-        // Extract it for every entry log except for the current one.
-        // Entry Log ID's are just a long value that starts at 0 and increments
-        // by 1 when the log fills up and we roll to a new one.
-        long curLogId = entryLogger.getLeastUnflushedLogId();
+        // Entry Log ID's are just a long value that starts at 0 and increments by 1 when the log fills up and we roll
+        // to a new one. We scan entry logs as follows:
+        // - entryLogPerLedgerEnabled is false: Extract it for every entry log except for the current one (un-flushed).
+        // - entryLogPerLedgerEnabled is true: Scan all flushed entry logs up to the highest known id.
+        Supplier<Long> finalEntryLog = () -> conf.isEntryLogPerLedgerEnabled() ? entryLogger.getLastLogId() :

Review comment:
       When the journal is replayed it writes the entries in the same way as an add entry request received from a client.
   
   I think at some point it might be a good idea to hide the GC and compaction code entirely inside the ledger storage implementations. Different storage engines work in different ways and we're trying to make each conform to a standard set of interfaces/behaviours. ELPL would then be one more ledger storage engine that is free to do GC/compaction in any way it likes without worrying about impacting other storage engines or conforming to a common interface.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org