You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by "horizonzy (via GitHub)" <gi...@apache.org> on 2023/04/06 08:25:37 UTC
[GitHub] [bookkeeper] horizonzy opened a new pull request, #3906: Fix ReclaimedSpaceViaDeletes stats incorrect problem.
horizonzy opened a new pull request, #3906:
URL: https://github.com/apache/bookkeeper/pull/3906
Descriptions of the changes in this PR:
https://github.com/apache/bookkeeper/blob/a6387d191f28a59af72f6613b633d6898980a0b0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java#L487-L517
At line 501, increases getReclaimedSpaceViaDeletes stats.
https://github.com/apache/bookkeeper/blob/a6387d191f28a59af72f6613b633d6898980a0b0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java#L734-L767
But at line 755, it didn't increases getReclaimedSpaceViaDeletes stats.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] horizonzy commented on a diff in pull request #3906: Fix ReclaimedSpaceViaDeletes stats incorrect problem.
Posted by "horizonzy (via GitHub)" <gi...@apache.org>.
horizonzy commented on code in PR #3906:
URL: https://github.com/apache/bookkeeper/pull/3906#discussion_r1161377575
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java:
##########
@@ -751,11 +751,12 @@ protected void extractMetaFromEntryLogs() throws EntryLogMetadataMapException {
EntryLogMetadata entryLogMeta = entryLogger.getEntryLogMetadata(entryLogId, throttler);
removeIfLedgerNotExists(entryLogMeta);
if (entryLogMeta.isEmpty()) {
- LOG.info("Entry log file {} is empty, delete it from disk.", Long.toHexString(entryLogId));
- entryLogger.removeEntryLog(entryLogId);
- // remove it from entrylogmetadata-map if it is present in
- // the map
- entryLogMetaMap.remove(entryLogId);
+ // This means the entry log is not associated with any active
+ // ledgers anymore.
+ // We can remove this entry log file now.
+ LOG.info("Deleting entryLogId {} as it has no active ledgers!", entryLogId);
+ removeEntryLog(entryLogId);
+ gcStats.getReclaimedSpaceViaDeletes().addCount(entryLogMeta.getTotalSize());
Review Comment:
https://github.com/apache/bookkeeper/blob/ceba60565cf7cb438e9be4ab7416a2808b9168a1/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogMetadata.java#L94-L102,
it only reduces the remainingSize, the totalSize won't be reduce to 0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on a diff in pull request #3906: Fix ReclaimedSpaceViaDeletes stats incorrect problem.
Posted by "hangc0276 (via GitHub)" <gi...@apache.org>.
hangc0276 commented on code in PR #3906:
URL: https://github.com/apache/bookkeeper/pull/3906#discussion_r1161383755
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java:
##########
@@ -751,11 +751,12 @@ protected void extractMetaFromEntryLogs() throws EntryLogMetadataMapException {
EntryLogMetadata entryLogMeta = entryLogger.getEntryLogMetadata(entryLogId, throttler);
removeIfLedgerNotExists(entryLogMeta);
if (entryLogMeta.isEmpty()) {
- LOG.info("Entry log file {} is empty, delete it from disk.", Long.toHexString(entryLogId));
- entryLogger.removeEntryLog(entryLogId);
- // remove it from entrylogmetadata-map if it is present in
- // the map
- entryLogMetaMap.remove(entryLogId);
+ // This means the entry log is not associated with any active
+ // ledgers anymore.
+ // We can remove this entry log file now.
+ LOG.info("Deleting entryLogId {} as it has no active ledgers!", entryLogId);
+ removeEntryLog(entryLogId);
+ gcStats.getReclaimedSpaceViaDeletes().addCount(entryLogMeta.getTotalSize());
Review Comment:
Oh sorry, it's my mistake.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on a diff in pull request #3906: Fix ReclaimedSpaceViaDeletes stats incorrect problem.
Posted by "hangc0276 (via GitHub)" <gi...@apache.org>.
hangc0276 commented on code in PR #3906:
URL: https://github.com/apache/bookkeeper/pull/3906#discussion_r1161374156
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java:
##########
@@ -751,11 +751,12 @@ protected void extractMetaFromEntryLogs() throws EntryLogMetadataMapException {
EntryLogMetadata entryLogMeta = entryLogger.getEntryLogMetadata(entryLogId, throttler);
removeIfLedgerNotExists(entryLogMeta);
if (entryLogMeta.isEmpty()) {
- LOG.info("Entry log file {} is empty, delete it from disk.", Long.toHexString(entryLogId));
- entryLogger.removeEntryLog(entryLogId);
- // remove it from entrylogmetadata-map if it is present in
- // the map
- entryLogMetaMap.remove(entryLogId);
+ // This means the entry log is not associated with any active
+ // ledgers anymore.
+ // We can remove this entry log file now.
+ LOG.info("Deleting entryLogId {} as it has no active ledgers!", entryLogId);
+ removeEntryLog(entryLogId);
+ gcStats.getReclaimedSpaceViaDeletes().addCount(entryLogMeta.getTotalSize());
Review Comment:
We don't need to update the stats because the entryLogMeta is empty and the `entryLogMeta.getTotalSize()` will be `0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] horizonzy commented on pull request #3906: Fix ReclaimedSpaceViaDeletes stats incorrect problem.
Posted by "horizonzy (via GitHub)" <gi...@apache.org>.
horizonzy commented on PR #3906:
URL: https://github.com/apache/bookkeeper/pull/3906#issuecomment-1499803228
rerun failure checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 merged pull request #3906: Fix ReclaimedSpaceViaDeletes stats incorrect problem.
Posted by "hangc0276 (via GitHub)" <gi...@apache.org>.
hangc0276 merged PR #3906:
URL: https://github.com/apache/bookkeeper/pull/3906
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] horizonzy commented on pull request #3906: Fix ReclaimedSpaceViaDeletes stats incorrect problem.
Posted by "horizonzy (via GitHub)" <gi...@apache.org>.
horizonzy commented on PR #3906:
URL: https://github.com/apache/bookkeeper/pull/3906#issuecomment-1499306745
rerun failure checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org