You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by "zifengmo (via GitHub)" <gi...@apache.org> on 2024/06/07 01:52:09 UTC
[PR] Update EntryLocationIndex.java [bookkeeper]
zifengmo opened a new pull request, #4423:
URL: https://github.com/apache/bookkeeper/pull/4423
Fix the issue that the data cannot be completely cleaned up because the removeOffsetFromDeletedLedgers method under EntryLocationIndex does not close after using RocksDBBatch to flush in batches.
Descriptions of the changes in this PR:
<!-- Either this PR fixes an issue, -->
Fix #4145
### Motivation
Fix the issue that the data under locations cannot be completely cleaned up when the data volume is large.
### Changes
Add `batch.close();` after `batch.flush();`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the issue that the data cannot be completely cleaned up under locations folder [bookkeeper]
Posted by "zifengmo (via GitHub)" <gi...@apache.org>.
zifengmo commented on PR #4423:
URL: https://github.com/apache/bookkeeper/pull/4423#issuecomment-2154092438
> Is that really fix? I see the batch in try clause, which means it will automatically close.
For more accuracy, I will observe for a few more days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the issue that the data cannot be completely cleaned up under locations folder [bookkeeper]
Posted by "danpi (via GitHub)" <gi...@apache.org>.
danpi commented on PR #4423:
URL: https://github.com/apache/bookkeeper/pull/4423#issuecomment-2670968825
> @danpi Could you please provider more information? I don't think the change code is a problem.
Sure, the issue #4554 provides a more detailed problem report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the issue that the data cannot be completely cleaned up under locations folder [bookkeeper]
Posted by "shoothzj (via GitHub)" <gi...@apache.org>.
shoothzj commented on PR #4423:
URL: https://github.com/apache/bookkeeper/pull/4423#issuecomment-2153720853
Is that really fix? I see the batch in try clause, which means it will automatically close.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the issue that the data cannot be completely cleaned up under locations folder [bookkeeper]
Posted by "zifengmo (via GitHub)" <gi...@apache.org>.
zifengmo closed pull request #4423: Fix the issue that the data cannot be completely cleaned up under locations folder
URL: https://github.com/apache/bookkeeper/pull/4423
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the issue that the data cannot be completely cleaned up under locations folder [bookkeeper]
Posted by "zifengmo (via GitHub)" <gi...@apache.org>.
zifengmo commented on PR #4423:
URL: https://github.com/apache/bookkeeper/pull/4423#issuecomment-2153722553
> Is that really fix? I see the batch in try clause, which means it will automatically close.
Yes, the number of files in the folder and disk usage have become normal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the issue that the data cannot be completely cleaned up under locations folder [bookkeeper]
Posted by "danpi (via GitHub)" <gi...@apache.org>.
danpi commented on PR #4423:
URL: https://github.com/apache/bookkeeper/pull/4423#issuecomment-2664937927
@zifengmo @hezhangjian Is there any update on this issue? After upgrading from Pulsar 2.8 to LTS 3.0, I encountered the same problem. In high-traffic scenarios, even though level_compaction_dynamic_level_bytes is enabled, the score of L6 remains 0, and old SST files cannot be deleted for a long time. This indirectly leads to excessively high physical memory usage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the issue that the data cannot be completely cleaned up under locations folder [bookkeeper]
Posted by "shoothzj (via GitHub)" <gi...@apache.org>.
shoothzj commented on PR #4423:
URL: https://github.com/apache/bookkeeper/pull/4423#issuecomment-2154135521
Sorry, I want to ask the batch is already closed before, what's the changes?
`try (Batch batch = locationsDb.newBatch()) {`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the issue that the data cannot be completely cleaned up under locations folder [bookkeeper]
Posted by "hezhangjian (via GitHub)" <gi...@apache.org>.
hezhangjian commented on PR #4423:
URL: https://github.com/apache/bookkeeper/pull/4423#issuecomment-2664970608
@danpi Could you please provider more information? I don't think the change code is a problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org