You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by "ArsenalKeeper (via GitHub)" <gi...@apache.org> on 2023/01/31 06:03:05 UTC
[GitHub] [bookkeeper] ArsenalKeeper opened a new issue, #3743: SortedLedgerStorage creates too many entrylog files
ArsenalKeeper opened a new issue, #3743:
URL: https://github.com/apache/bookkeeper/issues/3743
*BUG REPORT*
***Describe the bug***
Recently our production bookie encounters 'too many open files' error, we found bookie create much more entrylog file files than we expected, and the metric bookie.bookie_gc_ACTIVE_ENTRY_LOG_COUNT_gauge like this,
<img width="1903" alt="image" src="https://user-images.githubusercontent.com/32755366/212839438-d4c8ea2d-23fc-4f11-8d5a-4f6cbea7333b.png">
There are many files which is very small and not satisfy 1GB size.
<img width="724" alt="image" src="https://user-images.githubusercontent.com/32755366/212854370-11a0d4f3-120b-413f-896e-07c565191ae5.png">
we use the **SortedLedgerStorage** and entryLogPerLedgerEnabled is **false**.
we use arthas to track the create entrylog logic and found this:
It seems InterleavedLedgerStorage will re-startcheckpoint during rotate entrylog, this may cause memtable flush some bytes during perivous and this checkpoin, and createNewlog is invoked again, this situation will keep loop and cannot be broken.
<img width="818" alt="image" src="https://user-images.githubusercontent.com/32755366/212856037-4697c1ec-67e2-4e71-bd67-492df7dfc209.png">
<img width="748" alt="image" src="https://user-images.githubusercontent.com/32755366/212856359-820172d4-0a3d-4541-bf8d-a2f0d7366116.png">
Please confirm is it a bug or not, if you need more information, please tell me.
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] ArsenalKeeper closed issue #3743: SortedLedgerStorage creates too many entrylog files
Posted by "ArsenalKeeper (via GitHub)" <gi...@apache.org>.
ArsenalKeeper closed issue #3743: SortedLedgerStorage creates too many entrylog files
URL: https://github.com/apache/bookkeeper/issues/3743
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] ArsenalKeeper commented on issue #3743: SortedLedgerStorage creates too many entrylog files
Posted by "ArsenalKeeper (via GitHub)" <gi...@apache.org>.
ArsenalKeeper commented on issue #3743:
URL: https://github.com/apache/bookkeeper/issues/3743#issuecomment-1425123730
<img width="441" alt="image" src="https://user-images.githubusercontent.com/32755366/217992744-d328789c-9dda-48c1-af6e-d970c0691f54.png">
In fact, even we use InterleavedLedgerStorage, it still add InterleavedLedgerStorage to entrylog's listeners twice. Perhaps it a bug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] ArsenalKeeper commented on issue #3743: SortedLedgerStorage creates too many entrylog files
Posted by "ArsenalKeeper (via GitHub)" <gi...@apache.org>.
ArsenalKeeper commented on issue #3743:
URL: https://github.com/apache/bookkeeper/issues/3743#issuecomment-1409808495
Bookkeeper version is 4.15.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] ArsenalKeeper commented on issue #3743: SortedLedgerStorage creates too many entrylog files
Posted by "ArsenalKeeper (via GitHub)" <gi...@apache.org>.
ArsenalKeeper commented on issue #3743:
URL: https://github.com/apache/bookkeeper/issues/3743#issuecomment-1425113957
>
no, we have not configure 'useTransactionalCompaction', does it related to this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] ArsenalKeeper commented on issue #3743: SortedLedgerStorage creates too many entrylog files
Posted by "ArsenalKeeper (via GitHub)" <gi...@apache.org>.
ArsenalKeeper commented on issue #3743:
URL: https://github.com/apache/bookkeeper/issues/3743#issuecomment-1425121471
<img width="500" alt="image" src="https://user-images.githubusercontent.com/32755366/217991792-fee5c87e-73b8-491b-ba59-7b73c3393121.png">
I'm a bit confused, when we use SortedLedgerStorage, did we still should add InterleavedLedgerStorage into entrylog's listeners? In our prod scenario , it cause too many small entrylog files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] ArsenalKeeper commented on issue #3743: SortedLedgerStorage creates too many entrylog files
Posted by "ArsenalKeeper (via GitHub)" <gi...@apache.org>.
ArsenalKeeper commented on issue #3743:
URL: https://github.com/apache/bookkeeper/issues/3743#issuecomment-1409810903
Bookkeeper version is 4.15.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] dlg99 commented on issue #3743: SortedLedgerStorage creates too many entrylog files
Posted by "dlg99 (via GitHub)" <gi...@apache.org>.
dlg99 commented on issue #3743:
URL: https://github.com/apache/bookkeeper/issues/3743#issuecomment-1421495382
do you have transactional compaction enabled?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org