You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2020/03/13 07:08:01 UTC
[GitHub] [bookkeeper] hangc0276 opened a new pull request #2287: Add Journal
PageCache flush interval control when journalSyncData is disabled
hangc0276 opened a new pull request #2287: Add Journal PageCache flush interval control when journalSyncData is disabled
URL: https://github.com/apache/bookkeeper/pull/2287
Descriptions of the changes in this PR:
### Motivation
There was a weird behavior observed: when journalSyncData is disabled, it will trigger callbacks when the data is flushed to filesystem(PageCache) but still enqueue a fsync request to force write queue to fsync PageCache data to disk frequently. Though the fsync thread dequeue fsync request and execute fsync operation asynchronously, it still cause disk io util increase when fsync frequency high enough. when disk io util increase, especially reach 100%, it will block new journal file creation and lead to journal sync time increases to mutiple seconds.
### Changes
when journalSyncData is disabled, we introduce `journalPageCacheFlushIntervalMSec` to control journal PageCache flush frequency to reduce disk io util, default value is 1000ms
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [bookkeeper] hangc0276 commented on issue #2287: Add Journal
PageCache flush interval control when journalSyncData is disabled
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on issue #2287: Add Journal PageCache flush interval control when journalSyncData is disabled
URL: https://github.com/apache/bookkeeper/pull/2287#issuecomment-598644378
ok, i will add test cases for this change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [bookkeeper] hangc0276 commented on issue #2287: Add Journal
PageCache flush interval control when journalSyncData is disabled
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on issue #2287: Add Journal PageCache flush interval control when journalSyncData is disabled
URL: https://github.com/apache/bookkeeper/pull/2287#issuecomment-600025577
I add a test case, please take a look @jiazhai @eolivelli @sijie @merlimat
when an entry add into Journal and after journalPageCacheFlushInterval passed, no new entry added into journal, the last added entry will not be flushed to disk from PageCache.
In the test case `BookieJournalPageCacheFlushTest#testAckBeforeUnSyncPageCacheFlush`, I just sleep twice journalPageCacheFlushInterval, and add another entry into journal, then check whether foreWriteRequest add into forceWriteRequestQueue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [bookkeeper] sijie merged pull request #2287: Add Journal PageCache
flush interval control when journalSyncData is disabled
Posted by GitBox <gi...@apache.org>.
sijie merged pull request #2287: Add Journal PageCache flush interval control when journalSyncData is disabled
URL: https://github.com/apache/bookkeeper/pull/2287
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services