You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2020/03/13 07:08:01 UTC

[GitHub] [bookkeeper] hangc0276 opened a new pull request #2287: Add Journal PageCache flush interval control when journalSyncData is disabled

hangc0276 opened a new pull request #2287: Add Journal PageCache flush interval control when journalSyncData is disabled
URL: https://github.com/apache/bookkeeper/pull/2287
 
 
   Descriptions of the changes in this PR:
   ### Motivation
   There was a weird behavior observed: when journalSyncData is disabled, it will trigger callbacks when the data is flushed to filesystem(PageCache) but still enqueue a fsync request to force write queue to fsync PageCache data to disk frequently. Though the fsync thread dequeue fsync request and execute fsync operation asynchronously, it still cause disk io util increase when fsync frequency high enough. when disk io util increase, especially reach 100%, it will block new journal file creation and lead to journal sync time increases to mutiple seconds.
   
   ### Changes
   when journalSyncData is disabled, we introduce `journalPageCacheFlushIntervalMSec` to control journal PageCache flush frequency to reduce disk io util, default value is 1000ms

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [bookkeeper] hangc0276 commented on issue #2287: Add Journal PageCache flush interval control when journalSyncData is disabled

Posted by GitBox <gi...@apache.org>.
hangc0276 commented on issue #2287: Add Journal PageCache flush interval control when journalSyncData is disabled
URL: https://github.com/apache/bookkeeper/pull/2287#issuecomment-598644378
 
 
   ok, i will add test cases for this change.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [bookkeeper] hangc0276 commented on issue #2287: Add Journal PageCache flush interval control when journalSyncData is disabled

Posted by GitBox <gi...@apache.org>.
hangc0276 commented on issue #2287: Add Journal PageCache flush interval control when journalSyncData is disabled
URL: https://github.com/apache/bookkeeper/pull/2287#issuecomment-600025577
 
 
   I add a test case, please take a look @jiazhai @eolivelli @sijie @merlimat 
   
   when an entry add into Journal and after journalPageCacheFlushInterval passed, no new entry added into journal, the last added entry will not be flushed to disk from PageCache.
   
   In the test case `BookieJournalPageCacheFlushTest#testAckBeforeUnSyncPageCacheFlush`, I just sleep twice journalPageCacheFlushInterval, and add another entry into journal, then check whether foreWriteRequest add into forceWriteRequestQueue.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [bookkeeper] sijie merged pull request #2287: Add Journal PageCache flush interval control when journalSyncData is disabled

Posted by GitBox <gi...@apache.org>.
sijie merged pull request #2287: Add Journal PageCache flush interval control when journalSyncData is disabled
URL: https://github.com/apache/bookkeeper/pull/2287
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services