You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Ivan Kelly (JIRA)" <ji...@apache.org> on 2013/09/06 20:17:51 UTC
[jira] [Comment Edited] (BOOKKEEPER-664) Compaction increases
latency on journal writes
[ https://issues.apache.org/jira/browse/BOOKKEEPER-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760461#comment-13760461 ]
Ivan Kelly edited comment on BOOKKEEPER-664 at 9/6/13 6:15 PM:
---------------------------------------------------------------
{quote}
1) it would be better to add a test case to cover the case you described in the comment to ensure future changes would not break it.
{quote}
test attached.
{quote}
2) rotate listener defers the flushing, which would break some test cases in compaction test if I remembered correctly. so you might need to change those test cases.
{quote}
Moving somethingWritten into EntryLogger fixes this problem.
{quote}
3) add compactionMaxOutstandingRequests and compactionRate to document & config file
{quote}
Ive added to the config. I dont want to add to the doc yet. We havent played with the parameters ourselves yet. I'd prefer if users didnt either without asking us.
{quote}
4) use {} rather than "+" in log statements.
{quote}
fixed
{quote}5) you might need to notify flushLock when shutting down similar as what I do in the patch. since entry logger might not rotate during shutting down.{quote}
Fixed by checking 'running' in the wait loop
was (Author: ikelly):
{quote}
1) it would be better to add a test case to cover the case you described in the comment to ensure future changes would not break it.
{quote}
test attached.
{quote}
2) rotate listener defers the flushing, which would break some test cases in compaction test if I remembered correctly. so you might need to change those test cases.
{quote}
Moving somethingWritten into EntryLogger fixes this problem.
{quote}
3) add compactionMaxOutstandingRequests and compactionRate to document & config file
{quote}
Ive added to the config. I dont want to add to the doc yet. We havent played with the parameters ourselves yet. I'd prefer if users didnt either without asking us.
{fixed}
4) use {} rather than "+" in log statements.
{quote}
fixed
{quote}5) you might need to notify flushLock when shutting down similar as what I do in the patch. since entry logger might not rotate during shutting down.{quote}
Fixed by checking 'running' in the wait loop
> Compaction increases latency on journal writes
> ----------------------------------------------
>
> Key: BOOKKEEPER-664
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-664
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Affects Versions: 4.2.1
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.2, 4.3.0
>
> Attachments: 0001-BOOKKEEPER-664-branch-4.2-Compaction-increases-laten.patch, 0001-BOOKKEEPER-664-branch-4.2-Compaction-increases-laten.patch, 0001-BOOKKEEPER-664-branch-4.2-Compaction-increases-laten.patch, 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ_branch4.2.patch, 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ-branch-4.2.patch, 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ.patch, 0001-BOOKKEEPER-664-Compaction-increases-latency-on-journ_trunk.patch, 0002-BOOKKEEPER-664-trunk-Compaction-increases-latency-on.patch, bench.png, BOOKKEEPER-664.patch
>
>
> Compaction writes to the journal to avoid data loss (see BOOKKEEPER-530). BOOKKEEPER-530 correctly identified that this may affect latency on the journal but we have observed this since in production. It is possible to avoid the journal completely, as twitter do in their github branch. Basically, we need to write to the entrylogger first, flush the entry log and then add to the index.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira