[jira] [Comment Edited] (OAK-5158) Online compaction progress monitor

["Alex Parvulescu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15693618#comment-15693618 ] 

Alex Parvulescu edited comment on OAK-5158 at 11/24/16 4:04 PM:
----------------------------------------------------------------

so much feedback, so little time :)

bq. Let's be careful with the sleepPerCycle functionality. I wouldn't make this into an official configuration value until we have some preliminary test results.
right, that's why it's not in as OSGi yet, just system properties. the rest (jmx) is needed to verify on longevity tests anyway.

bq. I would actually prefer to leave this functionality for another issue (although I can see how tempting and easy it is to have it implemented with this one).
right. I see your point, but I prefer to have this included early. the overhead of adding the 2 at the same time is minimal compacred to only adding the progress tracker. also it helps with longevity testing to have them both from the beginning.

as requiested, I've separated the log cycle from the sleep cycle, but this means I had to tweak the system properties:
* {{oak.segment.compaction.gcLogCycle}} as before
* {{oak.segment.compaction.gcSleepCycle}} number of nodes for sleep cycle
* {{oak.segment.compaction.gcSleepMsPerCycle}} ms sleep per cycle
* and a surprise bonus {{oak.segment.compaction.gcSleepYield}} to control the timeout behavior. this is a boolean property and if enabled can switch to using {{Thread.yield}} instead of {{Thread.sleep}}. this was proposed by [~frm] as an alternative to the sleep-based approach, and I thought I'd included it as a feature flag disabled by default so it's easier to test together with the rest of the stuff.

if I left something out, let me know. looking forward to the continuous feedback!






was (Author: alex.parvulescu):
so much feedback, so little time :)

bq. Let's be careful with the sleepPerCycle functionality. I wouldn't make this into an official configuration value until we have some preliminary test results.
right, that's why it's not in as OSGi yet, just system properties. the test (jmx) is needed to test on longevity tests anyway.

bq. I would actually prefer to leave this functionality for another issue (although I can see how tempting and easy it is to have it implemented with this one).
right. I see your point, but I prefer to have this included early. the overhead of adding the 2 at the same time is minimal compacred to only adding the progress tracker. also it helps with longevity testing to have them both from the beginning.

as requiested, I've separated the log cycle from the sleep cycle, but this means I had to tweak the system properties:
* {{oak.segment.compaction.gcLogCycle}} as before
* {{oak.segment.compaction.gcSleepCycle}} number of nodes for sleep cycle
* {{oak.segment.compaction.gcSleepMsPerCycle}} ms sleep per cycle
* and a surprise bonus {{oak.segment.compaction.gcSleepYield}} to control the timeout behavior. this is a boolean property and if enabled can switch to using {{Thread.yield}} instead of {{Thread.sleep}}. this was proposed by [~frm] as an alternative to the sleep-based approach, and I thought I'd included it as a feature flag disabled by default so it's easier to test together with the rest of the stuff.

if I left something out, let me know. looking forward to the continuous feedback!





> Online compaction progress monitor
> ----------------------------------
>
>                 Key: OAK-5158
>                 URL: https://issues.apache.org/jira/browse/OAK-5158
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: segment-tar
>            Reporter: Alex Parvulescu
>            Assignee: Alex Parvulescu
>             Fix For: 1.6, 1.5.15
>
>         Attachments: OAK-5158-v2.patch, OAK-5158.patch
>
>
> Having an online compaction progress monitor open the door for providing a (fuzzy) percentage of completion. Similar to the offline progress tracker, it would be responsible for logging progress at a configurable interval but also providing {{current number of compacted nodes}} and {{estimated number of total nodes}}.
> I'd also like to add the possibility to sleep a few ms each cycle, this can introduce throttling options for online compaction in the case the impact is too great on the system.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)