[jira] [Updated] (SLING-9078) Keep skipped offsets in memory

["Timothee Maret (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SLING-9078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Timothee Maret updated SLING-9078:
----------------------------------
    Reporter: Timothee Maret  (was: Timothée Maret)

> Keep skipped offsets in memory
> ------------------------------
>
>                 Key: SLING-9078
>                 URL: https://issues.apache.org/jira/browse/SLING-9078
>             Project: Sling
>          Issue Type: Improvement
>          Components: Content Distribution
>    Affects Versions: Content Distribution Journal Core 0.1.8
>            Reporter: Timothee Maret
>            Assignee: Christian Schneider
>            Priority: Major
>             Fix For: Content Distribution Journal Core 0.1.20
>
>
> In SLING-8908 we introduced a mechanism to keep track of skipped packages such that in case of journal availability flip flop, the agents would still progress. The implementation was done by persisting the offset in the repository. In environments where many skipped packages will occur (e.g. large customer with multiple agents) then those commits will start to generate a non negligible load on the repository, consuming resources and creating bottlenecks.
>  
> A first mitigation to this issue has been added in SLING-8965, however for very large customers the mitigation will not be effective. Increasing the grouping of skipped packages (e.g. 1/100 instead of 1/10) reduce the load on the repository but may fail to ensure progress in case of flip/flop.
>  
> Instead of storing the skipped packages in the repository, we should keep them in memory. A new service that does not reference the {{JournalAvailable}} service could be added. The offsets would be consolidated on demand from the stored offsets and those stored in memory. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)