[jira] [Created] (SLING-8965) Avoid committing for each skipped package

["Timothee Maret (Jira)" <ji...@apache.org>]

Timothee Maret created SLING-8965:
-------------------------------------

             Summary: Avoid committing for each skipped package 
                 Key: SLING-8965
                 URL: https://issues.apache.org/jira/browse/SLING-8965
             Project: Sling
          Issue Type: Bug
          Components: Content Distribution
            Reporter: Timothee Maret


With the SLING-8908 change, we ensure that all package messages (even skipped ones) are marked as processed. By design, we mark processed package by committing to the local repository. Committing to the repository is not cheap and we should minimise it. For setups that generate a lot of skipped packages such as A) multiple farms support with multiple consumer agents or B) multi-region deployments with varying latencies, the amount of commits will increase even more.

To reduce the commits, we should either redo SLING-8908 such that skipped offsets are kept in memory or simply only commit skipped packages as aggregates (e.g. 1 commits for 10 skipped packages).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)