[jira] [Created] (OAK-1032) SegmentMK: One SegmentWriter per SegmentNodeStore

["Jukka Zitting (JIRA)" <ji...@apache.org>]

Jukka Zitting created OAK-1032:
----------------------------------

             Summary: SegmentMK: One SegmentWriter per SegmentNodeStore
                 Key: OAK-1032
                 URL: https://issues.apache.org/jira/browse/OAK-1032
             Project: Jackrabbit Oak
          Issue Type: Sub-task
          Components: core
            Reporter: Jukka Zitting
            Priority: Minor


Currently each SegmentNodeStoreBranch (and thus Root) instance has it's own SegmentWriter that it uses to construct new segments. This allows maximum write concurrency as there's no need for any synchronization at the SegmentWriter level.

The downside of this solution is that the resulting segments are often very small (< 100 bytes), which increases write overhead (in TarMK each segment needs at least 1kB of disk space) and reduces locality of reference (related content gets scattered across many segments). And concurrent write throughput is in any case constrained when more than one writer is updating the same journal.

Thus I think it would be good to instead have just one SegmentWriter per SegmentNodeStore and have all branches use that shared writer. This requires synchronization when writing individual records, and can thus lead to extra contention, but since records are fine-grained and all record writes happen in memory, the amount of contention should be fairly small for normal workloads (and for truly heavy concurrent writing, we'd in any case be using multiple processes across many cluster nodes, in which case this limit wouldn't apply).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira