[jira] Updated: (JCR-926) Global data store for binaries

["Jukka Zitting (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/JCR-926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jukka Zitting updated JCR-926:
------------------------------

    Attachment: DataStore2.patch

Attached a prototype patch that integrates the data store concept in Jackrabbit. The integration is very ugly at this stage and the attached code is definitely not meant for inclusion as-is.

For very rough performance testing I created a simple test application that creates a versionable folder with 100 files in it, each containing about 270kB of application/octet-stream data. Once populated, the entire folder was checked into the version store. The numbers, averaged over a couple of test runs, are: 

Current svn trunk:

    100 added files: 5625 milliseconds
    checkin everything: 11094 milliseconds

With this patch:

    100 added files: 2750 milliseconds
    checkin everything: 1906 milliseconds


> Global data store for binaries
> ------------------------------
>
>                 Key: JCR-926
>                 URL: https://issues.apache.org/jira/browse/JCR-926
>             Project: Jackrabbit
>          Issue Type: New Feature
>          Components: core
>            Reporter: Jukka Zitting
>         Attachments: DataStore.patch, DataStore2.patch
>
>
> There are three main problems with the way Jackrabbit currently handles large binary values:
> 1) Persisting a large binary value blocks access to the persistence layer for extended amounts of time (see JCR-314)
> 2) At least two copies of binary streams are made when saving them through the JCR API: one in the transient space, and one when persisting the value
> 3) Versioining and copy operations on nodes or subtrees that contain large binary values can quickly end up consuming excessive amounts of storage space.
> To solve these issues (and to get other nice benefits), I propose that we implement a global "data store" concept in the repository. A data store is an append-only set of binary values that uses short identifiers to identify and access the stored binary values. The data store would trivially fit the requirements of transient space and transaction handling due to the append-only nature. An explicit mark-and-sweep garbage collection process could be added to avoid concerns about storing garbage values.
> See the recent NGP value record discussion, especially [1], for more background on this idea.
> [1] http://mail-archives.apache.org/mod_mbox/jackrabbit-dev/200705.mbox/%3c510143ac0705120919k37d48dc1jc7474b23c9f02cbd@mail.gmail.com%3e

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.