[jira] [Created] (OAK-2808) Active deletion of 'deleted' Lucene index files from DataStore without relying on full scale Blob GC

["Chetan Mehrotra (JIRA)" <ji...@apache.org>]

Chetan Mehrotra created OAK-2808:
------------------------------------

             Summary: 	Active deletion of 'deleted' Lucene index files from DataStore without relying on full scale Blob GC
                 Key: OAK-2808
                 URL: https://issues.apache.org/jira/browse/OAK-2808
             Project: Jackrabbit Oak
          Issue Type: Improvement
          Components: lucene
            Reporter: Chetan Mehrotra
             Fix For: 1.3.0


With storing of Lucene index files within DataStore our usage pattern
of DataStore has changed between JR2 and Oak.

With JR2 the writes were mostly application based i.e. if application
stores a pdf/image file then that would be stored in DataStore. JR2 by
default would not write stuff to DataStore. Further in deployment
where large number of binary content is present then systems tend to
share the DataStore to avoid duplication of storage. In such cases
running Blob GC is a non trivial task as it involves a manual step and
coordination across multiple deployments. Due to this systems tend to
delay frequency of GC

Now with Oak apart from application the Oak system itself *actively*
uses the DataStore to store the index files for Lucene and there the
churn might be much higher i.e. frequency of creation and deletion of
index file is lot higher. This would accelerate the rate of garbage
generation and thus put lot more pressure on the DataStore storage
requirements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)