[Jackrabbit Wiki] Update of "Composite Blob Store Storage Filters" by MattRyan

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Jackrabbit Wiki" for change notification.

The "Composite Blob Store Storage Filters" page has been changed by MattRyan:
https://wiki.apache.org/jackrabbit/Composite%20Blob%20Store%20Storage%20Filters

New page:
= Composite Blob Store Storage Filters =

NOTE:  The current status of this component is a '''rejected feature''' for Oak 1.8, and a '''possible future feature''' for later Oak versions.

== Overview ==
Storage filters add capability to [[Composite Blob Store]] by allowing configuration to define criteria that restrict which blobs can be stored in a composite blob store delegate.  Storage filters would then be an optional element of delegate configuration.  Delegates would not be required to have any storage filters at all.

This is a possible future feature that can add a lot of value to the composite blob store concept.  A major hurdle to this implementation is that JCR node information is not provided to data store implementations; the normal case is that a binary is created before the corresponding node.  This problem has to be resolved in order to implement this feature.  More discussion on this below.

== Technical Details ==
=== Storage Filter Types ===
Storage filters operate on any combination of the following:
 * JCR path
 * JCR node type
 * Existence of JCR property
 * JCR property value

=== Impacts on Composite Blob Store ===
The existence of a storage filter on a delegate blob store may have an impact on the delegate traversal strategy implementation.  For example, an implementation may consider that a blob store with filters should take precedence over a blob store without filters in terms of choosing a write destination.

==== Impact on Intelligent Delegate Traversal Strategy ====
If storage filters were added to composite blob store, it is expected that the following changes would be added to the Intelligent Delegate Traversal Strategy, which is the default traversal strategy.

===== Delegate Search Order =====
Speaking generally, delegates with storage filters would take highest precedence, and would be tried first for reads and writes.

===== Writes =====
Delegates that have storage filters always have write precedence over delegates that do not have storage filters - meaning writes always happen to delegates with filters if the filters match what is being written.

The algorithm for writing a blob to the composite blob store would be changed:
 * When determining where to write the blob, start with delegates that have storage filters.  Prefer writing to a delegate with a storage filter that matches the blob being written.
 * When searching for other versions of the same blob, '''also''' search using delegates that have filters that don't match the blob being written (after searching delegates without filters).
   * Configuration change can cause this situation - a blob may originally have been written to a blob store, and afterward configuration changes such that the blob would not have been written to that blob store originally. See "Using Non-matching Delegates" below for more.
   * If this case occurs, the code should also asynchronously remove the blob from the incorrect location once it has been written to the correct location.

===== Reads =====
Delegates that have storage filters always have read precedence over delegates that do not have storage filters - meaning we always attempt to read from them first, because matching filters is faster than checking a delegate to see if a blob exists.

The algorithm for reading a blob from the composite blob store would be changed:
 * When searching for a blob, start with delegates that have storage filters that match the blob.
 * If no match, search delegates without storage filters.
 * If no match, search delegates that have storage filters that do not match the blob.
   * Configuration change can cause this situation - a blob may originally have been written to a blob store, and afterward configuration changes such that the blob would not have been written to that blob store originally. See "Using Non-matching Delegates" below for more.
   * If this case occurs, the code should also asynchronously move the blob from the incorrect location to the correct location.

===== Using Non-matching Delegates =====

This step is necessary to handle situations where blobs are temporarily located in the "wrong" blob store - in other words, when a blob is located in a delegate where it would not be written according to configuration. The most obvious case where this could occur is in the case of configuration change. A delegate D may be configured with certain storage filters, causing Blob B to be written there. Then the configuration is changed such that if B were being written now it would not have been written to D. This final step allows B to be found in D even though it doesn't match the storage filters.

When this situation is encountered, the composite blob store should also initiate an asynchronous background job to move the blob from it's current location to the proper one - the location where it would be found if it were being created now - on read requests, or for write requests should write to the correct location and remove the blob asynchronously in the background from the current location after the write is done.