[jira] [Created] (JAMES-2921) Improve blob store usage

["Benoit Tellier (Jira)" <se...@james.apache.org>]

Benoit Tellier created JAMES-2921:
-------------------------------------

             Summary: Improve blob store usage
                 Key: JAMES-2921
                 URL: https://issues.apache.org/jira/browse/JAMES-2921
             Project: James Server
          Issue Type: Wish
          Components: Blob
            Reporter: Benoit Tellier
         Attachments: adr-0003-blobstore-storage-policies.md, adr-0004-objectstorage-blobid-list.md

Please find attached the two technical decisions being proposed.

These needs performance feedback before being adopted.

h2. Proposal: Add storage policies for BlobStore

Introduce StoragePolicies at the level of the BlobStore API.

The proposed policies includes:

 - SizeBasedStoragePolicy: The blob underlying storage medium will be chosen depending of its size. 
 - LowCostStoragePolicy: The blob is expected to be saved in low cost storage. Access is expected to be unfrequent.
 - PerformantStoragePolicy: The blob is expected to be saved in performant storage. Access is expected to be unfrequent.

The UnionBlobStore will be reworked to choose between Cassandra and ObjectStorage implementations depending on the policies.

DeletedMessageVault, BlobExport & MailRepository will rely on LowCostStoragePolicy. Other BlobStore users will rely on SizeBasedStoragePolicy.

h2. Proposal: persist BlobIds for avoiding persisting several time the same blobs within ObjectStorage

Rely on a StoredBlobIdsList API to know which blob is persisted or not in object storage. Provide a Cassandra implementation of it. Located in blob-api for convenience, this in not a top level API. It is intended to be used by some blobStore implementations (here only ObjectStorage).

 - When saving a blob with precomputed blobId, we can check the existance of the blob in storage, avoiding possibly the expensive "save".
 - When saving a blob too big to precompute its blobId, once the blob had been stream using a temporary random blobId, copy operation can be avoided and the temporary blob could be directly removed.

Cassandra is faster doing "write every time" rather than "read before write" so we should not use the storedblob projection for it



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org