You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Gautier DI FOLCO (Jira)" <se...@james.apache.org> on 2019/10/11 09:09:00 UTC
[jira] [Updated] (JAMES-2921) Improve blob store usage
[ https://issues.apache.org/jira/browse/JAMES-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gautier DI FOLCO updated JAMES-2921:
------------------------------------
Attachment: adr-0004-objectstorage-blobid-list.md
adr-0003-blobstore-storage-policies.md
> Improve blob store usage
> ------------------------
>
> Key: JAMES-2921
> URL: https://issues.apache.org/jira/browse/JAMES-2921
> Project: James Server
> Issue Type: Wish
> Components: Blob
> Reporter: Benoit Tellier
> Priority: Major
> Attachments: adr-0003-blobstore-storage-policies.md, adr-0003-blobstore-storage-policies.md, adr-0004-objectstorage-blobid-list.md, adr-0004-objectstorage-blobid-list.md
>
>
> Please find attached the two technical decisions being proposed.
> These needs performance feedback before being adopted.
> h2. Proposal: Add storage policies for BlobStore
> Introduce StoragePolicies at the level of the BlobStore API.
> The proposed policies includes:
> - SizeBasedStoragePolicy: The blob underlying storage medium will be chosen depending of its size.
> - LowCostStoragePolicy: The blob is expected to be saved in low cost storage. Access is expected to be unfrequent.
> - PerformantStoragePolicy: The blob is expected to be saved in performant storage. Access is expected to be unfrequent.
> The UnionBlobStore will be reworked to choose between Cassandra and ObjectStorage implementations depending on the policies.
> DeletedMessageVault, BlobExport & MailRepository will rely on LowCostStoragePolicy. Other BlobStore users will rely on SizeBasedStoragePolicy.
> h2. Proposal: persist BlobIds for avoiding persisting several time the same blobs within ObjectStorage
> Rely on a StoredBlobIdsList API to know which blob is persisted or not in object storage. Provide a Cassandra implementation of it. Located in blob-api for convenience, this in not a top level API. It is intended to be used by some blobStore implementations (here only ObjectStorage).
> - When saving a blob with precomputed blobId, we can check the existance of the blob in storage, avoiding possibly the expensive "save".
> - When saving a blob too big to precompute its blobId, once the blob had been stream using a temporary random blobId, copy operation can be avoided and the temporary blob could be directly removed.
> Cassandra is faster doing "write every time" rather than "read before write" so we should not use the storedblob projection for it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org