You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Benoit Tellier (Jira)" <se...@james.apache.org> on 2024/06/03 15:23:00 UTC
[jira] [Closed] (JAMES-3555) Allow 'real' streaming upon BlobStore reads
[ https://issues.apache.org/jira/browse/JAMES-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Tellier closed JAMES-3555.
---------------------------------
Resolution: Won't Fix
> Allow 'real' streaming upon BlobStore reads
> -------------------------------------------
>
> Key: JAMES-3555
> URL: https://issues.apache.org/jira/browse/JAMES-3555
> Project: James Server
> Issue Type: Improvement
> Components: Blob
> Affects Versions: 3.6.0
> Reporter: Benoit Tellier
> Priority: Major
>
> I investigated a build instability... https://github.com/apache/james-project/pull/370
> One time out of 25 the uploaded content was altered (Different SHA 256 in
> the data store). Several Apache CI builds had been failing because of this.
> Investigating this, I found that a piped input stream strategy was not
> hitting this pitfall (RectorUtils::toInputStream). I thus incriminate
> some weird data races on the Reactor side...
> Other areas of the code are likely affected as both the Cassandra and the
> S3 blobStores relies on those for streaming (only relied upon for
> attachment upload).
> Changing the BlobStore API should likely be considered:
> {code:java}
> public interface BlobStore {
> Flux<ByteBuffer> read(BucketName bucketName, BlobId blobId);
> }
> {code}
> Why:
> - Supported by both implementation (Cassandra S3)
> - Avoids intermediate transformations requiring blocking
> operations/dedicated thread resources
> - We convert a Flux[ByteBuffer] => InputStream (to conform to BlobStore API) => Flux[ByteBuffer] (reactor-netty layer)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org