[GitHub] [rocketmq] ShadowySpirits added a comment to the discussion: Different implements about suporting S3 backend for tiered storage

[GitBox <gi...@apache.org>]

GitHub user ShadowySpirits added a comment to the discussion: Different implements about suporting S3 backend for tiered storage

We should not add the pre-fetching mechanism in the backend service provider. Similar to the file system in OS, the underlying interface reads files according to the specified position and length, and the OS maintains the page cache, which is a read-ahead cache in tiered storage (see [TieredMessageFetcher](https://github.com/apache/rocketmq/blob/develop/tieredstore/src/main/java/org/apache/rocketmq/tieredstore/TieredMessageFetcher.java) for more details).

If you want to batch upload, a shared upload buffer is okay. But when the messages are uploaded to s3, the data in the buffer should be cleaned immediately because we do not expect to access hot data from tiered storage.

GitHub link: https://github.com/apache/rocketmq/discussions/6176#discussioncomment-5109325

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org