[I] Owned Buffer Design for `oio::Read` and `oio::Write` [opendal]

["Xuanwo (via GitHub)" <gi...@apache.org>]

Xuanwo opened a new issue, #4401:
URL: https://github.com/apache/opendal/issues/4401

   I've made numerous updates to `oio::Read` and `oio::Write`, learning a great deal with each refactor. Each one has been unique and incremental because we can't overhaul them all at once, and I've had to familiarize myself with the design's limitations and advantages. I believe this will be the final iteration. If it performs well, we'll aim to move this design towards release.
   
   ---
   
   ## Current Design
   
   use `async fn` for more readable:
   
   ```rust
   pub trait Read {
       async fn read_at(&self, offset: u64, limit: usize) -> Result<oio::Buffer>;
   }
   
   pub trait ReadDyn: Unpin + Send + Sync {
       fn read_at_dyn(&self, offset: u64, limit: usize) -> BoxedFuture<Result<oio::Buffer>>;
   }
   
   pub trait Write: Unpin + Send + Sync {
       async fn write(&mut self, bs: Bytes) -> Result<usize>;
       async fn close(&mut self) -> Result<()>;
       async fn abort(&mut self) -> Result<()>;
   }
   
   pub trait WriteDyn: Unpin + Send + Sync {
       fn write_dyn(&mut self, bs: Bytes) -> BoxedFuture<Result<usize>>;
       fn close_dyn(&mut self) -> BoxedFuture<Result<()>>;
       fn abort_dyn(&mut self) -> BoxedFuture<Result<()>>;
   }
   ```
   
   ## Proposed Design
   
   ```rust
   pub trait Read {
       async fn read_at<B: BufMut>(&self, buf: B, offset: u64) -> (B, Result<usize>);
   }
   
   pub trait ReadDyn: Unpin + Send + Sync {
       fn read_at_dyn<B: BufMut>(&self, buf: B, offset: u64) -> BoxedFuture<(B, Result<usize>)>;    
   }
   
   pub trait Write: Unpin + Send + Sync {
       async fn write<B: Buf>(&mut self, buf: B) -> (B, Result<usize>);
       async fn close(&mut self) -> Result<()>;
       async fn abort(&mut self) -> Result<()>;
   }
   
   pub trait WriteDyn: Unpin + Send + Sync {
       fn write_dyn<B: Buf>(&mut self, bs: B) -> BoxedFuture<(B, Result<usize>)>;
       fn close_dyn(&mut self) -> BoxedFuture<Result<()>>;
       fn abort_dyn(&mut self) -> BoxedFuture<Result<()>>;
   }
   ```
   
   ## Motivation
   
   - Remove the allocation of `oio::Buffer`
   - Make it possible to users pass a buffer pool by accepting `impl Buf` and `impl BufMut`
   - Allowing vectored IO by default by [Buf::chunks_vectored](https://docs.rs/bytes/latest/bytes/buf/trait.Buf.html#method.chunks_vectored)
   - Remove the extra copy while users calling opendal via `futures::AsyncRead` or `futures::AsyncWrite`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Owned Buffer Design for `oio::Read` and `oio::Write` [opendal]

["Xuanwo (via GitHub)" <gi...@apache.org>]

Xuanwo closed issue #4401: Owned Buffer Design for `oio::Read` and `oio::Write`
URL: https://github.com/apache/opendal/issues/4401


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org