[GitHub] [incubator-ratis] szetszwo commented on pull request #298: [WIP] RATIS-1111. Change the FileStore example to use Streaming

[GitBox <gi...@apache.org>]

szetszwo commented on pull request #298:
URL: https://github.com/apache/incubator-ratis/pull/298#issuecomment-733402768


   Consider the existing code for async:
   ```
   //FileStoreClient
     private static <OUTPUT, THROWABLE extends Throwable> OUTPUT writeImpl(
         CheckedFunction<ByteString, OUTPUT, THROWABLE> sendFunction,
         String path, long offset, boolean close, ByteBuffer data)
         throws THROWABLE {
       final WriteRequestHeaderProto.Builder header = WriteRequestHeaderProto.newBuilder()
           .setPath(ProtoUtils.toByteString(path))
           .setOffset(offset)
           .setLength(data.position())
           .setClose(close);
   
       final WriteRequestProto.Builder write = WriteRequestProto.newBuilder()
           .setHeader(header)
           .setData(ByteString.copyFrom(data));
   
       final FileStoreRequestProto request = FileStoreRequestProto.newBuilder().setWrite(write).build();
       return sendFunction.apply(request.toByteString());
     }
   ```
   The idea is to define the streaming message format as below:
   ```
   <header-length><header><data-length><data>
   ```
   The entire message could be big, say 100MB.  We divide it in chunks, say 1MB, and send them using DataStreamOutput.writeAsync.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org