[jira] [Issue Comment Deleted] (RATIS-979) Ratis streaming

["runzhiwang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/RATIS-979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

runzhiwang updated RATIS-979:
-----------------------------
    Comment: was deleted

(was: [~szetszwo] Hi, I have test the performance of HDFS and Ratis streaming. But Ratis streaming is about 30% slower than HDFS. When 2 clients * 128MB * 200 files, Ratis streaming cost about 80 seconds, HDFS cost about 60 seconds. Not sure why.

In my test, as the following code shows,  I did not call startTransaction to avoid submitting raft request, so that we can only compare the performance of network and write disk.

If you need the flame graph, I can attach them.

{code:java}
composeAsync(info.getPrevious(), requestExecutor, n -> JavaUtils.allOf(remoteWrites)
        .thenCombineAsync(localWrite, (v, bytesWritten) -> {
          if (request.getType() == Type.STREAM_HEADER
              || (request.getType() == Type.STREAM_DATA && !close)) {
            sendReply(remoteWrites, request, bytesWritten, ctx);
          } else if (close) {
            sendReply(remoteWrites, request, bytesWritten, ctx);
  //            if (info.isPrimary()) {
  //              // after all server close stream, primary server start transaction
  //              startTransaction(info, request, bytesWritten, ctx);
  //            } else {
 //              sendReply(remoteWrites, request, bytesWritten, ctx);
 //            }
          } else {
            throw new IllegalStateException(this + ": Unexpected type " + request.getType() + ", request=" + request);
          }
          return null;
        }, requestExecutor)).whenComplete((v, exception) -> {
{code}
)

> Ratis streaming
> ---------------
>
>                 Key: RATIS-979
>                 URL: https://issues.apache.org/jira/browse/RATIS-979
>             Project: Ratis
>          Issue Type: New Feature
>          Components: Streaming
>            Reporter: Tsz-wo Sze
>            Assignee: Tsz-wo Sze
>            Priority: Major
>         Attachments: RatisStreaming20200929.pdf, screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> In this JIRA, we design and implement Ratis Streaming with zero buffer copying and asynchronous event driven.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)