You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2019/11/07 18:50:00 UTC

[jira] [Created] (IMPALA-9134) Parallelise flush in data stream sender

Tim Armstrong created IMPALA-9134:
-------------------------------------

             Summary: Parallelise flush in data stream sender
                 Key: IMPALA-9134
                 URL: https://issues.apache.org/jira/browse/IMPALA-9134
             Project: IMPALA
          Issue Type: Improvement
          Components: Distributed Exec
            Reporter: Tim Armstrong


The data stream sender currently does a synchronous RPC to close each channel https://github.com/apache/impala/blob/d4648e8/be/src/runtime/krpc-data-stream-sender.cc#L565.

This is suboptimal because it serializes the network round-trips and takes sum(RTT) over all the destinations in the best case, where no data needs to be flushed or  2 * sum(RTT) in the worst case if all channels need to flush data.

If the RPCs were done asynchronously and overlapped with each other, we could get this down to 2 * max(RTT).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)