You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2017/07/24 18:04:00 UTC

[jira] [Created] (IMPALA-5706) Parallelise read I/O in sorter

Tim Armstrong created IMPALA-5706:
-------------------------------------

             Summary: Parallelise read I/O in sorter
                 Key: IMPALA-5706
                 URL: https://issues.apache.org/jira/browse/IMPALA-5706
             Project: IMPALA
          Issue Type: Sub-task
          Components: Backend
    Affects Versions: Impala 2.10.0
            Reporter: Tim Armstrong


IMPALA-3200 offers an opportunity to improve the spilling sort algorithm:

* Use the reliability of reservations to select the most efficient order to conduct merges in (rather than greedily trying to maximise the fan-in of the current merge). We want to minimise the depth of the merge tree, then structure the tree based on the preferred fan-in.
* Do multiple-buffering of the stream being written (this happens automatically if there are free buffers in the BufferPool client).
* Do multiple-buffering of the streams being read, instead of blocking on read I/O frequently.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)