You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Rajesh Balamohan (Jira)" <ji...@apache.org> on 2020/07/28 10:55:00 UTC

[jira] [Created] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

Rajesh Balamohan created TEZ-4208:
-------------------------------------

             Summary: Pipelinesorter uses single SortSpan after spill
                 Key: TEZ-4208
                 URL: https://issues.apache.org/jira/browse/TEZ-4208
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Rajesh Balamohan
         Attachments: q67_sorter.log

Though it could have created multiple spans, tez always uses the first span after spill. It is quite possible that other spans are bigger compared to the first one, due to progressive space allocation.  Fixing this would help in reducing the number of spills (depending on the jobs) and lesser load for indexcache entries (as lesser number of files have to be opened).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)