[jira] [Updated] (TEZ-1446) Move the fetch code for local disk fetch from data movement event handlers to fetcher

["Prakash Ramachandran (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prakash Ramachandran updated TEZ-1446:
--------------------------------------

    Attachment: TEZ-1446.1.patch

TEZ-1446.1.patch changes
- move fetch code for local disk from shuffle input event handler to fetcher
- use symlinks instead of directly pointing to local file to fix the issue where FileChunkPath getting added to TreeSet causes issues while merging.
- unit tests

> Move the fetch code for local disk fetch from data movement event handlers to fetcher
> -------------------------------------------------------------------------------------
>
>                 Key: TEZ-1446
>                 URL: https://issues.apache.org/jira/browse/TEZ-1446
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Prakash Ramachandran
>            Assignee: Prakash Ramachandran
>         Attachments: TEZ-1446.1.patch
>
>
> doing the fetch in the EventHandler has the drawback of tying up the EventHandling thread, in case it takes time to read the Index file. Even, for a healthy disk - we can do without any delays on this thread, so that the fetcher have some work to do.  
> Also Need to add Unit test cases for the local fetch.



--
This message was sent by Atlassian JIRA
(v6.2#6252)