You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2020/02/21 13:33:00 UTC

[jira] [Assigned] (TEZ-3350) Shuffle spills are not spilled to a container-specific directory

     [ https://issues.apache.org/jira/browse/TEZ-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

László Bodor reassigned TEZ-3350:
---------------------------------

    Assignee: László Bodor

> Shuffle spills are not spilled to a container-specific directory
> ----------------------------------------------------------------
>
>                 Key: TEZ-3350
>                 URL: https://issues.apache.org/jira/browse/TEZ-3350
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.7.1
>            Reporter: Jason Darrell Lowe
>            Assignee: László Bodor
>            Priority: Major
>
> If a Tez task receives too much input data and needs to spill the inputs to disk it ends up using a path that is not container-specific.  Therefore YARN will not automatically cleanup these files when the container exits as it should, and instead the files linger until the entire application completes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)