You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Kuhu Shukla (JIRA)" <ji...@apache.org> on 2017/02/03 21:20:51 UTC

[jira] [Updated] (TEZ-3607) ShuffleScheduler#allEventsReceived check is too tight

     [ https://issues.apache.org/jira/browse/TEZ-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kuhu Shukla updated TEZ-3607:
-----------------------------
    Attachment: TEZ-3607.001.patch

[~jeagles], fixes the check assuming the given/current design does not remove the entries from the map. Request for review/comments. Thanks a lot!

> ShuffleScheduler#allEventsReceived check is too tight
> -----------------------------------------------------
>
>                 Key: TEZ-3607
>                 URL: https://issues.apache.org/jira/browse/TEZ-3607
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>         Attachments: TEZ-3607.001.patch
>
>
> In case of skipped records being non zero, the allEventsReceived() call can return false even when all the events have in fact been received since pathtoAttemptsMap size plus the skipped counter may be more than the total number of inputs since we don't take out entries from the map if skipped counter is incremented.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)