You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Ming Ma (JIRA)" <ji...@apache.org> on 2017/03/27 05:55:41 UTC

[jira] [Updated] (TEZ-3666) Integer overflow in ShuffleVertexManagerBase

     [ https://issues.apache.org/jira/browse/TEZ-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ming Ma updated TEZ-3666:
-------------------------
    Attachment: TEZ-3666.patch

Here is the draft patch. The code path with the issue is only used by FairShuffleVertexManager. Besides the actual fix, also added additional loggings for the estimated partition stats and how partitions are assigned to destination tasks.

> Integer overflow in ShuffleVertexManagerBase
> --------------------------------------------
>
>                 Key: TEZ-3666
>                 URL: https://issues.apache.org/jira/browse/TEZ-3666
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Ming Ma
>         Attachments: TEZ-3666.patch
>
>
> In function getExpectedStatsInAtIndex, {{statsInMB[index] * numTasks / numVMEventsReceived}} could cause Integer overflow, for example when statsInMB[index]  == 30000 and numTasks == 200000.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)