You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Siddharth Seth (JIRA)" <ji...@apache.org> on 2015/04/03 01:10:53 UTC

[jira] [Commented] (TEZ-2265) All inputs/outputs in a task share the same counter object

    [ https://issues.apache.org/jira/browse/TEZ-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393691#comment-14393691 ] 

Siddharth Seth commented on TEZ-2265:
-------------------------------------

Patch looks good. Is enabling per IO counters (which should be the default soon), not sufficient to ensure separation ?

> All inputs/outputs in a task share the same counter object
> ----------------------------------------------------------
>
>                 Key: TEZ-2265
>                 URL: https://issues.apache.org/jira/browse/TEZ-2265
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: TEZ-2265.1.patch
>
>
> This makes one input which reads the value of a counter to see updates that were made by some other input/output. Any decisions based on those counter values could be wrong and as of today, the output data size reported  in vertex manager event or by shuffle pipelining is potentially wrong.
> Simple fix would be to have each IO have their own counters and merge them before reporting back up in the heartbeat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)