You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2014/09/03 05:48:51 UTC

[jira] [Updated] (MAPREDUCE-6067) native-task: spilled records counter is incorrect

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated MAPREDUCE-6067:
-----------------------------------
    Attachment: trunk-counters.html
                native-counters.html

Attached the counters pages from two runs (one native and one trunk implementation). It seems like the "map output bytes", "map output materialized bytes", and "map output records" counters are also not being tracked properly by the native collector.

> native-task: spilled records counter is incorrect
> -------------------------------------------------
>
>                 Key: MAPREDUCE-6067
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6067
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: task
>            Reporter: Todd Lipcon
>         Attachments: native-counters.html, trunk-counters.html
>
>
> After running a terasort, I see the spilled records counter at 5028651606, which is about half what I expected to see. Using the non-native collector I see the expected count of 10000000000. It seems the correct number of records were indeed spilled, because the job's output record count is correct.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)