You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Niels Basjes (JIRA)" <ji...@apache.org> on 2012/12/29 21:54:12 UTC

[jira] [Commented] (MAPREDUCE-346) Report Map-Reduce Framework Counters in pipeline order

    [ https://issues.apache.org/jira/browse/MAPREDUCE-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540994#comment-13540994 ] 

Niels Basjes commented on MAPREDUCE-346:
----------------------------------------

I just ran the wordcount example from the current trunk and the output looks like this:
{code}
	File System Counters
		FILE: Number of bytes read=1355162
		FILE: Number of bytes written=4112351
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=1082
		Map output records=3884
		Map output bytes=53040
		Map output materialized bytes=42292
		Input split bytes=2785
		Combine input records=3884
		Combine output records=2263
		Reduce input groups=990
		Reduce shuffle bytes=0
		Reduce input records=2263
		Reduce output records=990
		Spilled Records=5299
		Shuffled Maps =0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=677
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=8783134720
	File Input Format Counters 
		Bytes Read=38869
	File Output Format Counters 
		Bytes Written=21444
{code}

To me the order of a few items in this list appear "out of place": I would expect 
"Shuffled Maps", "Failed Shuffles" and "Merged Map outputs" somewhere between the map and the reduce instead of after the reduce.

Other than that (and the trailing space at the end of "Shuffled Maps ") I propose we close this improvement as "Fixed".

                
> Report Map-Reduce Framework Counters in pipeline order
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-346
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-346
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>         Attachments: 5216_v1.patch
>
>
> Currently there is no order in which counters are printed. It would be more user friendly if Map-Reduce Framework counters are reported in the pipeline order.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira