You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kevin Wilfong (Updated) (JIRA)" <ji...@apache.org> on 2011/10/04 19:17:35 UTC
[jira] [Updated] (HIVE-2479) Log more Hadoop task counter values in the MapRedStats class.

     [ https://issues.apache.org/jira/browse/HIVE-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kevin Wilfong updated HIVE-2479:
--------------------------------

    Description: 
We should log more of the Hadoop task tracker counters in the MapRedStats class, in order to make them available to hooks and improve logging.  Specifically these are the counters we should add:

    MAP_SPILL_CPU,
    MAP_SPILL_WALLCLOCK,
    MAP_SPILL_NUMBER,
    MAP_SPILL_BYTES,
    MAP_MEM_SORT_CPU,
    MAP_MEM_SORT_WALLCLOCK,
    MAP_MERGE_CPU,
    MAP_MERGE_WALLCLOCK,
    REDUCE_SHUFFLE_BYTES,
    REDUCE_COPY_WALLCLOCK,
    REDUCE_COPY_CPU,
    REDUCE_SORT_WALLCLOCK,
    REDUCE_SORT_CPU,
    MAP_TASK_WALLCLOCK,
    REDUCE_TASK_WALLCLOCK,
    MAP_INPUT_BYTES

  was:
We should log more of the Hadoop task tracker counters in the MapRedStats object, in order to make them available to hooks and improve logging.  Specifically these are the counters we should add:

    MAP_SPILL_CPU,
    MAP_SPILL_WALLCLOCK,
    MAP_SPILL_NUMBER,
    MAP_SPILL_BYTES,
    MAP_MEM_SORT_CPU,
    MAP_MEM_SORT_WALLCLOCK,
    MAP_MERGE_CPU,
    MAP_MERGE_WALLCLOCK,
    REDUCE_SHUFFLE_BYTES,
    REDUCE_COPY_WALLCLOCK,
    REDUCE_COPY_CPU,
    REDUCE_SORT_WALLCLOCK,
    REDUCE_SORT_CPU,
    MAP_TASK_WALLCLOCK,
    REDUCE_TASK_WALLCLOCK,
    MAP_INPUT_BYTES

        Summary: Log more Hadoop task counter values in the MapRedStats class.  (was: Log more Hadoop task counter values in the MapRedStats object.)
    
> Log more Hadoop task counter values in the MapRedStats class.
> -------------------------------------------------------------
>
>                 Key: HIVE-2479
>                 URL: https://issues.apache.org/jira/browse/HIVE-2479
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>
> We should log more of the Hadoop task tracker counters in the MapRedStats class, in order to make them available to hooks and improve logging.  Specifically these are the counters we should add:
>     MAP_SPILL_CPU,
>     MAP_SPILL_WALLCLOCK,
>     MAP_SPILL_NUMBER,
>     MAP_SPILL_BYTES,
>     MAP_MEM_SORT_CPU,
>     MAP_MEM_SORT_WALLCLOCK,
>     MAP_MERGE_CPU,
>     MAP_MERGE_WALLCLOCK,
>     REDUCE_SHUFFLE_BYTES,
>     REDUCE_COPY_WALLCLOCK,
>     REDUCE_COPY_CPU,
>     REDUCE_SORT_WALLCLOCK,
>     REDUCE_SORT_CPU,
>     MAP_TASK_WALLCLOCK,
>     REDUCE_TASK_WALLCLOCK,
>     MAP_INPUT_BYTES

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira