You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2009/05/20 20:28:45 UTC

[jira] Commented: (HADOOP-5876) Shuffling information logged to userlogs/attempt_####_###_r_###_#/syslogs

    [ https://issues.apache.org/jira/browse/HADOOP-5876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711277#action_12711277 ] 

Arun C Murthy commented on HADOOP-5876:
---------------------------------------

What kind of shuffle information are you looking for? 

Overall it doesn't make sense to start pushing task-logs to the tasktracker logs - it's just easier to collect userlogs. Even more, it makes absolutely no sense to push it to job-history, seems to me that we should be using some metrics at best.

> Shuffling information logged to userlogs/attempt_####_###_r_###_#/syslogs
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-5876
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5876
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.19.0, 0.19.1, 0.20.0
>         Environment: Redhat EL 5.1, Java 6
>            Reporter: Eric Yang
>            Priority: Critical
>
> Shuffling information is currently logged to userlogs, it would be ideal to have this information consolidated in task tracker log or job history log file for down stream log collection and analysis program (Chukwa) to pickup.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.