You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Jiaqi Tan (JIRA)" <ji...@apache.org> on 2009/06/11 22:50:07 UTC

[jira] Created: (HADOOP-6013) Add reduce ID to shuffle clienttrace

Add reduce ID to shuffle clienttrace
------------------------------------

                 Key: HADOOP-6013
                 URL: https://issues.apache.org/jira/browse/HADOOP-6013
             Project: Hadoop Core
          Issue Type: Improvement
          Components: mapred
    Affects Versions: 0.20.0
            Reporter: Jiaqi Tan
             Fix For: 0.21.0


Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-6013) Add reduce ID to shuffle clienttrace

Posted by "Jiaqi Tan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jiaqi Tan updated HADOOP-6013:
------------------------------

    Status: Patch Available  (was: Open)

Add reduce ID to shuffle clienttrace messages

> Add reduce ID to shuffle clienttrace
> ------------------------------------
>
>                 Key: HADOOP-6013
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6013
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Jiaqi Tan
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-6013.patch
>
>
> Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-6013) Add reduce ID to shuffle clienttrace

Posted by "Jiaqi Tan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720760#action_12720760 ] 

Jiaqi Tan commented on HADOOP-6013:
-----------------------------------

Both failed unit tests are not related to this patch.

> Add reduce ID to shuffle clienttrace
> ------------------------------------
>
>                 Key: HADOOP-6013
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6013
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Jiaqi Tan
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-6013.patch
>
>
> Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-6013) Add reduce ID to shuffle clienttrace

Posted by "Jiaqi Tan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jiaqi Tan updated HADOOP-6013:
------------------------------

    Attachment: HADOOP-6013.patch

Adds reduceID to shuffle clienttrace messages

> Add reduce ID to shuffle clienttrace
> ------------------------------------
>
>                 Key: HADOOP-6013
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6013
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Jiaqi Tan
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-6013.patch
>
>
> Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-6013) Add reduce ID to shuffle clienttrace

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719973#action_12719973 ] 

Hadoop QA commented on HADOOP-6013:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12410427/HADOOP-6013.patch
  against trunk revision 785065.

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no tests are needed for this patch.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    -1 core tests.  The patch failed core unit tests.

    -1 contrib tests.  The patch failed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/502/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/502/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/502/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/502/console

This message is automatically generated.

> Add reduce ID to shuffle clienttrace
> ------------------------------------
>
>                 Key: HADOOP-6013
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6013
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Jiaqi Tan
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-6013.patch
>
>
> Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-6013) Add reduce ID to shuffle clienttrace

Posted by "Jiaqi Tan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jiaqi Tan updated HADOOP-6013:
------------------------------

    Priority: Minor  (was: Major)

Lowered severity to minor.

> Add reduce ID to shuffle clienttrace
> ------------------------------------
>
>                 Key: HADOOP-6013
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6013
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Jiaqi Tan
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-6013.patch
>
>
> Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-6013) Add reduce ID to shuffle clienttrace

Posted by "Jiaqi Tan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718980#action_12718980 ] 

Jiaqi Tan commented on HADOOP-6013:
-----------------------------------

Destination reduceID of shuffle needed in Chukwa for building causal tracing between maps and reduces; this saves the effort of having to parse task logs for shuffle information, and greatly reduces the number of adaptors needed (eliminates need for per-task adaptors).

> Add reduce ID to shuffle clienttrace
> ------------------------------------
>
>                 Key: HADOOP-6013
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6013
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Jiaqi Tan
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-6013.patch
>
>
> Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.