You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Suhas Gogate (JIRA)" <ji...@apache.org> on 2009/06/01 18:17:07 UTC
[jira] Created: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Existing diagnostic rules fail for MAP ONLY jobs
------------------------------------------------
Key: HADOOP-5950
URL: https://issues.apache.org/jira/browse/HADOOP-5950
Project: Hadoop Core
Issue Type: Bug
Reporter: Suhas Gogate
some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Affects Version/s: 0.20.0
Fix Version/s: 0.21.0
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Attachment: vaidya-patch-06082009.patch
Added JOBTYPE and EXECUTION_TIME KEYS. Also printing the job information summary in XML reports.
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch-06082009.patch, vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Release Note: hadoop vaidya counter names LOCAL_BYTES_READ and LOCAL_BYTES_WRITTEN are changed to respectively FILE_BYTES_READ, FILE_BYTES_WRITTEN as per current hadoop counter names.
Status: Patch Available (was: Open)
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Status: Open (was: Patch Available)
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715441#action_12715441 ]
Hadoop QA commented on HADOOP-5950:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12409569/vaidya-patch1-06012009.patch
against trunk revision 780875.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/449/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/449/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/449/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/449/console
This message is automatically generated.
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Status: Open (was: Patch Available)
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch-06082009.patch, vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721878#action_12721878 ]
Suhas Gogate commented on HADOOP-5950:
--------------------------------------
Failed tests are not related to this patch or vaidya contrib project. So can someone pl. commit the patch as I would like to add a patch for some new rules. Thanks!
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch-06082009.patch, vaidya-patch-06092009.patch, vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Status: Patch Available (was: Open)
Submitted the revised patch
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch-06082009.patch, vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Attachment: vaidya-patch-06092009.patch
previous patch had a minor problem of printing job information on console. Commented out the print statement.
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch-06082009.patch, vaidya-patch-06092009.patch, vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718721#action_12718721 ]
Hadoop QA commented on HADOOP-5950:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12410257/vaidya-patch-06092009.patch
against trunk revision 783672.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/492/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/492/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/492/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/492/console
This message is automatically generated.
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch-06082009.patch, vaidya-patch-06092009.patch, vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Status: Patch Available (was: Open)
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch-06082009.patch, vaidya-patch-06092009.patch, vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717545#action_12717545 ]
Hadoop QA commented on HADOOP-5950:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12410164/vaidya-patch-06082009.patch
against trunk revision 782708.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/483/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/483/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/483/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/483/console
This message is automatically generated.
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch-06082009.patch, vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5950:
---------------------------------
Attachment: vaidya-patch1-06012009.patch
This patch also fixes minor problem of printing error messages to STDOUT.
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5950) Existing diagnostic rules fail for
MAP ONLY jobs
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715559#action_12715559 ]
Suhas Gogate commented on HADOOP-5950:
--------------------------------------
Failed test case is not related to vaidya contrib project and not caused by current patch. So this patch can be committed.
> Existing diagnostic rules fail for MAP ONLY jobs
> ------------------------------------------------
>
> Key: HADOOP-5950
> URL: https://issues.apache.org/jira/browse/HADOOP-5950
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-patch1-06012009.patch
>
>
> some of the existing rules fail with divide by zero or arithmetic exception as map only jobs do not log reduce side counters. Vaidya driver code and Rules need to be modified to take care of such jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.