You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hemanth Yamijala (JIRA)" <ji...@apache.org> on 2008/03/10 05:03:46 UTC
[jira] Created: (HADOOP-2982) [HOD] checknodes should look for free
nodes without the jobs attribute
[HOD] checknodes should look for free nodes without the jobs attribute
----------------------------------------------------------------------
Key: HADOOP-2982
URL: https://issues.apache.org/jira/browse/HADOOP-2982
Project: Hadoop Core
Issue Type: Bug
Components: contrib/hod
Affects Versions: 0.16.0
Reporter: Hemanth Yamijala
Fix For: 0.17.0
Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2982) [HOD] checknodes should look for
free nodes without the jobs attribute
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12582995#action_12582995 ]
Hudson commented on HADOOP-2982:
--------------------------------
Integrated in Hadoop-trunk #444 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/444/])
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: 2982.patch
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-2982) [HOD] checknodes should look for
free nodes without the jobs attribute
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala reassigned HADOOP-2982:
----------------------------------------
Assignee: Hemanth Yamijala
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.17.0
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2982) [HOD] checknodes should look for free
nodes without the jobs attribute
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-2982:
-------------------------------------
Attachment: 2982.patch
The attached file modifies the way we compute free nodes. The previous computation had a problem in some corner cases. When there are runaway jobs, the pbsnodes utility reports that some nodes are free, but the pbsmoms report that jobs are running. Such nodes should not be counted as free, as jobs will not successfully run on them. The patch discards such nodes.
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: 2982.patch
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2982) [HOD] checknodes should look for
free nodes without the jobs attribute
Posted by "Rajiv Chittajallu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12582246#action_12582246 ]
Rajiv Chittajallu commented on HADOOP-2982:
-------------------------------------------
+1 for the patch.
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: 2982.patch
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2982) [HOD] checknodes should look for free
nodes without the jobs attribute
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-2982:
-------------------------------------
Release Note: The number of free nodes in the cluster is computed using a better algorithm that filters out inconsistencies in node status as reported by Torque.
Hadoop Flags: [Reviewed]
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: 2982.patch
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2982) [HOD] checknodes should look for free
nodes without the jobs attribute
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-2982:
------------------------------------
Fix Version/s: (was: 0.17.0)
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2982) [HOD] checknodes should look for
free nodes without the jobs attribute
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12582338#action_12582338 ]
Hadoop QA commented on HADOOP-2982:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378622/2982.patch
against trunk revision 619744.
@author +1. The patch does not contain any @author tags.
tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/console
This message is automatically generated.
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: 2982.patch
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2982) [HOD] checknodes should look for free
nodes without the jobs attribute
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-2982:
-------------------------------------
Status: Patch Available (was: Open)
Running past Hudson. No unit test for this patch, as it is a script file change which relies on an external utility.
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: 2982.patch
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2982) [HOD] checknodes should look for free
nodes without the jobs attribute
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-2982:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Hemanth!
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: 2982.patch
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2982) [HOD] checknodes should look for free
nodes without the jobs attribute
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-2982:
-------------------------------------
Priority: Blocker (was: Major)
Fix Version/s: 0.17.0
Marking as blocker, as this will be needed for our Ops team.
> [HOD] checknodes should look for free nodes without the jobs attribute
> ----------------------------------------------------------------------
>
> Key: HADOOP-2982
> URL: https://issues.apache.org/jira/browse/HADOOP-2982
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.17.0
>
>
> Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.