You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hemanth Yamijala (JIRA)" <ji...@apache.org> on 2008/09/05 10:37:44 UTC
[jira] Created: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Check for invalid queues in capacity scheduler
----------------------------------------------
Key: HADOOP-4079
URL: https://issues.apache.org/jira/browse/HADOOP-4079
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.19.0
Reporter: Hemanth Yamijala
Assignee: Hemanth Yamijala
Priority: Blocker
Fix For: 0.19.0
The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635301#action_12635301 ]
Hadoop QA commented on HADOOP-4079:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12391004/HADOOP-4079-2.patch
against trunk revision 699676.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3389/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3389/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3389/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3389/console
This message is automatically generated.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Status: Patch Available (was: Open)
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch, HADOOP-4079-4.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Status: Patch Available (was: Open)
Running the patch thro' Hudson.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634480#action_12634480 ]
Hemanth Yamijala commented on HADOOP-4079:
------------------------------------------
- The warning statement refers to hadoop-defaults, but the configuration can be in hadoop-site as well. I would suggest we print something like: "Queue <queue-name> is not configured in mapred.queue.names property in site configuration."
- The test case {{testGetQueues}} must configure more than one queue and verify they are present. The names should also be different from 'default'.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4079-1.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628593#action_12628593 ]
Hemanth Yamijala commented on HADOOP-4079:
------------------------------------------
A simple proposal is to do what is done in HADOOP-3698. That is, we should parse the configuration at initialization and populate a mapping between queue names and the configured properties. This way, it will be easy to check if the map contains the queue name being requested in the APIs.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633623#action_12633623 ]
Owen O'Malley commented on HADOOP-4079:
---------------------------------------
I'd suggest that we warn in the JobTracker log at start up if the capacity scheduler configuration defines a queue that isn't present in the QueueManager. It will most likely be a typo on one side or the other. Not defining properties should be ok, since they will just fall back to the defaults defined via HADOOP-4178.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-4079:
------------------------------
Status: Open (was: Patch Available)
This patch does not apply anymore, so removing from queue while this is fixed.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch, HADOOP-4079-4.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Attachment: HADOOP-4079-4.patch
Attaching latest patch merged with the trunk
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch, HADOOP-4079-4.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-4079:
-------------------------------------
Status: Patch Available (was: Open)
Running through Hudson.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-4079:
-------------------------------------
Component/s: contrib/capacity-sched
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Status: Open (was: Patch Available)
Patch does not apply to the trunk cleanly anymore
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch, HADOOP-4079-4.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635953#action_12635953 ]
Sreekanth Ramakrishnan commented on HADOOP-4079:
------------------------------------------------
The core test failed is not releated to this issue.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648310#action_12648310 ]
Hadoop QA commented on HADOOP-4079:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12391192/HADOOP-4079-3.patch
against trunk revision 718232.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3600/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3600/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3600/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3600/console
This message is automatically generated.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635951#action_12635951 ]
Hadoop QA commented on HADOOP-4079:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12391192/HADOOP-4079-3.patch
against trunk revision 700628.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3408/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3408/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3408/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3408/console
This message is automatically generated.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Attachment: HADOOP-4079-1.patch
Attaching patch with following changes:
- Added {code:title=CapacitySchedulerConf} Set<String> getQueues() {code}
- Modified _CapacityTaskScheduler_ to check the number of queues configured in _capacity_scheduler.xml_ against queues in _QueueManager_
- Added a test case to test new method in _CapacitySchedulerConf_
- Modified _TestCapacityScheduler_ to support _getQueues()_
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4079-1.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-4079:
----------------------------------
Fix Version/s: (was: 0.19.0)
0.20.0
Status: Open (was: Patch Available)
This patch doesn't apply to trunk any more.
It doesn't seem to be a blocker for 0.19, so I'm moving it to 0.20.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-4079:
----------------------------------
Priority: Major (was: Blocker)
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633221#action_12633221 ]
Hemanth Yamijala commented on HADOOP-4079:
------------------------------------------
Considering how the code has turned out, the Capacity Scheduler iterates over queues it gets from the framework's {{QueueManager}} class. Now, when it asks the {{CapacitySchedulerConf}} for a property for this queue, if nothing is configured for this queue in {{capacity-scheduler.xml}}, a hard-coded default value is returned. This may not be what the administrator intended, because the defaults are not defined by him/her. Hence, this issue exists.
HADOOP-4178 is trying to implement a mechanism to make these default values itself configurable. If it does that, then it would be a feature that if a property is not configured for a queue in the capacity scheduler's configuration, the configured default is used. This makes it very simple for the administrator to define new queues. And it would invalidate the need to check for invalid queues, as required by this issue.
So in summary, if HADOOP-4178 is fixed, we can mark this bug as invalid. Comments ?
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Attachment: HADOOP-4079-2.patch
Attaching patch with comments from Hemanth, incorporated
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-4079:
-------------------------------------
Assignee: Sreekanth Ramakrishnan (was: Hemanth Yamijala)
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Status: Patch Available (was: Open)
Resubmitting the patch for running thro' Hudson, the patch still applies against the trunk.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635281#action_12635281 ]
Hemanth Yamijala commented on HADOOP-4079:
------------------------------------------
Looks good to me.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654686#action_12654686 ]
Hadoop QA commented on HADOOP-4079:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12395256/HADOOP-4079-4.patch
against trunk revision 724531.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3691/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3691/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3691/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3691/console
This message is automatically generated.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch, HADOOP-4079-4.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4079) Check for invalid queues in
capacity scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655080#action_12655080 ]
Sreekanth Ramakrishnan commented on HADOOP-4079:
------------------------------------------------
The core tests failed are not related to the pacth. The patch modifies only the contrib project capacity scheduler.
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch, HADOOP-4079-4.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Status: Open (was: Patch Available)
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4079) Check for invalid queues in capacity
scheduler
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-4079:
-------------------------------------------
Attachment: HADOOP-4079-3.patch
Attaching patch merging with the latest trunk. The conflict was because of the changes which went into [HADOOP-4178|https://issues.apache.org/jira/browse/HADOOP-4178]
> Check for invalid queues in capacity scheduler
> ----------------------------------------------
>
> Key: HADOOP-4079
> URL: https://issues.apache.org/jira/browse/HADOOP-4079
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.20.0
>
> Attachments: HADOOP-4079-1.patch, HADOOP-4079-2.patch, HADOOP-4079-3.patch
>
>
> The {{ResourceManagerConf}} class that is being moved to the capacity scheduler of HADOOP-3445 needs to check for a queue name that is not configured in the {{resource-manager-conf.xml}} file in its APIs and fail if it is not available. This feature was originally available, but due to subsequent changes in HADOOP-3698, queues are no longer being configured as part of the mentioned configuration file. Hence we need a different mechanism to check for valid queues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.