You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Craig Macdonald (JIRA)" <ji...@apache.org> on 2008/08/15 17:15:44 UTC
[jira] Created: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
[HOD] --resource_manager.options is not passed to qsub
------------------------------------------------------
Key: HADOOP-3959
URL: https://issues.apache.org/jira/browse/HADOOP-3959
Project: Hadoop Core
Issue Type: Bug
Components: contrib/hod
Affects Versions: 0.17.1
Environment: Torque
Reporter: Craig Macdonald
--resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632109#action_12632109 ]
Vinod K V commented on HADOOP-3959:
-----------------------------------
+1 for the latest fix.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.patch, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631764#action_12631764 ]
Craig Macdonald commented on HADOOP-3959:
-----------------------------------------
+1. Works fine for me.
bin/hod allocate -d /tmp/a --resource_manager.options=l:arch=x86_64
or
bin/hod allocate -d /tmp/a --resource_manager.options=l:arch=x86
Worth updating documentation?
C
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated HADOOP-3959:
------------------------------------
Fix Version/s: 0.19.0
Status: Patch Available (was: Open)
resubmitting to Hudson.
I'm keen that something like this makes 0.19
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631256#action_12631256 ]
Hadoop QA commented on HADOOP-3959:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12390109/HADOOP-3959-20080915.txt
against trunk revision 695690.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3267/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3267/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3267/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3267/console
This message is automatically generated.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-3959:
------------------------------------
Release Note: (was: HOD client was modified to correctly pass options specified in the resource_manager.options variable to job submission commands like qsub.)
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.patch, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated HADOOP-3959:
------------------------------------
Status: Patch Available (was: Open)
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626529#action_12626529 ]
Hemanth Yamijala commented on HADOOP-3959:
------------------------------------------
bq. Hemanth, can you review this patch?
Craig, thank you for the patch. I am watching this issue and will surely review it in a couple of days time.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated HADOOP-3959:
------------------------------------
Attachment: HADOOP-3959.v1.patch
Patch attached. Copy the resource_manager.options into resource_manager.attrs.
Also, options can be of type int when substituting the values, so do stringification first.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated HADOOP-3959:
------------------------------------
Status: Open (was: Patch Available)
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632112#action_12632112 ]
Hemanth Yamijala commented on HADOOP-3959:
------------------------------------------
Results of ant test-patch:
[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
The -1 on tests is explained above.
Also, I ran hod unit tests manually and they passed. I will commit this patch now.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.patch, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated HADOOP-3959:
------------------------------------
Status: Patch Available (was: Open)
resubmitting to Hudson
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626414#action_12626414 ]
Hadoop QA commented on HADOOP-3959:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12388315/HADOOP-3959.v1.patch
against trunk revision 689666.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3130/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3130/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3130/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3130/console
This message is automatically generated.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-3959:
-------------------------------------
Resolution: Fixed
Release Note: HOD client was modified to correctly pass options specified in the resource_manager.options variable to job submission commands like qsub.
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Craig and Vinod !
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.patch, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633330#action_12633330 ]
Hudson commented on HADOOP-3959:
--------------------------------
Integrated in Hadoop-trunk #611 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/611/])
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.patch, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala reassigned HADOOP-3959:
----------------------------------------
Assignee: Vinod K V
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626525#action_12626525 ]
Craig Macdonald commented on HADOOP-3959:
-----------------------------------------
* TestMiniMRDFSSort.testMapReduceSort is the core test that failed. HOD is not used in core testing, so this failure is not related
* Contrib test failures were related to pipes. Again HOD not used here.
* The HOD test suite does not test that options are passed to the resource manager, hence I have added not test.
Hemanth, can you review this patch?
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated HADOOP-3959:
------------------------------------
Status: Open (was: Patch Available)
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Attachments: HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-3959:
-------------------------------------
Attachment: HADOOP-3959.patch
The attached patch file fixes a small bug which was breaking HOD when the newly enabled {{resource_manager.options}} was used along with wallclock time option, {{-l}}. Other than that patch is the same. I've tested it pass {{-l:ncpus=1}} to the resource_manager.options variable.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.patch, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-3959:
------------------------------
Attachment: HADOOP-3959-20080915.txt
Craig, sorry for holding this up this long. I've seen the patch you've attached, it has some problems. The way you are passing resource_manager.options as nodepooldesc is correct. But the way you've fixed the error that comes up while enabling resource_manager options is not correct. It also is making HOD fail after allocation for me, not for you?
In the code, all keyval pairs used anywhere are formed only by string keys and string values, so assuming an int value and casting it to a string seems to be incorrect. The actual reason why such an int came in was when you enable resource_manager.options, l:nodes is set by HOD to an integer value. I changed it to pass it as a string. This seems to be the correct fix, can you please check this and verify if this works for you? Thanks.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3959) [HOD] --resource_manager.options is
not passed to qsub
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632101#action_12632101 ]
Hemanth Yamijala commented on HADOOP-3959:
------------------------------------------
bq. Worth updating documentation?
Yes. And ideally should be part of this patch. However, given I'd like to get some fixes in before tomorrow, I'd like to update this after the feature freeze date. We normally do this anyway where we verify all the changes made in a release and update documentation for them.
> [HOD] --resource_manager.options is not passed to qsub
> ------------------------------------------------------
>
> Key: HADOOP-3959
> URL: https://issues.apache.org/jira/browse/HADOOP-3959
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Affects Versions: 0.17.1
> Environment: Torque
> Reporter: Craig Macdonald
> Assignee: Vinod K V
> Fix For: 0.19.0
>
> Attachments: HADOOP-3959-20080915.txt, HADOOP-3959.patch, HADOOP-3959.v1.patch
>
>
> --resource_manager.options is ignored in HOD. It isnt passed to the qsub command. This is a problem, as it makes it impossible to add additional resource contraints for scheduling (i.e. give me 64bit nodes)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.