You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ahmed Radwan (JIRA)" <ji...@apache.org> on 2012/06/02 08:33:23 UTC
[jira] [Created] (MAPREDUCE-4306) Problem running Distributed Shell
applications as a user other than the one started the daemons
Ahmed Radwan created MAPREDUCE-4306:
---------------------------------------
Summary: Problem running Distributed Shell applications as a user other than the one started the daemons
Key: MAPREDUCE-4306
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0-alpha
Reporter: Ahmed Radwan
Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4306) Problem running Distributed Shell
applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-4306:
------------------------------------
Attachment: MAPREDUCE-4306.patch
Here is the patch. I have manually tested it using a single-node cluster. Where I started the daemons using one user and then confirmed that a different user and the user started the daemons can both successfully run distributed shell jobs.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Attachments: MAPREDUCE-4306.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-4306) Problem running Distributed Shell
applications as a user other than the one started the daemons
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-4306.
------------------------------------
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.2-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293799#comment-13293799 ]
Tom White commented on MAPREDUCE-4306:
--------------------------------------
{noformat}
+ try {
+ amUser = cliParser.getOptionValue("user", UserGroupInformation
+ .getCurrentUser().getShortUserName());
+ } catch (IOException e) {
+ LOG.info("Getting current user info failed" + e.getMessage());
+ }
{noformat}
If a user has been specified on the command line then there is no need to find the current user. In fact, if the current user can't be found then {{amUser}} won't be set. (Also, there's a space missing in the info line.)
It's probably worth going through where user values are null or the empty string to check they are correct (e.g. on ContainerLaunchContext#setUser).
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291350#comment-13291350 ]
Hadoop QA commented on MAPREDUCE-4306:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12531308/MAPREDUCE-4306.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2445//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2445//console
This message is automatically generated.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan reassigned MAPREDUCE-4306:
---------------------------------------
Assignee: Ahmed Radwan
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397113#comment-13397113 ]
Hudson commented on MAPREDUCE-4306:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2443 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2443/])
MAPREDUCE-4306. Fix distributed shell to work with users other than the one running the daemons. (Contributed by Ahmed Radwan) (Revision 1351876)
Result = SUCCESS
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1351876
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294162#comment-13294162 ]
Hadoop QA commented on MAPREDUCE-4306:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12531919/MAPREDUCE-4306_rev2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2461//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2461//console
This message is automatically generated.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397102#comment-13397102 ]
Hudson commented on MAPREDUCE-4306:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2373 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2373/])
MAPREDUCE-4306. Fix distributed shell to work with users other than the one running the daemons. (Contributed by Ahmed Radwan) (Revision 1351876)
Result = SUCCESS
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1351876
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293013#comment-13293013 ]
Ahmed Radwan commented on MAPREDUCE-4306:
-----------------------------------------
As I mentioned earlier, I have manually tested this issue and confirmed the corrected behavior with the patch.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4306) Problem running Distributed Shell
applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-4306:
------------------------------------
Fix Version/s: 2.0.1-alpha
Status: Patch Available (was: Open)
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397143#comment-13397143 ]
Hudson commented on MAPREDUCE-4306:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2392 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2392/])
MAPREDUCE-4306. Fix distributed shell to work with users other than the one running the daemons. (Contributed by Ahmed Radwan) (Revision 1351876)
Result = FAILURE
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1351876
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397073#comment-13397073 ]
Siddharth Seth commented on MAPREDUCE-4306:
-------------------------------------------
Thanks Ahmed. This patch looks good, assuming you tried this out as well ?
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397098#comment-13397098 ]
Siddharth Seth commented on MAPREDUCE-4306:
-------------------------------------------
+1. Committing this. Thanks Ahmed.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397089#comment-13397089 ]
Ahmed Radwan commented on MAPREDUCE-4306:
-----------------------------------------
Yes, I did. Thanks Siddarth!
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4306) Problem running Distributed Shell
applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-4306:
------------------------------------
Attachment: MAPREDUCE-4306_rev3.patch
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397431#comment-13397431 ]
Hudson commented on MAPREDUCE-4306:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1082 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1082/])
MAPREDUCE-4306. Fix distributed shell to work with users other than the one running the daemons. (Contributed by Ahmed Radwan) (Revision 1351876)
Result = FAILURE
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1351876
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397529#comment-13397529 ]
Hudson commented on MAPREDUCE-4306:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1115 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1115/])
MAPREDUCE-4306. Fix distributed shell to work with users other than the one running the daemons. (Contributed by Ahmed Radwan) (Revision 1351876)
Result = FAILURE
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1351876
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4306) Problem running Distributed Shell
applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-4306:
------------------------------------
Attachment: MAPREDUCE-4306_rev2.patch
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291321#comment-13291321 ]
Ahmed Radwan commented on MAPREDUCE-4306:
-----------------------------------------
To reproduce this issue using the tarball on a single node cluster:
1- Start all the daemons using user1.
2- Switch to user2 and try to submit a distributed shell job:
{code}
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar org.apache.hadoop.yarn.applications.distributedshell.Client --jar ./share/hadoop/mapreduce/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar --shell_command ls --num_containers 1 --debug
{code}
I'll be uploading a patch momentarily.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396223#comment-13396223 ]
Ahmed Radwan commented on MAPREDUCE-4306:
-----------------------------------------
Thanks Siddharth for the review!
I agree, I think it is better to completely remove the "-user" option. I originally thought of just keeping it in case it can be used for testing or other purposes. But leaving it now may lead to confusion, and also setting it to something other than the original user will lead to failure as described above.
Also reading ApplicationConstants.Environment.USER is simpler than reevaluating the username from ugi (which will give the same result after all). I have updated the patch accordingly. Thanks!
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294134#comment-13294134 ]
Ahmed Radwan commented on MAPREDUCE-4306:
-----------------------------------------
Thanks Tom for the review!
I have updated the patch to handle the case you highlighted (when the current user cannot be found and it is also not specified on the command line).
For the case of empty or null user, this will still be invalid, since the user name (picked from ContainerLaunchContext#setUser) is used by DefaultContainerExecutor.launchContainer(..) to construct the container directory. Basically the FileNotFoundException that was preventing the application from running was originating from line 126: lfs.mkdir(containerDir, null, false) because the parent directory was not correct, as it was using the wrong user in constructing the path.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4306) Problem running Distributed Shell
applications as a user other than the one started the daemons
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-4306:
--------------------------------------
Resolution: Fixed
Target Version/s: 2.0.1-alpha
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk and branch-2.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396242#comment-13396242 ]
Hadoop QA commented on MAPREDUCE-4306:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12532455/MAPREDUCE-4306_rev3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2471//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2471//console
This message is automatically generated.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch, MAPREDUCE-4306_rev3.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4306) Problem running Distributed
Shell applications as a user other than the one started the daemons
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396080#comment-13396080 ]
Siddharth Seth commented on MAPREDUCE-4306:
-------------------------------------------
The "-user" option in general seems to be broken. Even after this patch, the AM will be localized as the original user - since the RM picks up the username from ugi.
Maybe we should remove the "-user" option completely? and use ApplicationConstants.Environment.USER in the AM - which is anyway set by the RM, based on the logged in user.
> Problem running Distributed Shell applications as a user other than the one started the daemons
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4306
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-4306.patch, MAPREDUCE-4306_rev2.patch
>
>
> Using the tarball, if you start the yarn daemons using one user and then switch to a different user. You can successfully run MR jobs, but DS jobs fail to run. Only able to run DS jobs using the user who started the daemons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira