You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Mahadev konar (Created) (JIRA)" <ji...@apache.org> on 2011/11/21 18:40:51 UTC
[jira] [Created] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Oozie jobs are running as oozie user even though they create the jobclient as doAs.
-----------------------------------------------------------------------------------
Key: MAPREDUCE-3443
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Mahadev konar
Assignee: Mahadev konar
Priority: Blocker
Fix For: 0.23.1
Oozie is having issues with job submission, since it does the following:
{code}
doAs(userwhosubmittedjob) {
jobclient = new JobClient(jobconf);
}
jobclient.submitjob()
{code}
In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161633#comment-13161633 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #915 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/915/])
MAPREDUCE-3443. JobClient and Job should function in the context of the UGI which created them. (Contributed by Mahadev Konar)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209231
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Siddharth Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3443:
--------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
+1 based on John's testing and some local testing. The additional javac warning is a deprecation warning.
Committed to trunk and branch-0.23. Thanks Mahadev, and John for trying out the patch.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3443:
-------------------------------------
Attachment: MAPREDUCE-3443.patch
Updated patch merged with the latest codebase in trunk and 0.23.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3443:
-------------------------------------
Status: Open (was: Patch Available)
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161590#comment-13161590 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Hdfs-HAbranch-build #3 (See [https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/3/])
merge MAPREDUCE-3443 from trunk. Contributed by Mahadev Konar
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209232
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161154#comment-13161154 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #242 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/242/])
merge MAPREDUCE-3443 from trunk. Contributed by Mahadev Konar
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209232
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161142#comment-13161142 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1425 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1425/])
MAPREDUCE-3443. JobClient and Job should function in the context of the UGI which created them. (Contributed by Mahadev Konar)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209231
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161612#comment-13161612 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #113 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/113/])
merge MAPREDUCE-3443 from trunk. Contributed by Mahadev Konar
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209232
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159523#comment-13159523 ]
Hadoop QA commented on MAPREDUCE-3443:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505527/MAPREDUCE-3443.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 12 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1363//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1363//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1363//console
This message is automatically generated.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3443:
-------------------------------------
Attachment: MAPREDUCE-3443.patch
Preliminary patch. This add a ugi to JobClient and uses the already existing ugi in JobContextImpl to make sure all the calls to clientprotocol api's are wrapped around doAs with the ugi of the user when the jobcleint/Job was created. Still testing on a secure cluster.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3443:
-------------------------------------
Status: Patch Available (was: Open)
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3443:
-------------------------------------
Status: Patch Available (was: Open)
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161166#comment-13161166 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1375 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1375/])
MAPREDUCE-3443. JobClient and Job should function in the context of the UGI which created them. (Contributed by Mahadev Konar)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209231
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161183#comment-13161183 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #252 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/252/])
merge MAPREDUCE-3443 from trunk. Contributed by Mahadev Konar
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209232
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159581#comment-13159581 ]
Mahadev konar commented on MAPREDUCE-3443:
------------------------------------------
I cant really think of unit testing this, since this is security related.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161583#comment-13161583 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #95 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/95/])
merge MAPREDUCE-3443 from trunk. Contributed by Mahadev Konar
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209232
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161599#comment-13161599 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #882 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/882/])
MAPREDUCE-3443. JobClient and Job should function in the context of the UGI which created them. (Contributed by Mahadev Konar)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209231
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3443:
-------------------------------------
Attachment: MAPREDUCE-3443.patch
Addressed Sid's comments.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159907#comment-13159907 ]
Siddharth Seth commented on MAPREDUCE-3443:
-------------------------------------------
Spent some time understanding what the issue was. The patch in it's current state looks good mostly. Couple of possible changes
- job = clientUgi.doAs() is repeated all over JobClient. That could move into a separate function.
- JobClient.jobsToComplete / getAllJobs - needs to be wrapped in a ugi.doAs
In 0.20, things were a little simpler - since the connection to the JobTracker was being established by the JobClient itself - using the ugi at JobClient creation time.
That may be an alternate for Yarn as well - YARNRunner, ClientServiceDelegate and ResourceManagerDelegate would need to keep an instance of the ugi they are created with - and use this for each RPC proxy creation.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161149#comment-13161149 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #239 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/239/])
merge MAPREDUCE-3443 from trunk. Contributed by Mahadev Konar
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209232
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160614#comment-13160614 ]
Hadoop QA commented on MAPREDUCE-3443:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505707/MAPREDUCE-3443.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 1827 javac compiler warnings (more than the trunk's current 1826 warnings).
-1 findbugs. The patch appears to introduce 12 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1376//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1376//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1376//console
This message is automatically generated.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3443:
-------------------------------------
Attachment: MAPREDUCE-3443.patch
Updated patch with some bug fixes.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3443:
-------------------------------------
Attachment: MAPREDUCE-3443.patch
Fixed the bug that John pointed out. Thanks a lot John, you've been great help in testing and debugging this.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "John George (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159422#comment-13159422 ]
John George commented on MAPREDUCE-3443:
----------------------------------------
Tested the patch with oozie on a cluster. The patch seemed to work as long as in JobClient.submitJob(), when we do a Job.getInstance(), we do it with the conf passed in by the user - Job.getInstance(conf). After that the oozie web UI displayed 'success' and the job passed.
Good job Mahadev.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155588#comment-13155588 ]
Mahadev konar commented on MAPREDUCE-3443:
------------------------------------------
This still needs some testing with Oozie. [~johnvijoe] has graciously volunteered to help with the testing.
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3443) Oozie jobs are running as oozie
user even though they create the jobclient as doAs.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161143#comment-13161143 ]
Hudson commented on MAPREDUCE-3443:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1351 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1351/])
MAPREDUCE-3443. JobClient and Job should function in the context of the UGI which created them. (Contributed by Mahadev Konar)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1209231
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
> Oozie jobs are running as oozie user even though they create the jobclient as doAs.
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch, MAPREDUCE-3443.patch
>
>
> Oozie is having issues with job submission, since it does the following:
> {code}
> doAs(userwhosubmittedjob) {
> jobclient = new JobClient(jobconf);
> }
> jobclient.submitjob()
> {code}
> In 0.20.2** this works because the JT proxy is created as soon as we call new JobClient(). But in 0.23 this is no longer true since the client has to talk to multiple servers (AM/RM/JHS). To keep this behavior we will have to store the ugi in new JobClient() and make sure all the calls are run with a doAs() inside the jobclient.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira