You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by praveenesh kumar <pr...@gmail.com> on 2012/04/02 14:15:14 UTC
How can I configure oozie to submit different workflows from
different users ?
Hi all,
I want to use oozie to submit different workflows from different users.
These users are able to submit hadoop jobs.
I am using hadoop 0.20.205 and oozie 3.1.3
I have a hadoop user as a oozie-user
I have set the following things :
conf/oozie-site.xml :
< property >
< name >oozie.services.ext< /name >
< value >org.apache.oozie.service.HadoopAccessorService
< /value >
< description >
To add/replace services defined in 'oozie.services' with custom
implementations.Class names must be separated by commas.
< /description >
< /property >
conf/core-site.xml
< property>
< name>hadoop.proxyuser.hadoop.hosts< /name>
< value>*< / value>
< /property>
< property>
< name>hadoop.proxyuser.hadoop.groups< /name>
< value>*< /value>
< /property>
When I am submitting jobs as a hadoop user, I am able to run it properly.
But when I am able to submit the same work flow from a different user, who
can submit the simple MR jobs to my hadoop cluster, I am getting the
following error:
JA009: java.io.IOException: java.io.IOException: The username kumar
obtained from the conf doesn't match the username hadoop the user
authenticated as at
org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
Caused by: java.io.IOException: The username kumar obtained from the conf
doesn't match the username hadoop the user authenticated as
at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
... 11 more
Re: How can I configure oozie to submit different workflows from
different users ?
Posted by praveenesh kumar <pr...@gmail.com>.
Is this a problem of proxy setting ? because after specifying the group
name also, I am not able to run it. Its still giving me the same error.
Thanks,
Praveenesh
On Mon, Apr 2, 2012 at 6:05 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> multiple value are comma separated. keep in mind that valid values for
> proxyuser groups, as the property name states are GROUPS, not USERS.
>
> thxs.
>
> Alejandro
>
> On Mon, Apr 2, 2012 at 2:27 PM, praveenesh kumar <praveenesh@gmail.com
> >wrote:
>
> > How can I specify multiple users /groups for proxy user setting ?
> > Can I give comma separated values in these settings ?
> >
> > Thanks,
> > Praveenesh
> >
> > On Mon, Apr 2, 2012 at 5:52 PM, Alejandro Abdelnur <tucu@cloudera.com
> > >wrote:
> >
> > > Praveenesh,
> > >
> > > If I'm not mistaken 0.20.205 does not support wildcards for the
> proxyuser
> > > (hosts/groups) settings. You have to use explicit hosts/groups.
> > >
> > > Thxs.
> > >
> > > Alejandro
> > > PS: please follow up this thread in the
> oozie-users@incubator.apache.org
> > >
> > > On Mon, Apr 2, 2012 at 2:15 PM, praveenesh kumar <praveenesh@gmail.com
> > > >wrote:
> > >
> > > > Hi all,
> > > >
> > > > I want to use oozie to submit different workflows from different
> users.
> > > > These users are able to submit hadoop jobs.
> > > > I am using hadoop 0.20.205 and oozie 3.1.3
> > > > I have a hadoop user as a oozie-user
> > > >
> > > > I have set the following things :
> > > >
> > > > conf/oozie-site.xml :
> > > >
> > > > < property >
> > > > < name >oozie.services.ext< /name >
> > > > < value >org.apache.oozie.service.HadoopAccessorService
> > > > < /value >
> > > > < description >
> > > > To add/replace services defined in 'oozie.services' with custom
> > > > implementations.Class names must be separated by commas.
> > > > < /description >
> > > > < /property >
> > > >
> > > > conf/core-site.xml
> > > > < property>
> > > > < name>hadoop.proxyuser.hadoop.hosts< /name>
> > > > < value>*< / value>
> > > > < /property>
> > > > < property>
> > > > < name>hadoop.proxyuser.hadoop.groups< /name>
> > > > < value>*< /value>
> > > > < /property>
> > > >
> > > > When I am submitting jobs as a hadoop user, I am able to run it
> > properly.
> > > > But when I am able to submit the same work flow from a different
> user,
> > > who
> > > > can submit the simple MR jobs to my hadoop cluster, I am getting the
> > > > following error:
> > > >
> > > > JA009: java.io.IOException: java.io.IOException: The username kumar
> > > > obtained from the conf doesn't match the username hadoop the user
> > > > authenticated as at
> > > > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > at
> > > >
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > >
> > > > at
> > > >
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > >
> > > > at java.lang.reflect.Method.invoke(Method.java:597)
> > > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> > > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> > > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> > > > at java.security.AccessController.doPrivileged(Native Method)
> > > > at javax.security.auth.Subject.doAs(Subject.java:396)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> > > >
> > > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> > > >
> > > > Caused by: java.io.IOException: The username kumar obtained from the
> > conf
> > > > doesn't match the username hadoop the user authenticated as
> > > > at
> > org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
> > > > at
> org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
> > > > ... 11 more
> > > >
> > >
> >
>
Re: How can I configure oozie to submit different workflows from
different users ?
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
multiple value are comma separated. keep in mind that valid values for
proxyuser groups, as the property name states are GROUPS, not USERS.
thxs.
Alejandro
On Mon, Apr 2, 2012 at 2:27 PM, praveenesh kumar <pr...@gmail.com>wrote:
> How can I specify multiple users /groups for proxy user setting ?
> Can I give comma separated values in these settings ?
>
> Thanks,
> Praveenesh
>
> On Mon, Apr 2, 2012 at 5:52 PM, Alejandro Abdelnur <tucu@cloudera.com
> >wrote:
>
> > Praveenesh,
> >
> > If I'm not mistaken 0.20.205 does not support wildcards for the proxyuser
> > (hosts/groups) settings. You have to use explicit hosts/groups.
> >
> > Thxs.
> >
> > Alejandro
> > PS: please follow up this thread in the oozie-users@incubator.apache.org
> >
> > On Mon, Apr 2, 2012 at 2:15 PM, praveenesh kumar <praveenesh@gmail.com
> > >wrote:
> >
> > > Hi all,
> > >
> > > I want to use oozie to submit different workflows from different users.
> > > These users are able to submit hadoop jobs.
> > > I am using hadoop 0.20.205 and oozie 3.1.3
> > > I have a hadoop user as a oozie-user
> > >
> > > I have set the following things :
> > >
> > > conf/oozie-site.xml :
> > >
> > > < property >
> > > < name >oozie.services.ext< /name >
> > > < value >org.apache.oozie.service.HadoopAccessorService
> > > < /value >
> > > < description >
> > > To add/replace services defined in 'oozie.services' with custom
> > > implementations.Class names must be separated by commas.
> > > < /description >
> > > < /property >
> > >
> > > conf/core-site.xml
> > > < property>
> > > < name>hadoop.proxyuser.hadoop.hosts< /name>
> > > < value>*< / value>
> > > < /property>
> > > < property>
> > > < name>hadoop.proxyuser.hadoop.groups< /name>
> > > < value>*< /value>
> > > < /property>
> > >
> > > When I am submitting jobs as a hadoop user, I am able to run it
> properly.
> > > But when I am able to submit the same work flow from a different user,
> > who
> > > can submit the simple MR jobs to my hadoop cluster, I am getting the
> > > following error:
> > >
> > > JA009: java.io.IOException: java.io.IOException: The username kumar
> > > obtained from the conf doesn't match the username hadoop the user
> > > authenticated as at
> > > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > >
> > > at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:597)
> > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at javax.security.auth.Subject.doAs(Subject.java:396)
> > > at
> > >
> > >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> > >
> > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> > >
> > > Caused by: java.io.IOException: The username kumar obtained from the
> conf
> > > doesn't match the username hadoop the user authenticated as
> > > at
> org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
> > > at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
> > > ... 11 more
> > >
> >
>
Re: How can I configure oozie to submit different workflows from
different users ?
Posted by praveenesh kumar <pr...@gmail.com>.
How can I specify multiple users /groups for proxy user setting ?
Can I give comma separated values in these settings ?
Thanks,
Praveenesh
On Mon, Apr 2, 2012 at 5:52 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> Praveenesh,
>
> If I'm not mistaken 0.20.205 does not support wildcards for the proxyuser
> (hosts/groups) settings. You have to use explicit hosts/groups.
>
> Thxs.
>
> Alejandro
> PS: please follow up this thread in the oozie-users@incubator.apache.org
>
> On Mon, Apr 2, 2012 at 2:15 PM, praveenesh kumar <praveenesh@gmail.com
> >wrote:
>
> > Hi all,
> >
> > I want to use oozie to submit different workflows from different users.
> > These users are able to submit hadoop jobs.
> > I am using hadoop 0.20.205 and oozie 3.1.3
> > I have a hadoop user as a oozie-user
> >
> > I have set the following things :
> >
> > conf/oozie-site.xml :
> >
> > < property >
> > < name >oozie.services.ext< /name >
> > < value >org.apache.oozie.service.HadoopAccessorService
> > < /value >
> > < description >
> > To add/replace services defined in 'oozie.services' with custom
> > implementations.Class names must be separated by commas.
> > < /description >
> > < /property >
> >
> > conf/core-site.xml
> > < property>
> > < name>hadoop.proxyuser.hadoop.hosts< /name>
> > < value>*< / value>
> > < /property>
> > < property>
> > < name>hadoop.proxyuser.hadoop.groups< /name>
> > < value>*< /value>
> > < /property>
> >
> > When I am submitting jobs as a hadoop user, I am able to run it properly.
> > But when I am able to submit the same work flow from a different user,
> who
> > can submit the simple MR jobs to my hadoop cluster, I am getting the
> > following error:
> >
> > JA009: java.io.IOException: java.io.IOException: The username kumar
> > obtained from the conf doesn't match the username hadoop the user
> > authenticated as at
> > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:396)
> > at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> >
> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> >
> > Caused by: java.io.IOException: The username kumar obtained from the conf
> > doesn't match the username hadoop the user authenticated as
> > at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
> > at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
> > ... 11 more
> >
>
Re: How can I configure oozie to submit different workflows from
different users ?
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Praveenesh,
If I'm not mistaken 0.20.205 does not support wildcards for the proxyuser
(hosts/groups) settings. You have to use explicit hosts/groups.
Thxs.
Alejandro
PS: please follow up this thread in the oozie-users@incubator.apache.org
On Mon, Apr 2, 2012 at 2:15 PM, praveenesh kumar <pr...@gmail.com>wrote:
> Hi all,
>
> I want to use oozie to submit different workflows from different users.
> These users are able to submit hadoop jobs.
> I am using hadoop 0.20.205 and oozie 3.1.3
> I have a hadoop user as a oozie-user
>
> I have set the following things :
>
> conf/oozie-site.xml :
>
> < property >
> < name >oozie.services.ext< /name >
> < value >org.apache.oozie.service.HadoopAccessorService
> < /value >
> < description >
> To add/replace services defined in 'oozie.services' with custom
> implementations.Class names must be separated by commas.
> < /description >
> < /property >
>
> conf/core-site.xml
> < property>
> < name>hadoop.proxyuser.hadoop.hosts< /name>
> < value>*< / value>
> < /property>
> < property>
> < name>hadoop.proxyuser.hadoop.groups< /name>
> < value>*< /value>
> < /property>
>
> When I am submitting jobs as a hadoop user, I am able to run it properly.
> But when I am able to submit the same work flow from a different user, who
> can submit the simple MR jobs to my hadoop cluster, I am getting the
> following error:
>
> JA009: java.io.IOException: java.io.IOException: The username kumar
> obtained from the conf doesn't match the username hadoop the user
> authenticated as at
> org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> Caused by: java.io.IOException: The username kumar obtained from the conf
> doesn't match the username hadoop the user authenticated as
> at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
> at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
> ... 11 more
>
Re: Fwd: How can I configure oozie to submit different workflows from
different users ?
Posted by praveenesh kumar <pr...@gmail.com>.
So I figured out the issue.
I was using wild cards in proxy settings. I think wild cards are not
allowed in hadoop 0.20.205. My bad, sorry.
Anyways giving the right values, made things work for me.
Even with "KerberosHadoopAccessorService", its working fine now :-).
Regards,
Praveenesh
On Wed, Apr 4, 2012 at 12:07 PM, praveenesh kumar <pr...@gmail.com>wrote:
> Okay so I added the following entries in oozie-site.xml :
>
> <property>
> <name>oozie.services</name>
> <value>
> org.apache.oozie.service.SchedulerService,
> org.apache.oozie.service.InstrumentationService,
> org.apache.oozie.service.CallableQueueService,
> org.apache.oozie.service.UUIDService,
> org.apache.oozie.service.ELService,
> org.apache.oozie.service.AuthorizationService,
> org.apache.oozie.service.HadoopAccessorService,
> org.apache.oozie.service.MemoryLocksService,
> org.apache.oozie.service.DagXLogInfoService,
> org.apache.oozie.service.SchemaService,
> org.apache.oozie.service.LiteWorkflowAppService,
> org.apache.oozie.service.JPAService,
> org.apache.oozie.service.StoreService,
> org.apache.oozie.service.CoordinatorStoreService,
> org.apache.oozie.service.SLAStoreService,
> org.apache.oozie.service.DBLiteWorkflowStoreService,
> org.apache.oozie.service.CallbackService,
> org.apache.oozie.service.ActionService,
> org.apache.oozie.service.ActionCheckerService,
> org.apache.oozie.service.RecoveryService,
> org.apache.oozie.service.PurgeService,
> org.apache.oozie.service.CoordinatorEngineService,
> org.apache.oozie.service.BundleEngineService,
> org.apache.oozie.service.DagEngineService,
> org.apache.oozie.service.CoordMaterializeTriggerService,
> org.apache.oozie.service.StatusTransitService,
> org.apache.oozie.service.PauseTransitService
> </value>
> <description>
> All services to be created and managed by Oozie Services
> singleton.
>
> Class names must be separated by commas.
> </description>
> </property>
>
>
> Following is the oozie.log after setting the above parameters, showing
> that oozie is not using "KerberosHadoopAccessorService" now:
>
>
> *******************************************************************************
> STARTUP MSG: Oozie BUILD_VERSION [3.1.3-incubating] compiled by [kamrul]
> on [2012.02.29-18:52:47GMT]
> STARTUP MSG: revision [unavailable]@[unavailable]
>
> *******************************************************************************
> 2012-04-04 01:12:44,543 INFO XLogService:539 - Log4j configuration file
> [oozie-log4j.properties]
> 2012-04-04 01:12:44,544 INFO XLogService:539 - Log4j configuration file
> loaded from [/usr/local/hadoop/oozie/conf]
> 2012-04-04 01:12:44,544 INFO XLogService:539 - Log4j reload interval [10
> sec]
> 2012-04-04 01:12:44,551 INFO ConfigurationService:539 - USER[-] GROUP[-]
> Oozie home dir [/usr/local/hadoop/oozie]
> 2012-04-04 01:12:44,551 INFO ConfigurationService:539 - USER[-] GROUP[-]
> Oozie conf dir [/usr/local/hadoop/oozie/conf]
> 2012-04-04 01:12:44,552 INFO ConfigurationService:539 - USER[-] GROUP[-]
> Oozie conf file [oozie-site.xml]
> 2012-04-04 01:12:44,573 INFO ConfigurationService:539 - USER[-] GROUP[-]
> Configuration change via System Property, [oozie.base.url]=[
> http://lxe9700:11000/oozie]
> 2012-04-04 01:12:44,575 WARN Services:542 - USER[-] GROUP[-] System ID
> [oozie-hado] exceeds maximum length [10], trimming
> 2012-04-04 01:12:44,576 INFO Services:539 - USER[-] GROUP[-] Exiting null
> Entering NORMAL
> 2012-04-04 01:12:44,577 INFO Services:539 - USER[-] GROUP[-] Initialized
> runtime directory
> [/usr/local/hadoop/oozie/oozie-server/temp/oozie-hado4823910154936740088.dir]
> 2012-04-04 01:12:44,639 WARN AuthorizationService:542 - USER[-] GROUP[-]
> Oozie running with security disabled
> 2012-04-04 01:12:44,639 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
> JOB_TRACKER_WHITELIST :[ ], Total entries :0
> 2012-04-04 01:12:44,640 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
> NAME_NODE_WHITELIST :[ ], Total entries :0
> 2012-04-04 01:12:47,010 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
> APP[-] JOB[-] ACTION[-] All entities initialized
> 2012-04-04 01:12:47,011 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
> APP[-] JOB[-] ACTION[-] JPA configuration:
> DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/usr/local/hadoop/oozie/data/oozie-db;create=true,Username=sa,Password=,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false
> 2012-04-04 01:12:47,056 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] JOB_TRACKER_WHITELIST :[ ], Total entries
> :0
> 2012-04-04 01:12:47,057 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] NAME_NODE_WHITELIST :[ ], Total entries :0
> 2012-04-04 01:12:47,061 INFO Services:539 - Initialized
> 2012-04-04 01:12:47,064 INFO Services:539 - Running with JARs for Hadoop
> version [0.20.205.0]
> 2012-04-04 01:12:47,064 INFO Services:539 - Oozie System ID [oozie-hado]
> started!
> 2012-04-04 01:12:57,060 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired
> lock for [org.apache.oozie.service.StatusTransitService]
> 2012-04-04 01:12:57,061 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
> coordinator status service first instance
> 2012-04-04 01:12:57,077 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
> 2012-04-04 01:12:57,081 INFO PauseTransitService:539 - USER[-] GROUP[-]
> Acquired lock for [org.apache.oozie.service.PauseTransitService]
> 2012-04-04 01:12:57,083 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
> 2012-04-04 01:12:57,083 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
> 2012-04-04 01:12:57,274 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command [purge] key
> [null]
> 2012-04-04 01:12:57,274 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null]
> 2012-04-04 01:12:57,275 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Workflow-Purge Attempting to purge
> Jobs older than [30] days.
> 2012-04-04 01:12:57,275 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Workflow-Purge no workflow job to be
> deleted
> 2012-04-04 01:12:57,288 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
> [coord_purge] key [null]
> 2012-04-04 01:12:57,288 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [coord_purge] key [null]
> 2012-04-04 01:12:57,288 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Coord-Purge to purge Jobs older
> than [7] days.
> 2012-04-04 01:12:57,288 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
> bundle status service first instance
> 2012-04-04 01:12:57,288 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Coord-Purge no Coord job to be
> deleted
> 2012-04-04 01:12:57,288 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
> [bundle_purge] key [null]
> 2012-04-04 01:12:57,289 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [bundle_purge] key [null]
> 2012-04-04 01:12:57,289 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Bundle-Purge to purge Jobs older
> than [7] days.
> 2012-04-04 01:12:57,290 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Bundle-Purge no Bundle job to be
> deleted
> 2012-04-04 01:12:57,295 DEBUG
> CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:545 -
> USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
> CoordMaterializeTriggerService - Curr Date= Wed Apr 04 01:17:57 CDT 2012,
> Num jobs to materialize = 0
> 2012-04-04 01:12:57,299 DEBUG ActionCheckerService$ActionCheckRunnable:545
> - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for
> potential checking
> 2012-04-04 01:12:57,324 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released
> lock for [org.apache.oozie.service.StatusTransitService]
> 2012-04-04 01:12:57,341 INFO PauseTransitService:539 - USER[-] GROUP[-]
> Released lock for [org.apache.oozie.service.PauseTransitService]
> 2012-04-04 01:12:57,353 DEBUG RecoveryService$RecoveryRunnable:545 -
> USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0,
> COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential
> recovery
> 2012-04-04 01:13:57,308 DEBUG ActionCheckerService$ActionCheckRunnable:545
> - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for
> potential checking
> 2012-04-04 01:13:57,325 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired
> lock for [org.apache.oozie.service.StatusTransitService]
> 2012-04-04 01:13:57,329 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
> coordinator status service from last instance time = 2012-04-04T06:12Z
> 2012-04-04 01:13:57,340 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
> bundle status service from last instance time = 2012-04-04T06:12Z
> 2012-04-04 01:13:57,341 INFO PauseTransitService:539 - USER[-] GROUP[-]
> Acquired lock for [org.apache.oozie.service.PauseTransitService]
> 2012-04-04 01:13:57,347 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released
> lock for [org.apache.oozie.service.StatusTransitService]
> 2012-04-04 01:13:57,348 INFO PauseTransitService:539 - USER[-] GROUP[-]
> Released lock for [org.apache.oozie.service.PauseTransitService]
> 2012-04-04 01:13:57,358 DEBUG RecoveryService$RecoveryRunnable:545 -
> USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0,
> COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential
> recovery
>
>
> However, I am still getting the same error. No success :-(
>
> Regards,
> Praveenesh
>
>
>
>
>
> On Wed, Apr 4, 2012 at 12:19 AM, Mohammad Islam <mi...@yahoo.com>wrote:
>
>> Hi Praveenesh,
>> Sorry for the inconvenience.
>> Looks like your service ext to use HadoopAccessorService is not
>> effective. I knew there was an issue with this setting and it was resolved.
>>
>> Anyway, you could try the following a workaround:
>> Copy/Paste oozie.services property tag set from oozie-default.xml (could
>> be found in webapps/oozie/docs) to oozie-site.xml. Replace
>> org.apache.oozie.service.KerberosHadoopAccessorService with
>> HadoopAccessorService. Restart oozie. Make sure there is
>> no KerberosHadoopAccessorService in oozie log.
>> Please let us know how it goes.
>>
>> Regards,
>> Mohammad
>>
>>
>> ________________________________
>> From: praveenesh kumar <pr...@gmail.com>
>> To: oozie-users@incubator.apache.org; Mohammad Islam <mi...@yahoo.com>
>> Sent: Tuesday, April 3, 2012 2:23 AM
>> Subject: Re: Fwd: How can I configure oozie to submit different workflows
>> from different users ?
>>
>>
>> Following is the oozie.log file output after starting oozie:
>>
>>
>>
>> *******************************************************************************
>> STARTUP MSG: Oozie BUILD_VERSION [3.1.3-incubating] compiled by
>> [kamrul] on [2012.02.29-18:52:47GMT]
>> STARTUP MSG: revision [unavailable]@[unavailable]
>>
>> *******************************************************************************
>> 2012-04-03 04:18:59,026 INFO XLogService:539 - Log4j configuration file
>> [oozie-log4j.properties]
>> 2012-04-03 04:18:59,026 INFO XLogService:539 - Log4j configuration file
>> loaded from [/usr/local/hadoop/oozie/conf]
>> 2012-04-03 04:18:59,027 INFO XLogService:539 - Log4j reload interval [10
>> sec]
>> 2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
>> Oozie home dir [/usr/local/hadoop/oozie]
>> 2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
>> Oozie conf dir [/usr/local/hadoop/oozie/conf]
>> 2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
>> Oozie conf file [oozie-site.xml]
>> 2012-04-03 04:18:59,056 INFO ConfigurationService:539 - USER[-] GROUP[-]
>> Configuration change via System Property, [oozie.base.url]=[
>> http://lxe9700:11000/oozie]
>> 2012-04-03 04:18:59,057 WARN Services:542 - USER[-] GROUP[-] System ID
>> [oozie-hado] exceeds maximum length [10], trimming
>> 2012-04-03 04:18:59,058 INFO Services:539 - USER[-] GROUP[-] Exiting
>> null Entering NORMAL
>> 2012-04-03 04:18:59,059 INFO Services:539 - USER[-] GROUP[-] Initialized
>> runtime directory
>> [/usr/local/hadoop/oozie/oozie-server/temp/oozie-hado8638076398752938513.dir]
>> 2012-04-03 04:18:59,122 WARN AuthorizationService:542 - USER[-] GROUP[-]
>> Oozie running with security disabled
>> 2012-04-03 04:18:59,122 INFO KerberosHadoopAccessorService:539 - USER[-]
>> GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0
>> 2012-04-03 04:18:59,122 INFO KerberosHadoopAccessorService:539 - USER[-]
>> GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0
>> 2012-04-03 04:18:59,123 INFO KerberosHadoopAccessorService:539 - USER[-]
>> GROUP[-] Oozie Kerberos Authentication [disabled]
>> 2012-04-03 04:19:02,590 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
>> APP[-] JOB[-] ACTION[-] All entities initialized
>> 2012-04-03 04:19:02,591 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
>> APP[-] JOB[-] ACTION[-] JPA configuration:
>> DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/usr/local/hadoop/oozie/data/oozie-db;create=true,Username=sa,Password=,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false
>> 2012-04-03 04:19:02,633 INFO HadoopAccessorService:539 - USER[-]
>> GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JOB_TRACKER_WHITELIST :[ ], Total
>> entries :0
>> 2012-04-03 04:19:02,634 INFO HadoopAccessorService:539 - USER[-]
>> GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] NAME_NODE_WHITELIST :[ ], Total
>> entries :0
>> 2012-04-03 04:19:02,638 INFO Services:539 - Initialized
>> 2012-04-03 04:19:02,641 INFO Services:539 - Running with JARs for Hadoop
>> version [0.20.205.0]
>> 2012-04-03 04:19:02,641 INFO Services:539 - Oozie System ID [oozie-hado]
>> started!
>> 2012-04-03 04:19:12,645 INFO
>> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired
>> lock for [org.apache.oozie.service.StatusTransitService]
>> 2012-04-03 04:19:12,645 INFO
>> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
>> coordinator status service first instance
>> 2012-04-03 04:19:12,653 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
>> 2012-04-03 04:19:12,663 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
>> 2012-04-03 04:19:12,664 INFO PauseTransitService:539 - USER[-] GROUP[-]
>> Acquired lock for [org.apache.oozie.service.PauseTransitService]
>> 2012-04-03 04:19:12,665 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
>> 2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command [purge] key
>> [null]
>> 2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null]
>> 2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Workflow-Purge Attempting to purge
>> Jobs older than [30] days.
>> 2012-04-03 04:19:12,883 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Workflow-Purge no workflow job to be
>> deleted
>> 2012-04-03 04:19:12,887 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
>> [bundle_purge] key [null]
>> 2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [bundle_purge] key [null]
>> 2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Bundle-Purge to purge Jobs older
>> than [7] days.
>> 2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Bundle-Purge no Bundle job to be
>> deleted
>> 2012-04-03 04:19:12,891 INFO
>> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
>> bundle status service first instance
>> 2012-04-03 04:19:12,891 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
>> [coord_purge] key [null]
>> 2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [coord_purge] key [null]
>> 2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Coord-Purge to purge Jobs older
>> than [7] days.
>> 2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Coord-Purge no Coord job to be
>> deleted
>> 2012-04-03 04:19:12,902 DEBUG
>> CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:545 -
>> USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
>> CoordMaterializeTriggerService - Curr Date= Tue Apr 03 04:24:12 CDT 2012,
>> Num jobs to materialize = 0
>> 2012-04-03 04:19:12,914 DEBUG
>> ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-]
>> APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking
>> 2012-04-03 04:19:12,933 INFO
>> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released
>> lock for [org.apache.oozie.service.StatusTransitService]
>> 2012-04-03 04:19:12,949 INFO PauseTransitService:539 - USER[-] GROUP[-]
>> Released lock for [org.apache.oozie.service.PauseTransitService]
>> 2012-04-03 04:19:12,990 DEBUG RecoveryService$RecoveryRunnable:545 -
>> USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0,
>> COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential
>> recovery
>>
>>
>> I am newbie in oozie. Can you please help me out in resolving this issue ?
>>
>> Thanks,
>> Praveenesh
>>
>>
>> On Tue, Apr 3, 2012 at 2:32 PM, Mohammad Islam <mi...@yahoo.com>
>> wrote:
>>
>> Hi,
>> >Do you have kerberos on for hadoop?
>> >
>> >Can you please check what service
>> (HadoopAccessorService/KerberosHadoopAccessorService) is actually active?
>> >
>> >You can find from the oozie.log when the oozie server was instantiated.
>> Search for the above two service class name.
>> >
>> >Regards,
>> >Mohammad
>> >
>> >
>> >
>> >
>> >
>> >----- Original Message -----
>> >From: praveenesh kumar <pr...@gmail.com>
>> >To: oozie-users@incubator.apache.org
>> >Cc:
>> >Sent: Monday, April 2, 2012 6:19 AM
>> >Subject: Fwd: How can I configure oozie to submit different workflows
>> from different users ?
>> >
>> >---------- Forwarded message ----------
>> >From: praveenesh kumar <pr...@gmail.com>
>> >Date: Mon, Apr 2, 2012 at 5:45 PM
>> >Subject: How can I configure oozie to submit different workflows from
>> >different users ?
>> >To: common-user@hadoop.apache.org
>> >
>> >
>> >Hi all,
>> >
>> >I want to use oozie to submit different workflows from different users.
>> >These users are able to submit hadoop jobs.
>> >I am using hadoop 0.20.205 and oozie 3.1.3
>> >I have a hadoop user as a oozie-user
>> >
>> >I have set the following things :
>> >
>> >conf/oozie-site.xml :
>> >
>> >< property >
>> >< name >oozie.services.ext< /name >
>> >< value >org.apache.oozie.service.HadoopAccessorService
>> >< /value >
>> >< description >
>> >To add/replace services defined in 'oozie.services' with custom
>> >implementations.Class names must be separated by commas.
>> >< /description >
>> >< /property >
>> >
>> >conf/core-site.xml
>> >< property>
>> >< name>hadoop.proxyuser.hadoop.hosts< /name>
>> >< value>*< / value>
>> >< /property>
>> >< property>
>> >< name>hadoop.proxyuser.hadoop.groups< /name>
>> >< value>*< /value>
>> >< /property>
>> >
>> >When I am submitting jobs as a hadoop user, I am able to run it properly.
>> >But when I am able to submit the same work flow from a different user,
>> who
>> >can submit the simple MR jobs to my hadoop cluster, I am getting the
>> >following error:
>> >
>> >JA009: java.io.IOException: java.io.IOException: The username kumar
>> >obtained from the conf doesn't match the username hadoop the user
>> >authenticated as at
>> >org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
>> >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >at
>>
>> >sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >
>> >at
>>
>> >sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >
>> >at java.lang.reflect.Method.invoke(Method.java:597)
>> >at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> >at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> >at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> >at java.security.AccessController.doPrivileged(Native Method)
>> >at javax.security.auth.Subject.doAs(Subject.java:396)
>> >at
>>
>> >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>> >
>> >at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>> >
>> >Caused by: java.io.IOException: The username kumar obtained from the conf
>> >doesn't match the username hadoop the user authenticated as
>> >at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
>> >at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
>> >... 11 more
>> >
>> >
>>
>
>
Re: Fwd: How can I configure oozie to submit different workflows from
different users ?
Posted by praveenesh kumar <pr...@gmail.com>.
Okay so I added the following entries in oozie-site.xml :
<property>
<name>oozie.services</name>
<value>
org.apache.oozie.service.SchedulerService,
org.apache.oozie.service.InstrumentationService,
org.apache.oozie.service.CallableQueueService,
org.apache.oozie.service.UUIDService,
org.apache.oozie.service.ELService,
org.apache.oozie.service.AuthorizationService,
org.apache.oozie.service.HadoopAccessorService,
org.apache.oozie.service.MemoryLocksService,
org.apache.oozie.service.DagXLogInfoService,
org.apache.oozie.service.SchemaService,
org.apache.oozie.service.LiteWorkflowAppService,
org.apache.oozie.service.JPAService,
org.apache.oozie.service.StoreService,
org.apache.oozie.service.CoordinatorStoreService,
org.apache.oozie.service.SLAStoreService,
org.apache.oozie.service.DBLiteWorkflowStoreService,
org.apache.oozie.service.CallbackService,
org.apache.oozie.service.ActionService,
org.apache.oozie.service.ActionCheckerService,
org.apache.oozie.service.RecoveryService,
org.apache.oozie.service.PurgeService,
org.apache.oozie.service.CoordinatorEngineService,
org.apache.oozie.service.BundleEngineService,
org.apache.oozie.service.DagEngineService,
org.apache.oozie.service.CoordMaterializeTriggerService,
org.apache.oozie.service.StatusTransitService,
org.apache.oozie.service.PauseTransitService
</value>
<description>
All services to be created and managed by Oozie Services
singleton.
Class names must be separated by commas.
</description>
</property>
Following is the oozie.log after setting the above parameters, showing that
oozie is not using "KerberosHadoopAccessorService" now:
*******************************************************************************
STARTUP MSG: Oozie BUILD_VERSION [3.1.3-incubating] compiled by [kamrul]
on [2012.02.29-18:52:47GMT]
STARTUP MSG: revision [unavailable]@[unavailable]
*******************************************************************************
2012-04-04 01:12:44,543 INFO XLogService:539 - Log4j configuration file
[oozie-log4j.properties]
2012-04-04 01:12:44,544 INFO XLogService:539 - Log4j configuration file
loaded from [/usr/local/hadoop/oozie/conf]
2012-04-04 01:12:44,544 INFO XLogService:539 - Log4j reload interval [10
sec]
2012-04-04 01:12:44,551 INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie home dir [/usr/local/hadoop/oozie]
2012-04-04 01:12:44,551 INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie conf dir [/usr/local/hadoop/oozie/conf]
2012-04-04 01:12:44,552 INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie conf file [oozie-site.xml]
2012-04-04 01:12:44,573 INFO ConfigurationService:539 - USER[-] GROUP[-]
Configuration change via System Property, [oozie.base.url]=[
http://lxe9700:11000/oozie]
2012-04-04 01:12:44,575 WARN Services:542 - USER[-] GROUP[-] System ID
[oozie-hado] exceeds maximum length [10], trimming
2012-04-04 01:12:44,576 INFO Services:539 - USER[-] GROUP[-] Exiting null
Entering NORMAL
2012-04-04 01:12:44,577 INFO Services:539 - USER[-] GROUP[-] Initialized
runtime directory
[/usr/local/hadoop/oozie/oozie-server/temp/oozie-hado4823910154936740088.dir]
2012-04-04 01:12:44,639 WARN AuthorizationService:542 - USER[-] GROUP[-]
Oozie running with security disabled
2012-04-04 01:12:44,639 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
JOB_TRACKER_WHITELIST :[ ], Total entries :0
2012-04-04 01:12:44,640 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-04 01:12:47,010 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] All entities initialized
2012-04-04 01:12:47,011 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] JPA configuration:
DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/usr/local/hadoop/oozie/data/oozie-db;create=true,Username=sa,Password=,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false
2012-04-04 01:12:47,056 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] JOB_TRACKER_WHITELIST :[ ], Total entries
:0
2012-04-04 01:12:47,057 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-04 01:12:47,061 INFO Services:539 - Initialized
2012-04-04 01:12:47,064 INFO Services:539 - Running with JARs for Hadoop
version [0.20.205.0]
2012-04-04 01:12:47,064 INFO Services:539 - Oozie System ID [oozie-hado]
started!
2012-04-04 01:12:57,060 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired
lock for [org.apache.oozie.service.StatusTransitService]
2012-04-04 01:12:57,061 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
coordinator status service first instance
2012-04-04 01:12:57,077 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-04 01:12:57,081 INFO PauseTransitService:539 - USER[-] GROUP[-]
Acquired lock for [org.apache.oozie.service.PauseTransitService]
2012-04-04 01:12:57,083 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-04 01:12:57,083 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-04 01:12:57,274 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Precondition check for command [purge] key [null]
2012-04-04 01:12:57,274 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Execute command [purge] key [null]
2012-04-04 01:12:57,275 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] STARTED Workflow-Purge Attempting to purge Jobs
older than [30] days.
2012-04-04 01:12:57,275 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] ENDED Workflow-Purge no workflow job to be deleted
2012-04-04 01:12:57,288 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
[coord_purge] key [null]
2012-04-04 01:12:57,288 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [coord_purge] key [null]
2012-04-04 01:12:57,288 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Coord-Purge to purge Jobs older
than [7] days.
2012-04-04 01:12:57,288 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
bundle status service first instance
2012-04-04 01:12:57,288 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Coord-Purge no Coord job to be
deleted
2012-04-04 01:12:57,288 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
[bundle_purge] key [null]
2012-04-04 01:12:57,289 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [bundle_purge] key [null]
2012-04-04 01:12:57,289 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Bundle-Purge to purge Jobs older
than [7] days.
2012-04-04 01:12:57,290 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Bundle-Purge no Bundle job to be
deleted
2012-04-04 01:12:57,295 DEBUG
CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:545 -
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
CoordMaterializeTriggerService - Curr Date= Wed Apr 04 01:17:57 CDT 2012,
Num jobs to materialize = 0
2012-04-04 01:12:57,299 DEBUG ActionCheckerService$ActionCheckRunnable:545
- USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for
potential checking
2012-04-04 01:12:57,324 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released
lock for [org.apache.oozie.service.StatusTransitService]
2012-04-04 01:12:57,341 INFO PauseTransitService:539 - USER[-] GROUP[-]
Released lock for [org.apache.oozie.service.PauseTransitService]
2012-04-04 01:12:57,353 DEBUG RecoveryService$RecoveryRunnable:545 -
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0,
COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential
recovery
2012-04-04 01:13:57,308 DEBUG ActionCheckerService$ActionCheckRunnable:545
- USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for
potential checking
2012-04-04 01:13:57,325 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired
lock for [org.apache.oozie.service.StatusTransitService]
2012-04-04 01:13:57,329 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
coordinator status service from last instance time = 2012-04-04T06:12Z
2012-04-04 01:13:57,340 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
bundle status service from last instance time = 2012-04-04T06:12Z
2012-04-04 01:13:57,341 INFO PauseTransitService:539 - USER[-] GROUP[-]
Acquired lock for [org.apache.oozie.service.PauseTransitService]
2012-04-04 01:13:57,347 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released
lock for [org.apache.oozie.service.StatusTransitService]
2012-04-04 01:13:57,348 INFO PauseTransitService:539 - USER[-] GROUP[-]
Released lock for [org.apache.oozie.service.PauseTransitService]
2012-04-04 01:13:57,358 DEBUG RecoveryService$RecoveryRunnable:545 -
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0,
COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential
recovery
However, I am still getting the same error. No success :-(
Regards,
Praveenesh
On Wed, Apr 4, 2012 at 12:19 AM, Mohammad Islam <mi...@yahoo.com> wrote:
> Hi Praveenesh,
> Sorry for the inconvenience.
> Looks like your service ext to use HadoopAccessorService is not effective.
> I knew there was an issue with this setting and it was resolved.
>
> Anyway, you could try the following a workaround:
> Copy/Paste oozie.services property tag set from oozie-default.xml (could
> be found in webapps/oozie/docs) to oozie-site.xml. Replace
> org.apache.oozie.service.KerberosHadoopAccessorService with
> HadoopAccessorService. Restart oozie. Make sure there is
> no KerberosHadoopAccessorService in oozie log.
> Please let us know how it goes.
>
> Regards,
> Mohammad
>
>
> ________________________________
> From: praveenesh kumar <pr...@gmail.com>
> To: oozie-users@incubator.apache.org; Mohammad Islam <mi...@yahoo.com>
> Sent: Tuesday, April 3, 2012 2:23 AM
> Subject: Re: Fwd: How can I configure oozie to submit different workflows
> from different users ?
>
>
> Following is the oozie.log file output after starting oozie:
>
>
>
> *******************************************************************************
> STARTUP MSG: Oozie BUILD_VERSION [3.1.3-incubating] compiled by [kamrul]
> on [2012.02.29-18:52:47GMT]
> STARTUP MSG: revision [unavailable]@[unavailable]
>
> *******************************************************************************
> 2012-04-03 04:18:59,026 INFO XLogService:539 - Log4j configuration file
> [oozie-log4j.properties]
> 2012-04-03 04:18:59,026 INFO XLogService:539 - Log4j configuration file
> loaded from [/usr/local/hadoop/oozie/conf]
> 2012-04-03 04:18:59,027 INFO XLogService:539 - Log4j reload interval [10
> sec]
> 2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
> Oozie home dir [/usr/local/hadoop/oozie]
> 2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
> Oozie conf dir [/usr/local/hadoop/oozie/conf]
> 2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
> Oozie conf file [oozie-site.xml]
> 2012-04-03 04:18:59,056 INFO ConfigurationService:539 - USER[-] GROUP[-]
> Configuration change via System Property, [oozie.base.url]=[
> http://lxe9700:11000/oozie]
> 2012-04-03 04:18:59,057 WARN Services:542 - USER[-] GROUP[-] System ID
> [oozie-hado] exceeds maximum length [10], trimming
> 2012-04-03 04:18:59,058 INFO Services:539 - USER[-] GROUP[-] Exiting null
> Entering NORMAL
> 2012-04-03 04:18:59,059 INFO Services:539 - USER[-] GROUP[-] Initialized
> runtime directory
> [/usr/local/hadoop/oozie/oozie-server/temp/oozie-hado8638076398752938513.dir]
> 2012-04-03 04:18:59,122 WARN AuthorizationService:542 - USER[-] GROUP[-]
> Oozie running with security disabled
> 2012-04-03 04:18:59,122 INFO KerberosHadoopAccessorService:539 - USER[-]
> GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0
> 2012-04-03 04:18:59,122 INFO KerberosHadoopAccessorService:539 - USER[-]
> GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0
> 2012-04-03 04:18:59,123 INFO KerberosHadoopAccessorService:539 - USER[-]
> GROUP[-] Oozie Kerberos Authentication [disabled]
> 2012-04-03 04:19:02,590 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
> APP[-] JOB[-] ACTION[-] All entities initialized
> 2012-04-03 04:19:02,591 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
> APP[-] JOB[-] ACTION[-] JPA configuration:
> DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/usr/local/hadoop/oozie/data/oozie-db;create=true,Username=sa,Password=,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false
> 2012-04-03 04:19:02,633 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] JOB_TRACKER_WHITELIST :[ ], Total entries
> :0
> 2012-04-03 04:19:02,634 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] NAME_NODE_WHITELIST :[ ], Total entries :0
> 2012-04-03 04:19:02,638 INFO Services:539 - Initialized
> 2012-04-03 04:19:02,641 INFO Services:539 - Running with JARs for Hadoop
> version [0.20.205.0]
> 2012-04-03 04:19:02,641 INFO Services:539 - Oozie System ID [oozie-hado]
> started!
> 2012-04-03 04:19:12,645 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired
> lock for [org.apache.oozie.service.StatusTransitService]
> 2012-04-03 04:19:12,645 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
> coordinator status service first instance
> 2012-04-03 04:19:12,653 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
> 2012-04-03 04:19:12,663 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
> 2012-04-03 04:19:12,664 INFO PauseTransitService:539 - USER[-] GROUP[-]
> Acquired lock for [org.apache.oozie.service.PauseTransitService]
> 2012-04-03 04:19:12,665 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
> 2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command [purge] key
> [null]
> 2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null]
> 2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Workflow-Purge Attempting to purge
> Jobs older than [30] days.
> 2012-04-03 04:19:12,883 DEBUG PurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Workflow-Purge no workflow job to be
> deleted
> 2012-04-03 04:19:12,887 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
> [bundle_purge] key [null]
> 2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [bundle_purge] key [null]
> 2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Bundle-Purge to purge Jobs older
> than [7] days.
> 2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Bundle-Purge no Bundle job to be
> deleted
> 2012-04-03 04:19:12,891 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
> bundle status service first instance
> 2012-04-03 04:19:12,891 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
> [coord_purge] key [null]
> 2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [coord_purge] key [null]
> 2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Coord-Purge to purge Jobs older
> than [7] days.
> 2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Coord-Purge no Coord job to be
> deleted
> 2012-04-03 04:19:12,902 DEBUG
> CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:545 -
> USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
> CoordMaterializeTriggerService - Curr Date= Tue Apr 03 04:24:12 CDT 2012,
> Num jobs to materialize = 0
> 2012-04-03 04:19:12,914 DEBUG ActionCheckerService$ActionCheckRunnable:545
> - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for
> potential checking
> 2012-04-03 04:19:12,933 INFO
> StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released
> lock for [org.apache.oozie.service.StatusTransitService]
> 2012-04-03 04:19:12,949 INFO PauseTransitService:539 - USER[-] GROUP[-]
> Released lock for [org.apache.oozie.service.PauseTransitService]
> 2012-04-03 04:19:12,990 DEBUG RecoveryService$RecoveryRunnable:545 -
> USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0,
> COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential
> recovery
>
>
> I am newbie in oozie. Can you please help me out in resolving this issue ?
>
> Thanks,
> Praveenesh
>
>
> On Tue, Apr 3, 2012 at 2:32 PM, Mohammad Islam <mi...@yahoo.com> wrote:
>
> Hi,
> >Do you have kerberos on for hadoop?
> >
> >Can you please check what service
> (HadoopAccessorService/KerberosHadoopAccessorService) is actually active?
> >
> >You can find from the oozie.log when the oozie server was instantiated.
> Search for the above two service class name.
> >
> >Regards,
> >Mohammad
> >
> >
> >
> >
> >
> >----- Original Message -----
> >From: praveenesh kumar <pr...@gmail.com>
> >To: oozie-users@incubator.apache.org
> >Cc:
> >Sent: Monday, April 2, 2012 6:19 AM
> >Subject: Fwd: How can I configure oozie to submit different workflows
> from different users ?
> >
> >---------- Forwarded message ----------
> >From: praveenesh kumar <pr...@gmail.com>
> >Date: Mon, Apr 2, 2012 at 5:45 PM
> >Subject: How can I configure oozie to submit different workflows from
> >different users ?
> >To: common-user@hadoop.apache.org
> >
> >
> >Hi all,
> >
> >I want to use oozie to submit different workflows from different users.
> >These users are able to submit hadoop jobs.
> >I am using hadoop 0.20.205 and oozie 3.1.3
> >I have a hadoop user as a oozie-user
> >
> >I have set the following things :
> >
> >conf/oozie-site.xml :
> >
> >< property >
> >< name >oozie.services.ext< /name >
> >< value >org.apache.oozie.service.HadoopAccessorService
> >< /value >
> >< description >
> >To add/replace services defined in 'oozie.services' with custom
> >implementations.Class names must be separated by commas.
> >< /description >
> >< /property >
> >
> >conf/core-site.xml
> >< property>
> >< name>hadoop.proxyuser.hadoop.hosts< /name>
> >< value>*< / value>
> >< /property>
> >< property>
> >< name>hadoop.proxyuser.hadoop.groups< /name>
> >< value>*< /value>
> >< /property>
> >
> >When I am submitting jobs as a hadoop user, I am able to run it properly.
> >But when I am able to submit the same work flow from a different user,
> who
> >can submit the simple MR jobs to my hadoop cluster, I am getting the
> >following error:
> >
> >JA009: java.io.IOException: java.io.IOException: The username kumar
> >obtained from the conf doesn't match the username hadoop the user
> >authenticated as at
> >org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
> >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >at
>
> >sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >
> >at
>
> >sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >
> >at java.lang.reflect.Method.invoke(Method.java:597)
> >at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> >at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> >at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> >at java.security.AccessController.doPrivileged(Native Method)
> >at javax.security.auth.Subject.doAs(Subject.java:396)
> >at
>
> >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> >
> >at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> >
> >Caused by: java.io.IOException: The username kumar obtained from the conf
> >doesn't match the username hadoop the user authenticated as
> >at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
> >at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
> >... 11 more
> >
> >
>
Re: Fwd: How can I configure oozie to submit different workflows from different users ?
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Praveenesh,
Sorry for the inconvenience.
Looks like your service ext to use HadoopAccessorService is not effective. I knew there was an issue with this setting and it was resolved.
Anyway, you could try the following a workaround:
Copy/Paste oozie.services property tag set from oozie-default.xml (could be found in webapps/oozie/docs) to oozie-site.xml. Replace org.apache.oozie.service.KerberosHadoopAccessorService with HadoopAccessorService. Restart oozie. Make sure there is no KerberosHadoopAccessorService in oozie log.
Please let us know how it goes.
Regards,
Mohammad
________________________________
From: praveenesh kumar <pr...@gmail.com>
To: oozie-users@incubator.apache.org; Mohammad Islam <mi...@yahoo.com>
Sent: Tuesday, April 3, 2012 2:23 AM
Subject: Re: Fwd: How can I configure oozie to submit different workflows from different users ?
Following is the oozie.log file output after starting oozie:
*******************************************************************************
STARTUP MSG: Oozie BUILD_VERSION [3.1.3-incubating] compiled by [kamrul] on [2012.02.29-18:52:47GMT]
STARTUP MSG: revision [unavailable]@[unavailable]
*******************************************************************************
2012-04-03 04:18:59,026 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties]
2012-04-03 04:18:59,026 INFO XLogService:539 - Log4j configuration file loaded from [/usr/local/hadoop/oozie/conf]
2012-04-03 04:18:59,027 INFO XLogService:539 - Log4j reload interval [10 sec]
2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie home dir [/usr/local/hadoop/oozie]
2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie conf dir [/usr/local/hadoop/oozie/conf]
2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie conf file [oozie-site.xml]
2012-04-03 04:18:59,056 INFO ConfigurationService:539 - USER[-] GROUP[-] Configuration change via System Property, [oozie.base.url]=[http://lxe9700:11000/oozie]
2012-04-03 04:18:59,057 WARN Services:542 - USER[-] GROUP[-] System ID [oozie-hado] exceeds maximum length [10], trimming
2012-04-03 04:18:59,058 INFO Services:539 - USER[-] GROUP[-] Exiting null Entering NORMAL
2012-04-03 04:18:59,059 INFO Services:539 - USER[-] GROUP[-] Initialized runtime directory [/usr/local/hadoop/oozie/oozie-server/temp/oozie-hado8638076398752938513.dir]
2012-04-03 04:18:59,122 WARN AuthorizationService:542 - USER[-] GROUP[-] Oozie running with security disabled
2012-04-03 04:18:59,122 INFO KerberosHadoopAccessorService:539 - USER[-] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0
2012-04-03 04:18:59,122 INFO KerberosHadoopAccessorService:539 - USER[-] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-03 04:18:59,123 INFO KerberosHadoopAccessorService:539 - USER[-] GROUP[-] Oozie Kerberos Authentication [disabled]
2012-04-03 04:19:02,590 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] All entities initialized
2012-04-03 04:19:02,591 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JPA configuration: DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/usr/local/hadoop/oozie/data/oozie-db;create=true,Username=sa,Password=,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false
2012-04-03 04:19:02,633 INFO HadoopAccessorService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0
2012-04-03 04:19:02,634 INFO HadoopAccessorService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-03 04:19:02,638 INFO Services:539 - Initialized
2012-04-03 04:19:02,641 INFO Services:539 - Running with JARs for Hadoop version [0.20.205.0]
2012-04-03 04:19:02,641 INFO Services:539 - Oozie System ID [oozie-hado] started!
2012-04-03 04:19:12,645 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService]
2012-04-03 04:19:12,645 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service first instance
2012-04-03 04:19:12,653 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,663 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,664 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService]
2012-04-03 04:19:12,665 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command [purge] key [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Workflow-Purge Attempting to purge Jobs older than [30] days.
2012-04-03 04:19:12,883 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Workflow-Purge no workflow job to be deleted
2012-04-03 04:19:12,887 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command [bundle_purge] key [null]
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [bundle_purge] key [null]
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Bundle-Purge to purge Jobs older than [7] days.
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Bundle-Purge no Bundle job to be deleted
2012-04-03 04:19:12,891 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service first instance
2012-04-03 04:19:12,891 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command [coord_purge] key [null]
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [coord_purge] key [null]
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Coord-Purge to purge Jobs older than [7] days.
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Coord-Purge no Coord job to be deleted
2012-04-03 04:19:12,902 DEBUG CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= Tue Apr 03 04:24:12 CDT 2012, Num jobs to materialize = 0
2012-04-03 04:19:12,914 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking
2012-04-03 04:19:12,933 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService]
2012-04-03 04:19:12,949 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService]
2012-04-03 04:19:12,990 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential recovery
I am newbie in oozie. Can you please help me out in resolving this issue ?
Thanks,
Praveenesh
On Tue, Apr 3, 2012 at 2:32 PM, Mohammad Islam <mi...@yahoo.com> wrote:
Hi,
>Do you have kerberos on for hadoop?
>
>Can you please check what service (HadoopAccessorService/KerberosHadoopAccessorService) is actually active?
>
>You can find from the oozie.log when the oozie server was instantiated. Search for the above two service class name.
>
>Regards,
>Mohammad
>
>
>
>
>
>----- Original Message -----
>From: praveenesh kumar <pr...@gmail.com>
>To: oozie-users@incubator.apache.org
>Cc:
>Sent: Monday, April 2, 2012 6:19 AM
>Subject: Fwd: How can I configure oozie to submit different workflows from different users ?
>
>---------- Forwarded message ----------
>From: praveenesh kumar <pr...@gmail.com>
>Date: Mon, Apr 2, 2012 at 5:45 PM
>Subject: How can I configure oozie to submit different workflows from
>different users ?
>To: common-user@hadoop.apache.org
>
>
>Hi all,
>
>I want to use oozie to submit different workflows from different users.
>These users are able to submit hadoop jobs.
>I am using hadoop 0.20.205 and oozie 3.1.3
>I have a hadoop user as a oozie-user
>
>I have set the following things :
>
>conf/oozie-site.xml :
>
>< property >
>< name >oozie.services.ext< /name >
>< value >org.apache.oozie.service.HadoopAccessorService
>< /value >
>< description >
>To add/replace services defined in 'oozie.services' with custom
>implementations.Class names must be separated by commas.
>< /description >
>< /property >
>
>conf/core-site.xml
>< property>
>< name>hadoop.proxyuser.hadoop.hosts< /name>
>< value>*< / value>
>< /property>
>< property>
>< name>hadoop.proxyuser.hadoop.groups< /name>
>< value>*< /value>
>< /property>
>
>When I am submitting jobs as a hadoop user, I am able to run it properly.
>But when I am able to submit the same work flow from a different user, who
>can submit the simple MR jobs to my hadoop cluster, I am getting the
>following error:
>
>JA009: java.io.IOException: java.io.IOException: The username kumar
>obtained from the conf doesn't match the username hadoop the user
>authenticated as at
>org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at
>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
>at
>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
>at java.lang.reflect.Method.invoke(Method.java:597)
>at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>at java.security.AccessController.doPrivileged(Native Method)
>at javax.security.auth.Subject.doAs(Subject.java:396)
>at
>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>
>at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
>Caused by: java.io.IOException: The username kumar obtained from the conf
>doesn't match the username hadoop the user authenticated as
>at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
>at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
>... 11 more
>
>
Re: Fwd: How can I configure oozie to submit different workflows from
different users ?
Posted by praveenesh kumar <pr...@gmail.com>.
Following is the oozie.log file output after starting oozie:
*******************************************************************************
STARTUP MSG: Oozie BUILD_VERSION [3.1.3-incubating] compiled by [kamrul]
on [2012.02.29-18:52:47GMT]
STARTUP MSG: revision [unavailable]@[unavailable]
*******************************************************************************
2012-04-03 04:18:59,026 INFO XLogService:539 - Log4j configuration file
[oozie-log4j.properties]
2012-04-03 04:18:59,026 INFO XLogService:539 - Log4j configuration file
loaded from [/usr/local/hadoop/oozie/conf]
2012-04-03 04:18:59,027 INFO XLogService:539 - Log4j reload interval [10
sec]
2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie home dir [/usr/local/hadoop/oozie]
2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie conf dir [/usr/local/hadoop/oozie/conf]
2012-04-03 04:18:59,034 INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie conf file [oozie-site.xml]
2012-04-03 04:18:59,056 INFO ConfigurationService:539 - USER[-] GROUP[-]
Configuration change via System Property, [oozie.base.url]=[
http://lxe9700:11000/oozie]
2012-04-03 04:18:59,057 WARN Services:542 - USER[-] GROUP[-] System ID
[oozie-hado] exceeds maximum length [10], trimming
2012-04-03 04:18:59,058 INFO Services:539 - USER[-] GROUP[-] Exiting null
Entering NORMAL
2012-04-03 04:18:59,059 INFO Services:539 - USER[-] GROUP[-] Initialized
runtime directory
[/usr/local/hadoop/oozie/oozie-server/temp/oozie-hado8638076398752938513.dir]
2012-04-03 04:18:59,122 WARN AuthorizationService:542 - USER[-] GROUP[-]
Oozie running with security disabled
2012-04-03 04:18:59,122 INFO KerberosHadoopAccessorService:539 - USER[-]
GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0
2012-04-03 04:18:59,122 INFO KerberosHadoopAccessorService:539 - USER[-]
GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-03 04:18:59,123 INFO KerberosHadoopAccessorService:539 - USER[-]
GROUP[-] Oozie Kerberos Authentication [disabled]
2012-04-03 04:19:02,590 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] All entities initialized
2012-04-03 04:19:02,591 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] JPA configuration:
DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/usr/local/hadoop/oozie/data/oozie-db;create=true,Username=sa,Password=,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false
2012-04-03 04:19:02,633 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] JOB_TRACKER_WHITELIST :[ ], Total entries
:0
2012-04-03 04:19:02,634 INFO HadoopAccessorService:539 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-03 04:19:02,638 INFO Services:539 - Initialized
2012-04-03 04:19:02,641 INFO Services:539 - Running with JARs for Hadoop
version [0.20.205.0]
2012-04-03 04:19:02,641 INFO Services:539 - Oozie System ID [oozie-hado]
started!
2012-04-03 04:19:12,645 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired
lock for [org.apache.oozie.service.StatusTransitService]
2012-04-03 04:19:12,645 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
coordinator status service first instance
2012-04-03 04:19:12,653 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,663 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,664 INFO PauseTransitService:539 - USER[-] GROUP[-]
Acquired lock for [org.apache.oozie.service.PauseTransitService]
2012-04-03 04:19:12,665 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Precondition check for command [purge] key [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Execute command [purge] key [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] STARTED Workflow-Purge Attempting to purge Jobs
older than [30] days.
2012-04-03 04:19:12,883 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] ENDED Workflow-Purge no workflow job to be deleted
2012-04-03 04:19:12,887 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
[bundle_purge] key [null]
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [bundle_purge] key [null]
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Bundle-Purge to purge Jobs older
than [7] days.
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Bundle-Purge no Bundle job to be
deleted
2012-04-03 04:19:12,891 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
bundle status service first instance
2012-04-03 04:19:12,891 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
[coord_purge] key [null]
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [coord_purge] key [null]
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Coord-Purge to purge Jobs older
than [7] days.
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Coord-Purge no Coord job to be
deleted
2012-04-03 04:19:12,902 DEBUG
CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:545 -
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
CoordMaterializeTriggerService - Curr Date= Tue Apr 03 04:24:12 CDT 2012,
Num jobs to materialize = 0
2012-04-03 04:19:12,914 DEBUG ActionCheckerService$ActionCheckRunnable:545
- USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for
potential checking
2012-04-03 04:19:12,933 INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released
lock for [org.apache.oozie.service.StatusTransitService]
2012-04-03 04:19:12,949 INFO PauseTransitService:539 - USER[-] GROUP[-]
Released lock for [org.apache.oozie.service.PauseTransitService]
2012-04-03 04:19:12,990 DEBUG RecoveryService$RecoveryRunnable:545 -
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0,
COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential
recovery
I am newbie in oozie. Can you please help me out in resolving this issue ?
Thanks,
Praveenesh
On Tue, Apr 3, 2012 at 2:32 PM, Mohammad Islam <mi...@yahoo.com> wrote:
> Hi,
> Do you have kerberos on for hadoop?
>
> Can you please check what service
> (HadoopAccessorService/KerberosHadoopAccessorService) is actually active?
>
> You can find from the oozie.log when the oozie server was instantiated.
> Search for the above two service class name.
>
> Regards,
> Mohammad
>
>
>
>
> ----- Original Message -----
> From: praveenesh kumar <pr...@gmail.com>
> To: oozie-users@incubator.apache.org
> Cc:
> Sent: Monday, April 2, 2012 6:19 AM
> Subject: Fwd: How can I configure oozie to submit different workflows from
> different users ?
>
> ---------- Forwarded message ----------
> From: praveenesh kumar <pr...@gmail.com>
> Date: Mon, Apr 2, 2012 at 5:45 PM
> Subject: How can I configure oozie to submit different workflows from
> different users ?
> To: common-user@hadoop.apache.org
>
>
> Hi all,
>
> I want to use oozie to submit different workflows from different users.
> These users are able to submit hadoop jobs.
> I am using hadoop 0.20.205 and oozie 3.1.3
> I have a hadoop user as a oozie-user
>
> I have set the following things :
>
> conf/oozie-site.xml :
>
> < property >
> < name >oozie.services.ext< /name >
> < value >org.apache.oozie.service.HadoopAccessorService
> < /value >
> < description >
> To add/replace services defined in 'oozie.services' with custom
> implementations.Class names must be separated by commas.
> < /description >
> < /property >
>
> conf/core-site.xml
> < property>
> < name>hadoop.proxyuser.hadoop.hosts< /name>
> < value>*< / value>
> < /property>
> < property>
> < name>hadoop.proxyuser.hadoop.groups< /name>
> < value>*< /value>
> < /property>
>
> When I am submitting jobs as a hadoop user, I am able to run it properly.
> But when I am able to submit the same work flow from a different user, who
> can submit the simple MR jobs to my hadoop cluster, I am getting the
> following error:
>
> JA009: java.io.IOException: java.io.IOException: The username kumar
> obtained from the conf doesn't match the username hadoop the user
> authenticated as at
> org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> Caused by: java.io.IOException: The username kumar obtained from the conf
> doesn't match the username hadoop the user authenticated as
> at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
> at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
> ... 11 more
>
>
Re: Fwd: How can I configure oozie to submit different workflows from different users ?
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi,
Do you have kerberos on for hadoop?
Can you please check what service (HadoopAccessorService/KerberosHadoopAccessorService) is actually active?
You can find from the oozie.log when the oozie server was instantiated. Search for the above two service class name.
Regards,
Mohammad
----- Original Message -----
From: praveenesh kumar <pr...@gmail.com>
To: oozie-users@incubator.apache.org
Cc:
Sent: Monday, April 2, 2012 6:19 AM
Subject: Fwd: How can I configure oozie to submit different workflows from different users ?
---------- Forwarded message ----------
From: praveenesh kumar <pr...@gmail.com>
Date: Mon, Apr 2, 2012 at 5:45 PM
Subject: How can I configure oozie to submit different workflows from
different users ?
To: common-user@hadoop.apache.org
Hi all,
I want to use oozie to submit different workflows from different users.
These users are able to submit hadoop jobs.
I am using hadoop 0.20.205 and oozie 3.1.3
I have a hadoop user as a oozie-user
I have set the following things :
conf/oozie-site.xml :
< property >
< name >oozie.services.ext< /name >
< value >org.apache.oozie.service.HadoopAccessorService
< /value >
< description >
To add/replace services defined in 'oozie.services' with custom
implementations.Class names must be separated by commas.
< /description >
< /property >
conf/core-site.xml
< property>
< name>hadoop.proxyuser.hadoop.hosts< /name>
< value>*< / value>
< /property>
< property>
< name>hadoop.proxyuser.hadoop.groups< /name>
< value>*< /value>
< /property>
When I am submitting jobs as a hadoop user, I am able to run it properly.
But when I am able to submit the same work flow from a different user, who
can submit the simple MR jobs to my hadoop cluster, I am getting the
following error:
JA009: java.io.IOException: java.io.IOException: The username kumar
obtained from the conf doesn't match the username hadoop the user
authenticated as at
org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
Caused by: java.io.IOException: The username kumar obtained from the conf
doesn't match the username hadoop the user authenticated as
at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
... 11 more
Fwd: How can I configure oozie to submit different workflows from
different users ?
Posted by praveenesh kumar <pr...@gmail.com>.
---------- Forwarded message ----------
From: praveenesh kumar <pr...@gmail.com>
Date: Mon, Apr 2, 2012 at 5:45 PM
Subject: How can I configure oozie to submit different workflows from
different users ?
To: common-user@hadoop.apache.org
Hi all,
I want to use oozie to submit different workflows from different users.
These users are able to submit hadoop jobs.
I am using hadoop 0.20.205 and oozie 3.1.3
I have a hadoop user as a oozie-user
I have set the following things :
conf/oozie-site.xml :
< property >
< name >oozie.services.ext< /name >
< value >org.apache.oozie.service.HadoopAccessorService
< /value >
< description >
To add/replace services defined in 'oozie.services' with custom
implementations.Class names must be separated by commas.
< /description >
< /property >
conf/core-site.xml
< property>
< name>hadoop.proxyuser.hadoop.hosts< /name>
< value>*< / value>
< /property>
< property>
< name>hadoop.proxyuser.hadoop.groups< /name>
< value>*< /value>
< /property>
When I am submitting jobs as a hadoop user, I am able to run it properly.
But when I am able to submit the same work flow from a different user, who
can submit the simple MR jobs to my hadoop cluster, I am getting the
following error:
JA009: java.io.IOException: java.io.IOException: The username kumar
obtained from the conf doesn't match the username hadoop the user
authenticated as at
org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
Caused by: java.io.IOException: The username kumar obtained from the conf
doesn't match the username hadoop the user authenticated as
at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
... 11 more