You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Boris Shulman <sh...@gmail.com> on 2012/07/19 09:01:28 UTC
Oozie Security/Impersonation issue
I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
*I put the following configuration in the core-site.xml (in both hadpp and
oozie conf):*
<property>
<name>hadoop.proxyuser.hadoop.groups</name>
<value>supergroup</value>
<description>Allow the superuser hadoop to impersonate any
members of any group</description>
</property>
<property>
<name>hadoop.proxyuser.hadoop.hosts</name>
<value>127.0.0.1</value>
<description>The superuser can connect only from
localhost to impersonate a user</description>
</property>
*Also I added the following in the oozie-site.xml *
<property>
<name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
<value>*</value>
<description>
</description>
</property>
<property>
<name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
<value>*</value>
<description>
</description>
When I try to submit oozie job I get the following error:
*In the oozie log:*
... 26 more
2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
http://localhost:11000/oozie/v1/jobs?action=start] error[E0902], E0902:
Exception occured: [org.apache.hadoop.ipc.RemoteException: User: hadoop is
not allowed to impersonate hadoop]
org.apache.oozie.servlet.XServletException: E0902: Exception occured:
[org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed to
impersonate hadoop]
at
org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
*in the NN log:*
2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 9000: readAndProcess threw exception
org.apache.hadoop.security.AccessControlException: Connection from
127.0.0.1:56413 for protocol org.apache.hadoop.hdfs.protocol.ClientProtocol
is unauthorized for user hadoop via hadoop. Count of bytes read: 0
org.apache.hadoop.security.AccessControlException: Connection from
127.0.0.1:56413 for protocol org.apache.hadoop.hdfs.protocol.ClientProtocol
is unauthorized for user hadoop via hadoop
at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
What do I miss here?
Regards,\
Boris.
Re: Oozie Security/Impersonation issue
Posted by Harsh J <ha...@cloudera.com>.
Good to know! Thanks for reverting back :)
On Thu, Jul 19, 2012 at 8:58 PM, Boris Shulman <sh...@gmail.com> wrote:
> Thanks, it worked. Putting the actual group name in the configuration
> helped.
>
> On Thu, Jul 19, 2012 at 2:44 PM, Harsh J <ha...@cloudera.com> wrote:
>
> > Boris,
> >
> > By default Hadoop tries to map groups/usernames from the NameNode's unix
> > list. You can see
> >
> >
> http://www.cloudera.com/blog/2012/03/authorization-and-authentication-in-hadoop/for
> > a brief but complete summary on how the system is.
> >
> > So unless hadoop is in one of the groups its been designated to allow, it
> > will not allow. Perhaps you can set the groups to *, to allow all groups,
> > or set to a group "hadoop" (or whatever group "hadoop" is in) to allow it
> > properly, and restart NN to check. Please let us know if that works.
> >
> > On Thu, Jul 19, 2012 at 4:56 PM, Boris Shulman <sh...@gmail.com>
> wrote:
> >
> > > 1) I did restart the NameNode.
> > > 2) Do you mean on the OS level? Than no, so do I need to specify OS
> level
> > > group in the config? (supergroup is the group that I see in hadoop fs
> -ls
> > > on the files..)
> > >
> > > On Thu, Jul 19, 2012 at 2:12 PM, Harsh J <ha...@cloudera.com> wrote:
> > >
> > > > Hey Boris,
> > > >
> > > > Did you restart the NameNode after making this change in the
> > > core-site.xml
> > > > of Hadoop?
> > > >
> > > > Also, is "hadoop" in the supergroup physically? That is, does "groups
> > > > hadoop" show supergroup in its output?
> > > >
> > > > On Thu, Jul 19, 2012 at 12:31 PM, Boris Shulman <sh...@gmail.com>
> > > > wrote:
> > > >
> > > > > I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
> > > > >
> > > > > *I put the following configuration in the core-site.xml (in both
> > hadpp
> > > > and
> > > > > oozie conf):*
> > > > >
> > > > > <property>
> > > > > <name>hadoop.proxyuser.hadoop.groups</name>
> > > > > <value>supergroup</value>
> > > > > <description>Allow the superuser hadoop to
> impersonate
> > > any
> > > > > members of any group</description>
> > > > > </property>
> > > > > <property>
> > > > > <name>hadoop.proxyuser.hadoop.hosts</name>
> > > > > <value>127.0.0.1</value>
> > > > > <description>The superuser can connect only from
> > > > > localhost to impersonate a user</description>
> > > > > </property>
> > > > >
> > > > >
> > > > >
> > > > > *Also I added the following in the oozie-site.xml *
> > > > >
> > > > > <property>
> > > > >
> > > > <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> > > > > <value>*</value>
> > > > > <description>
> > > > >
> > > > > </description>
> > > > > </property>
> > > > >
> > > > > <property>
> > > > >
> > > > <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> > > > > <value>*</value>
> > > > > <description>
> > > > >
> > > > > </description>
> > > > >
> > > > > When I try to submit oozie job I get the following error:
> > > > >
> > > > > *In the oozie log:*
> > > > >
> > > > > ... 26 more
> > > > > 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop]
> > GROUP[-]
> > > > > TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> > > > > http://localhost:11000/oozie/v1/jobs?action=start] error[E0902],
> > > E0902:
> > > > > Exception occured: [org.apache.hadoop.ipc.RemoteException: User:
> > hadoop
> > > > is
> > > > > not allowed to impersonate hadoop]
> > > > > org.apache.oozie.servlet.XServletException: E0902: Exception
> occured:
> > > > > [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed
> > to
> > > > > impersonate hadoop]
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> > > > > at
> > > >
> > org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> > > > >
> > > > >
> > > > >
> > > > > *in the NN log:*
> > > > >
> > > > > 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC
> Server
> > > > > listener on 9000: readAndProcess threw exception
> > > > > org.apache.hadoop.security.AccessControlException: Connection from
> > > > > 127.0.0.1:56413 for protocol
> > > > > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > > > > is unauthorized for user hadoop via hadoop. Count of bytes read: 0
> > > > >
> > > > > org.apache.hadoop.security.AccessControlException: Connection from
> > > > > 127.0.0.1:56413 for protocol
> > > > > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > > > > is unauthorized for user hadoop via hadoop
> > > > >
> > > > > at
> > > >
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
> > > > >
> > > > > at
> > > >
> > org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
> > > > >
> > > > > at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
> > > > >
> > > > > at
> org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
> > > > >
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >
> > > > > at java.lang.Thread.run(Thread.java:662)
> > > > >
> > > > >
> > > > > What do I miss here?
> > > > >
> > > > > Regards,\
> > > > > Boris.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Harsh J
> > > >
> > >
> >
> >
> >
> > --
> > Harsh J
> >
>
--
Harsh J
Re: Oozie Security/Impersonation issue
Posted by Boris Shulman <sh...@gmail.com>.
Thanks, it worked. Putting the actual group name in the configuration
helped.
On Thu, Jul 19, 2012 at 2:44 PM, Harsh J <ha...@cloudera.com> wrote:
> Boris,
>
> By default Hadoop tries to map groups/usernames from the NameNode's unix
> list. You can see
>
> http://www.cloudera.com/blog/2012/03/authorization-and-authentication-in-hadoop/for
> a brief but complete summary on how the system is.
>
> So unless hadoop is in one of the groups its been designated to allow, it
> will not allow. Perhaps you can set the groups to *, to allow all groups,
> or set to a group "hadoop" (or whatever group "hadoop" is in) to allow it
> properly, and restart NN to check. Please let us know if that works.
>
> On Thu, Jul 19, 2012 at 4:56 PM, Boris Shulman <sh...@gmail.com> wrote:
>
> > 1) I did restart the NameNode.
> > 2) Do you mean on the OS level? Than no, so do I need to specify OS level
> > group in the config? (supergroup is the group that I see in hadoop fs -ls
> > on the files..)
> >
> > On Thu, Jul 19, 2012 at 2:12 PM, Harsh J <ha...@cloudera.com> wrote:
> >
> > > Hey Boris,
> > >
> > > Did you restart the NameNode after making this change in the
> > core-site.xml
> > > of Hadoop?
> > >
> > > Also, is "hadoop" in the supergroup physically? That is, does "groups
> > > hadoop" show supergroup in its output?
> > >
> > > On Thu, Jul 19, 2012 at 12:31 PM, Boris Shulman <sh...@gmail.com>
> > > wrote:
> > >
> > > > I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
> > > >
> > > > *I put the following configuration in the core-site.xml (in both
> hadpp
> > > and
> > > > oozie conf):*
> > > >
> > > > <property>
> > > > <name>hadoop.proxyuser.hadoop.groups</name>
> > > > <value>supergroup</value>
> > > > <description>Allow the superuser hadoop to impersonate
> > any
> > > > members of any group</description>
> > > > </property>
> > > > <property>
> > > > <name>hadoop.proxyuser.hadoop.hosts</name>
> > > > <value>127.0.0.1</value>
> > > > <description>The superuser can connect only from
> > > > localhost to impersonate a user</description>
> > > > </property>
> > > >
> > > >
> > > >
> > > > *Also I added the following in the oozie-site.xml *
> > > >
> > > > <property>
> > > >
> > > <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> > > > <value>*</value>
> > > > <description>
> > > >
> > > > </description>
> > > > </property>
> > > >
> > > > <property>
> > > >
> > > <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> > > > <value>*</value>
> > > > <description>
> > > >
> > > > </description>
> > > >
> > > > When I try to submit oozie job I get the following error:
> > > >
> > > > *In the oozie log:*
> > > >
> > > > ... 26 more
> > > > 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop]
> GROUP[-]
> > > > TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> > > > http://localhost:11000/oozie/v1/jobs?action=start] error[E0902],
> > E0902:
> > > > Exception occured: [org.apache.hadoop.ipc.RemoteException: User:
> hadoop
> > > is
> > > > not allowed to impersonate hadoop]
> > > > org.apache.oozie.servlet.XServletException: E0902: Exception occured:
> > > > [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed
> to
> > > > impersonate hadoop]
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> > > > at
> > >
> org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> > > >
> > > >
> > > >
> > > > *in the NN log:*
> > > >
> > > > 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
> > > > listener on 9000: readAndProcess threw exception
> > > > org.apache.hadoop.security.AccessControlException: Connection from
> > > > 127.0.0.1:56413 for protocol
> > > > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > > > is unauthorized for user hadoop via hadoop. Count of bytes read: 0
> > > >
> > > > org.apache.hadoop.security.AccessControlException: Connection from
> > > > 127.0.0.1:56413 for protocol
> > > > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > > > is unauthorized for user hadoop via hadoop
> > > >
> > > > at
> > > org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
> > > >
> > > > at
> > >
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
> > > >
> > > > at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
> > > >
> > > > at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
> > > >
> > > > at
> > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >
> > > > at
> > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >
> > > > at java.lang.Thread.run(Thread.java:662)
> > > >
> > > >
> > > > What do I miss here?
> > > >
> > > > Regards,\
> > > > Boris.
> > > >
> > >
> > >
> > >
> > > --
> > > Harsh J
> > >
> >
>
>
>
> --
> Harsh J
>
Re: Oozie Security/Impersonation issue
Posted by Harsh J <ha...@cloudera.com>.
Boris,
By default Hadoop tries to map groups/usernames from the NameNode's unix
list. You can see
http://www.cloudera.com/blog/2012/03/authorization-and-authentication-in-hadoop/for
a brief but complete summary on how the system is.
So unless hadoop is in one of the groups its been designated to allow, it
will not allow. Perhaps you can set the groups to *, to allow all groups,
or set to a group "hadoop" (or whatever group "hadoop" is in) to allow it
properly, and restart NN to check. Please let us know if that works.
On Thu, Jul 19, 2012 at 4:56 PM, Boris Shulman <sh...@gmail.com> wrote:
> 1) I did restart the NameNode.
> 2) Do you mean on the OS level? Than no, so do I need to specify OS level
> group in the config? (supergroup is the group that I see in hadoop fs -ls
> on the files..)
>
> On Thu, Jul 19, 2012 at 2:12 PM, Harsh J <ha...@cloudera.com> wrote:
>
> > Hey Boris,
> >
> > Did you restart the NameNode after making this change in the
> core-site.xml
> > of Hadoop?
> >
> > Also, is "hadoop" in the supergroup physically? That is, does "groups
> > hadoop" show supergroup in its output?
> >
> > On Thu, Jul 19, 2012 at 12:31 PM, Boris Shulman <sh...@gmail.com>
> > wrote:
> >
> > > I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
> > >
> > > *I put the following configuration in the core-site.xml (in both hadpp
> > and
> > > oozie conf):*
> > >
> > > <property>
> > > <name>hadoop.proxyuser.hadoop.groups</name>
> > > <value>supergroup</value>
> > > <description>Allow the superuser hadoop to impersonate
> any
> > > members of any group</description>
> > > </property>
> > > <property>
> > > <name>hadoop.proxyuser.hadoop.hosts</name>
> > > <value>127.0.0.1</value>
> > > <description>The superuser can connect only from
> > > localhost to impersonate a user</description>
> > > </property>
> > >
> > >
> > >
> > > *Also I added the following in the oozie-site.xml *
> > >
> > > <property>
> > >
> > <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> > > <value>*</value>
> > > <description>
> > >
> > > </description>
> > > </property>
> > >
> > > <property>
> > >
> > <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> > > <value>*</value>
> > > <description>
> > >
> > > </description>
> > >
> > > When I try to submit oozie job I get the following error:
> > >
> > > *In the oozie log:*
> > >
> > > ... 26 more
> > > 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop] GROUP[-]
> > > TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> > > http://localhost:11000/oozie/v1/jobs?action=start] error[E0902],
> E0902:
> > > Exception occured: [org.apache.hadoop.ipc.RemoteException: User: hadoop
> > is
> > > not allowed to impersonate hadoop]
> > > org.apache.oozie.servlet.XServletException: E0902: Exception occured:
> > > [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed to
> > > impersonate hadoop]
> > > at
> > >
> > >
> >
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> > > at
> > org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> > >
> > >
> > >
> > > *in the NN log:*
> > >
> > > 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
> > > listener on 9000: readAndProcess threw exception
> > > org.apache.hadoop.security.AccessControlException: Connection from
> > > 127.0.0.1:56413 for protocol
> > > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > > is unauthorized for user hadoop via hadoop. Count of bytes read: 0
> > >
> > > org.apache.hadoop.security.AccessControlException: Connection from
> > > 127.0.0.1:56413 for protocol
> > > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > > is unauthorized for user hadoop via hadoop
> > >
> > > at
> > org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
> > >
> > > at
> > org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
> > >
> > > at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
> > >
> > > at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
> > >
> > > at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >
> > > at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >
> > > at java.lang.Thread.run(Thread.java:662)
> > >
> > >
> > > What do I miss here?
> > >
> > > Regards,\
> > > Boris.
> > >
> >
> >
> >
> > --
> > Harsh J
> >
>
--
Harsh J
Re: Oozie Security/Impersonation issue
Posted by Boris Shulman <sh...@gmail.com>.
1) I did restart the NameNode.
2) Do you mean on the OS level? Than no, so do I need to specify OS level
group in the config? (supergroup is the group that I see in hadoop fs -ls
on the files..)
On Thu, Jul 19, 2012 at 2:12 PM, Harsh J <ha...@cloudera.com> wrote:
> Hey Boris,
>
> Did you restart the NameNode after making this change in the core-site.xml
> of Hadoop?
>
> Also, is "hadoop" in the supergroup physically? That is, does "groups
> hadoop" show supergroup in its output?
>
> On Thu, Jul 19, 2012 at 12:31 PM, Boris Shulman <sh...@gmail.com>
> wrote:
>
> > I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
> >
> > *I put the following configuration in the core-site.xml (in both hadpp
> and
> > oozie conf):*
> >
> > <property>
> > <name>hadoop.proxyuser.hadoop.groups</name>
> > <value>supergroup</value>
> > <description>Allow the superuser hadoop to impersonate any
> > members of any group</description>
> > </property>
> > <property>
> > <name>hadoop.proxyuser.hadoop.hosts</name>
> > <value>127.0.0.1</value>
> > <description>The superuser can connect only from
> > localhost to impersonate a user</description>
> > </property>
> >
> >
> >
> > *Also I added the following in the oozie-site.xml *
> >
> > <property>
> >
> <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> > <value>*</value>
> > <description>
> >
> > </description>
> > </property>
> >
> > <property>
> >
> <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> > <value>*</value>
> > <description>
> >
> > </description>
> >
> > When I try to submit oozie job I get the following error:
> >
> > *In the oozie log:*
> >
> > ... 26 more
> > 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop] GROUP[-]
> > TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> > http://localhost:11000/oozie/v1/jobs?action=start] error[E0902], E0902:
> > Exception occured: [org.apache.hadoop.ipc.RemoteException: User: hadoop
> is
> > not allowed to impersonate hadoop]
> > org.apache.oozie.servlet.XServletException: E0902: Exception occured:
> > [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed to
> > impersonate hadoop]
> > at
> >
> >
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> > at
> org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> >
> >
> >
> > *in the NN log:*
> >
> > 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
> > listener on 9000: readAndProcess threw exception
> > org.apache.hadoop.security.AccessControlException: Connection from
> > 127.0.0.1:56413 for protocol
> > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > is unauthorized for user hadoop via hadoop. Count of bytes read: 0
> >
> > org.apache.hadoop.security.AccessControlException: Connection from
> > 127.0.0.1:56413 for protocol
> > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > is unauthorized for user hadoop via hadoop
> >
> > at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
> >
> > at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
> >
> > at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
> >
> > at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
> >
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >
> > at java.lang.Thread.run(Thread.java:662)
> >
> >
> > What do I miss here?
> >
> > Regards,\
> > Boris.
> >
>
>
>
> --
> Harsh J
>
Re: Oozie Security/Impersonation issue
Posted by Harsh J <ha...@cloudera.com>.
Hey Boris,
Did you restart the NameNode after making this change in the core-site.xml
of Hadoop?
Also, is "hadoop" in the supergroup physically? That is, does "groups
hadoop" show supergroup in its output?
On Thu, Jul 19, 2012 at 12:31 PM, Boris Shulman <sh...@gmail.com> wrote:
> I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
>
> *I put the following configuration in the core-site.xml (in both hadpp and
> oozie conf):*
>
> <property>
> <name>hadoop.proxyuser.hadoop.groups</name>
> <value>supergroup</value>
> <description>Allow the superuser hadoop to impersonate any
> members of any group</description>
> </property>
> <property>
> <name>hadoop.proxyuser.hadoop.hosts</name>
> <value>127.0.0.1</value>
> <description>The superuser can connect only from
> localhost to impersonate a user</description>
> </property>
>
>
>
> *Also I added the following in the oozie-site.xml *
>
> <property>
> <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> <value>*</value>
> <description>
>
> </description>
> </property>
>
> <property>
> <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> <value>*</value>
> <description>
>
> </description>
>
> When I try to submit oozie job I get the following error:
>
> *In the oozie log:*
>
> ... 26 more
> 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> http://localhost:11000/oozie/v1/jobs?action=start] error[E0902], E0902:
> Exception occured: [org.apache.hadoop.ipc.RemoteException: User: hadoop is
> not allowed to impersonate hadoop]
> org.apache.oozie.servlet.XServletException: E0902: Exception occured:
> [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed to
> impersonate hadoop]
> at
>
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
>
>
>
> *in the NN log:*
>
> 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
> listener on 9000: readAndProcess threw exception
> org.apache.hadoop.security.AccessControlException: Connection from
> 127.0.0.1:56413 for protocol
> org.apache.hadoop.hdfs.protocol.ClientProtocol
> is unauthorized for user hadoop via hadoop. Count of bytes read: 0
>
> org.apache.hadoop.security.AccessControlException: Connection from
> 127.0.0.1:56413 for protocol
> org.apache.hadoop.hdfs.protocol.ClientProtocol
> is unauthorized for user hadoop via hadoop
>
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
>
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
>
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
>
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
>
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
> What do I miss here?
>
> Regards,\
> Boris.
>
--
Harsh J
Re: Oozie Security/Impersonation issue
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Borks,
The*
file:/"/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201207190945_0007/jobToken"
*file name seems odd. Would you check in Hadoop 1.1 code to see if the
double quotes are part of the message of somehow the file name is coming
like that.
Thx
On Thu, Jul 19, 2012 at 6:56 PM, Boris Shulman <sh...@gmail.com> wrote:
> Now I am running into other issue. For a simple Map Reduce task I get the
> following error:
>
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.MapReduceMain], main() threw
> exception, Exception reading
>
> file:/"/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201207190945_0007/jobToken"
> java.io.IOException: Exception reading
>
> file:/"/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201207190945_0007/jobToken"
> at
> org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:136)
> at
> org.apache.hadoop.mapred.JobClient.readTokensFromFiles(JobClient.java:2042)
> at
> org.apache.hadoop.mapred.JobClient.populateTokenCache(JobClient.java:2078)
> at
> org.apache.hadoop.mapred.JobClient.access$200(JobClient.java:174)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:863)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1124)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
> at
> org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:91)
> at
> org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:57)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
> at
> org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:40)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:467)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:271)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1124)
> at org.apache.hadoop.mapred.Child.main(Child.java:265)
> Caused by: java.io.FileNotFoundException: File
>
> file:/"/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201207190945_0007/jobToken"
> does not exist.
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:424)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
> at
> org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:130)
> ... 27 more
>
> It seems like the impersonated user doesn't have the credentials to access
> this file...
>
> Any Suggestions?
> Boris.
>
>
> On Thu, Jul 19, 2012 at 8:13 PM, Harsh J <ha...@cloudera.com> wrote:
>
> > Sorry for not cross-posting onto this list too (didn't notice it was
> > cross-posted). I had answered Boris here:
> >
> >
> http://mail-archives.apache.org/mod_mbox/incubator-oozie-dev/201207.mbox/%3cCA+4sW8iPVz2V-3Ss8b0u6PuV7OPaT6jiKNQu_UO9Z2roycwkRA@mail.gmail.com%3e
> > over the dev lists.
> >
> > On Thu, Jul 19, 2012 at 10:37 PM, Matt Goeke <go...@gmail.com>
> > wrote:
> > > Boris,
> > >
> > > As Harish mentioned, your proxyuser properties
> > > (hadoop.proxyuser.<user>.groups and hadoop.proxyuser.<user>.hosts) must
> > be
> > > in the core-site.xml on the node(s) that your jobtracker and namenode
> are
> > > located.
> > >
> > > --
> > > Matt Goeke
> > >
> > > On Thu, Jul 19, 2012 at 9:36 AM, Harish Krishnan <
> > > harish.t.krishnan@gmail.com> wrote:
> > >
> > >> Hi Boris,
> > >>
> > >> Try this in your core-site.xml
> > >>
> > >> <property>
> > >> <name>hadoop.proxyuser.hadoop.hosts</name>
> > >> <value>*</value>
> > >> <description>The superuser can connect only from
> > >> localhost to impersonate a user</description>
> > >> </property>
> > >>
> > >>
> > >>
> > >> Thanks & Regards,
> > >> Harish.T.K
> > >>
> > >>
> > >> On Thu, Jul 19, 2012 at 12:01 AM, Boris Shulman <sh...@gmail.com>
> > >> wrote:
> > >>
> > >> > I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
> > >> >
> > >> > *I put the following configuration in the core-site.xml (in both
> hadpp
> > >> and
> > >> > oozie conf):*
> > >> >
> > >> > <property>
> > >> > <name>hadoop.proxyuser.hadoop.groups</name>
> > >> > <value>supergroup</value>
> > >> > <description>Allow the superuser hadoop to
> impersonate
> > any
> > >> > members of any group</description>
> > >> > </property>
> > >> > <property>
> > >> > <name>hadoop.proxyuser.hadoop.hosts</name>
> > >> > <value>127.0.0.1</value>
> > >> > <description>The superuser can connect only from
> > >> > localhost to impersonate a user</description>
> > >> > </property>
> > >> >
> > >> >
> > >> >
> > >> > *Also I added the following in the oozie-site.xml *
> > >> >
> > >> > <property>
> > >> >
> > >> <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> > >> > <value>*</value>
> > >> > <description>
> > >> >
> > >> > </description>
> > >> > </property>
> > >> >
> > >> > <property>
> > >> >
> > >> <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> > >> > <value>*</value>
> > >> > <description>
> > >> >
> > >> > </description>
> > >> >
> > >> > When I try to submit oozie job I get the following error:
> > >> >
> > >> > *In the oozie log:*
> > >> >
> > >> > ... 26 more
> > >> > 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop]
> > GROUP[-]
> > >> > TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> > >> > http://localhost:11000/oozie/v1/jobs?action=start] error[E0902],
> > E0902:
> > >> > Exception occured: [org.apache.hadoop.ipc.RemoteException: User:
> > hadoop
> > >> is
> > >> > not allowed to impersonate hadoop]
> > >> > org.apache.oozie.servlet.XServletException: E0902: Exception
> occured:
> > >> > [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed
> to
> > >> > impersonate hadoop]
> > >> > at
> > >> >
> > >> >
> > >>
> >
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> > >> > at
> > >>
> org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> > >> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> > >> >
> > >> >
> > >> >
> > >> > *in the NN log:*
> > >> >
> > >> > 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC
> Server
> > >> > listener on 9000: readAndProcess threw exception
> > >> > org.apache.hadoop.security.AccessControlException: Connection from
> > >> > 127.0.0.1:56413 for protocol
> > >> > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > >> > is unauthorized for user hadoop via hadoop. Count of bytes read: 0
> > >> >
> > >> > org.apache.hadoop.security.AccessControlException: Connection from
> > >> > 127.0.0.1:56413 for protocol
> > >> > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > >> > is unauthorized for user hadoop via hadoop
> > >> >
> > >> > at
> > >>
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
> > >> >
> > >> > at
> > >>
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
> > >> >
> > >> > at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
> > >> >
> > >> > at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
> > >> >
> > >> > at
> > >> >
> > >> >
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >> >
> > >> > at
> > >> >
> > >> >
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >> >
> > >> > at java.lang.Thread.run(Thread.java:662)
> > >> >
> > >> >
> > >> > What do I miss here?
> > >> >
> > >> > Regards,\
> > >> > Boris.
> > >> >
> > >>
> >
> >
> >
> > --
> > Harsh J
> >
>
--
Alejandro
Re: Oozie Security/Impersonation issue
Posted by Boris Shulman <sh...@gmail.com>.
Now I am running into other issue. For a simple Map Reduce task I get the
following error:
Failing Oozie Launcher, Main class
[org.apache.oozie.action.hadoop.MapReduceMain], main() threw
exception, Exception reading
file:/"/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201207190945_0007/jobToken"
java.io.IOException: Exception reading
file:/"/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201207190945_0007/jobToken"
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:136)
at org.apache.hadoop.mapred.JobClient.readTokensFromFiles(JobClient.java:2042)
at org.apache.hadoop.mapred.JobClient.populateTokenCache(JobClient.java:2078)
at org.apache.hadoop.mapred.JobClient.access$200(JobClient.java:174)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:863)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1124)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
at org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:91)
at org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:57)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:467)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:271)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1124)
at org.apache.hadoop.mapred.Child.main(Child.java:265)
Caused by: java.io.FileNotFoundException: File
file:/"/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201207190945_0007/jobToken"
does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:424)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:130)
... 27 more
It seems like the impersonated user doesn't have the credentials to access
this file...
Any Suggestions?
Boris.
On Thu, Jul 19, 2012 at 8:13 PM, Harsh J <ha...@cloudera.com> wrote:
> Sorry for not cross-posting onto this list too (didn't notice it was
> cross-posted). I had answered Boris here:
>
> http://mail-archives.apache.org/mod_mbox/incubator-oozie-dev/201207.mbox/%3cCA+4sW8iPVz2V-3Ss8b0u6PuV7OPaT6jiKNQu_UO9Z2roycwkRA@mail.gmail.com%3e
> over the dev lists.
>
> On Thu, Jul 19, 2012 at 10:37 PM, Matt Goeke <go...@gmail.com>
> wrote:
> > Boris,
> >
> > As Harish mentioned, your proxyuser properties
> > (hadoop.proxyuser.<user>.groups and hadoop.proxyuser.<user>.hosts) must
> be
> > in the core-site.xml on the node(s) that your jobtracker and namenode are
> > located.
> >
> > --
> > Matt Goeke
> >
> > On Thu, Jul 19, 2012 at 9:36 AM, Harish Krishnan <
> > harish.t.krishnan@gmail.com> wrote:
> >
> >> Hi Boris,
> >>
> >> Try this in your core-site.xml
> >>
> >> <property>
> >> <name>hadoop.proxyuser.hadoop.hosts</name>
> >> <value>*</value>
> >> <description>The superuser can connect only from
> >> localhost to impersonate a user</description>
> >> </property>
> >>
> >>
> >>
> >> Thanks & Regards,
> >> Harish.T.K
> >>
> >>
> >> On Thu, Jul 19, 2012 at 12:01 AM, Boris Shulman <sh...@gmail.com>
> >> wrote:
> >>
> >> > I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
> >> >
> >> > *I put the following configuration in the core-site.xml (in both hadpp
> >> and
> >> > oozie conf):*
> >> >
> >> > <property>
> >> > <name>hadoop.proxyuser.hadoop.groups</name>
> >> > <value>supergroup</value>
> >> > <description>Allow the superuser hadoop to impersonate
> any
> >> > members of any group</description>
> >> > </property>
> >> > <property>
> >> > <name>hadoop.proxyuser.hadoop.hosts</name>
> >> > <value>127.0.0.1</value>
> >> > <description>The superuser can connect only from
> >> > localhost to impersonate a user</description>
> >> > </property>
> >> >
> >> >
> >> >
> >> > *Also I added the following in the oozie-site.xml *
> >> >
> >> > <property>
> >> >
> >> <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> >> > <value>*</value>
> >> > <description>
> >> >
> >> > </description>
> >> > </property>
> >> >
> >> > <property>
> >> >
> >> <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> >> > <value>*</value>
> >> > <description>
> >> >
> >> > </description>
> >> >
> >> > When I try to submit oozie job I get the following error:
> >> >
> >> > *In the oozie log:*
> >> >
> >> > ... 26 more
> >> > 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop]
> GROUP[-]
> >> > TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> >> > http://localhost:11000/oozie/v1/jobs?action=start] error[E0902],
> E0902:
> >> > Exception occured: [org.apache.hadoop.ipc.RemoteException: User:
> hadoop
> >> is
> >> > not allowed to impersonate hadoop]
> >> > org.apache.oozie.servlet.XServletException: E0902: Exception occured:
> >> > [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed to
> >> > impersonate hadoop]
> >> > at
> >> >
> >> >
> >>
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> >> > at
> >> org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> >> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> >> >
> >> >
> >> >
> >> > *in the NN log:*
> >> >
> >> > 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
> >> > listener on 9000: readAndProcess threw exception
> >> > org.apache.hadoop.security.AccessControlException: Connection from
> >> > 127.0.0.1:56413 for protocol
> >> > org.apache.hadoop.hdfs.protocol.ClientProtocol
> >> > is unauthorized for user hadoop via hadoop. Count of bytes read: 0
> >> >
> >> > org.apache.hadoop.security.AccessControlException: Connection from
> >> > 127.0.0.1:56413 for protocol
> >> > org.apache.hadoop.hdfs.protocol.ClientProtocol
> >> > is unauthorized for user hadoop via hadoop
> >> >
> >> > at
> >> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
> >> >
> >> > at
> >> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
> >> >
> >> > at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
> >> >
> >> > at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
> >> >
> >> > at
> >> >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> >
> >> > at
> >> >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> >
> >> > at java.lang.Thread.run(Thread.java:662)
> >> >
> >> >
> >> > What do I miss here?
> >> >
> >> > Regards,\
> >> > Boris.
> >> >
> >>
>
>
>
> --
> Harsh J
>
Re: Oozie Security/Impersonation issue
Posted by Harsh J <ha...@cloudera.com>.
Sorry for not cross-posting onto this list too (didn't notice it was
cross-posted). I had answered Boris here:
http://mail-archives.apache.org/mod_mbox/incubator-oozie-dev/201207.mbox/%3cCA+4sW8iPVz2V-3Ss8b0u6PuV7OPaT6jiKNQu_UO9Z2roycwkRA@mail.gmail.com%3e
over the dev lists.
On Thu, Jul 19, 2012 at 10:37 PM, Matt Goeke <go...@gmail.com> wrote:
> Boris,
>
> As Harish mentioned, your proxyuser properties
> (hadoop.proxyuser.<user>.groups and hadoop.proxyuser.<user>.hosts) must be
> in the core-site.xml on the node(s) that your jobtracker and namenode are
> located.
>
> --
> Matt Goeke
>
> On Thu, Jul 19, 2012 at 9:36 AM, Harish Krishnan <
> harish.t.krishnan@gmail.com> wrote:
>
>> Hi Boris,
>>
>> Try this in your core-site.xml
>>
>> <property>
>> <name>hadoop.proxyuser.hadoop.hosts</name>
>> <value>*</value>
>> <description>The superuser can connect only from
>> localhost to impersonate a user</description>
>> </property>
>>
>>
>>
>> Thanks & Regards,
>> Harish.T.K
>>
>>
>> On Thu, Jul 19, 2012 at 12:01 AM, Boris Shulman <sh...@gmail.com>
>> wrote:
>>
>> > I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
>> >
>> > *I put the following configuration in the core-site.xml (in both hadpp
>> and
>> > oozie conf):*
>> >
>> > <property>
>> > <name>hadoop.proxyuser.hadoop.groups</name>
>> > <value>supergroup</value>
>> > <description>Allow the superuser hadoop to impersonate any
>> > members of any group</description>
>> > </property>
>> > <property>
>> > <name>hadoop.proxyuser.hadoop.hosts</name>
>> > <value>127.0.0.1</value>
>> > <description>The superuser can connect only from
>> > localhost to impersonate a user</description>
>> > </property>
>> >
>> >
>> >
>> > *Also I added the following in the oozie-site.xml *
>> >
>> > <property>
>> >
>> <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
>> > <value>*</value>
>> > <description>
>> >
>> > </description>
>> > </property>
>> >
>> > <property>
>> >
>> <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
>> > <value>*</value>
>> > <description>
>> >
>> > </description>
>> >
>> > When I try to submit oozie job I get the following error:
>> >
>> > *In the oozie log:*
>> >
>> > ... 26 more
>> > 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop] GROUP[-]
>> > TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
>> > http://localhost:11000/oozie/v1/jobs?action=start] error[E0902], E0902:
>> > Exception occured: [org.apache.hadoop.ipc.RemoteException: User: hadoop
>> is
>> > not allowed to impersonate hadoop]
>> > org.apache.oozie.servlet.XServletException: E0902: Exception occured:
>> > [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed to
>> > impersonate hadoop]
>> > at
>> >
>> >
>> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
>> > at
>> org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
>> >
>> >
>> >
>> > *in the NN log:*
>> >
>> > 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
>> > listener on 9000: readAndProcess threw exception
>> > org.apache.hadoop.security.AccessControlException: Connection from
>> > 127.0.0.1:56413 for protocol
>> > org.apache.hadoop.hdfs.protocol.ClientProtocol
>> > is unauthorized for user hadoop via hadoop. Count of bytes read: 0
>> >
>> > org.apache.hadoop.security.AccessControlException: Connection from
>> > 127.0.0.1:56413 for protocol
>> > org.apache.hadoop.hdfs.protocol.ClientProtocol
>> > is unauthorized for user hadoop via hadoop
>> >
>> > at
>> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
>> >
>> > at
>> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
>> >
>> > at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
>> >
>> > at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
>> >
>> > at
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> >
>> > at
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> >
>> > at java.lang.Thread.run(Thread.java:662)
>> >
>> >
>> > What do I miss here?
>> >
>> > Regards,\
>> > Boris.
>> >
>>
--
Harsh J
Re: Oozie Security/Impersonation issue
Posted by Matt Goeke <go...@gmail.com>.
Boris,
As Harish mentioned, your proxyuser properties
(hadoop.proxyuser.<user>.groups and hadoop.proxyuser.<user>.hosts) must be
in the core-site.xml on the node(s) that your jobtracker and namenode are
located.
--
Matt Goeke
On Thu, Jul 19, 2012 at 9:36 AM, Harish Krishnan <
harish.t.krishnan@gmail.com> wrote:
> Hi Boris,
>
> Try this in your core-site.xml
>
> <property>
> <name>hadoop.proxyuser.hadoop.hosts</name>
> <value>*</value>
> <description>The superuser can connect only from
> localhost to impersonate a user</description>
> </property>
>
>
>
> Thanks & Regards,
> Harish.T.K
>
>
> On Thu, Jul 19, 2012 at 12:01 AM, Boris Shulman <sh...@gmail.com>
> wrote:
>
> > I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
> >
> > *I put the following configuration in the core-site.xml (in both hadpp
> and
> > oozie conf):*
> >
> > <property>
> > <name>hadoop.proxyuser.hadoop.groups</name>
> > <value>supergroup</value>
> > <description>Allow the superuser hadoop to impersonate any
> > members of any group</description>
> > </property>
> > <property>
> > <name>hadoop.proxyuser.hadoop.hosts</name>
> > <value>127.0.0.1</value>
> > <description>The superuser can connect only from
> > localhost to impersonate a user</description>
> > </property>
> >
> >
> >
> > *Also I added the following in the oozie-site.xml *
> >
> > <property>
> >
> <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> > <value>*</value>
> > <description>
> >
> > </description>
> > </property>
> >
> > <property>
> >
> <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> > <value>*</value>
> > <description>
> >
> > </description>
> >
> > When I try to submit oozie job I get the following error:
> >
> > *In the oozie log:*
> >
> > ... 26 more
> > 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop] GROUP[-]
> > TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> > http://localhost:11000/oozie/v1/jobs?action=start] error[E0902], E0902:
> > Exception occured: [org.apache.hadoop.ipc.RemoteException: User: hadoop
> is
> > not allowed to impersonate hadoop]
> > org.apache.oozie.servlet.XServletException: E0902: Exception occured:
> > [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed to
> > impersonate hadoop]
> > at
> >
> >
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> > at
> org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> >
> >
> >
> > *in the NN log:*
> >
> > 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
> > listener on 9000: readAndProcess threw exception
> > org.apache.hadoop.security.AccessControlException: Connection from
> > 127.0.0.1:56413 for protocol
> > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > is unauthorized for user hadoop via hadoop. Count of bytes read: 0
> >
> > org.apache.hadoop.security.AccessControlException: Connection from
> > 127.0.0.1:56413 for protocol
> > org.apache.hadoop.hdfs.protocol.ClientProtocol
> > is unauthorized for user hadoop via hadoop
> >
> > at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
> >
> > at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
> >
> > at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
> >
> > at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
> >
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >
> > at java.lang.Thread.run(Thread.java:662)
> >
> >
> > What do I miss here?
> >
> > Regards,\
> > Boris.
> >
>
Re: Oozie Security/Impersonation issue
Posted by Harish Krishnan <ha...@gmail.com>.
Hi Boris,
Try this in your core-site.xml
<property>
<name>hadoop.proxyuser.hadoop.hosts</name>
<value>*</value>
<description>The superuser can connect only from
localhost to impersonate a user</description>
</property>
Thanks & Regards,
Harish.T.K
On Thu, Jul 19, 2012 at 12:01 AM, Boris Shulman <sh...@gmail.com> wrote:
> I am trying to run Oozie 3.2.0 on Hadoop 1.1.0 build.
>
> *I put the following configuration in the core-site.xml (in both hadpp and
> oozie conf):*
>
> <property>
> <name>hadoop.proxyuser.hadoop.groups</name>
> <value>supergroup</value>
> <description>Allow the superuser hadoop to impersonate any
> members of any group</description>
> </property>
> <property>
> <name>hadoop.proxyuser.hadoop.hosts</name>
> <value>127.0.0.1</value>
> <description>The superuser can connect only from
> localhost to impersonate a user</description>
> </property>
>
>
>
> *Also I added the following in the oozie-site.xml *
>
> <property>
> <name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
> <value>*</value>
> <description>
>
> </description>
> </property>
>
> <property>
> <name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
> <value>*</value>
> <description>
>
> </description>
>
> When I try to submit oozie job I get the following error:
>
> *In the oozie log:*
>
> ... 26 more
> 2012-07-19 09:45:32,861 WARN V1JobsServlet:542 - USER[hadoop] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST
> http://localhost:11000/oozie/v1/jobs?action=start] error[E0902], E0902:
> Exception occured: [org.apache.hadoop.ipc.RemoteException: User: hadoop is
> not allowed to impersonate hadoop]
> org.apache.oozie.servlet.XServletException: E0902: Exception occured:
> [org.apache.hadoop.ipc.RemoteException: User: hadoop is not allowed to
> impersonate hadoop]
> at
>
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:199)
> at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
>
>
>
> *in the NN log:*
>
> 2012-07-19 09:59:10,840 INFO org.apache.hadoop.ipc.Server: IPC Server
> listener on 9000: readAndProcess threw exception
> org.apache.hadoop.security.AccessControlException: Connection from
> 127.0.0.1:56413 for protocol
> org.apache.hadoop.hdfs.protocol.ClientProtocol
> is unauthorized for user hadoop via hadoop. Count of bytes read: 0
>
> org.apache.hadoop.security.AccessControlException: Connection from
> 127.0.0.1:56413 for protocol
> org.apache.hadoop.hdfs.protocol.ClientProtocol
> is unauthorized for user hadoop via hadoop
>
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1292)
>
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1187)
>
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:538)
>
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:345)
>
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
> What do I miss here?
>
> Regards,\
> Boris.
>