You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Namit Kabra <na...@gmail.com> on 2019/02/27 12:54:05 UTC
Oozie on Kerberos
Hi,
I am searching for some documentation on how to run a mapreduce job using
oozie on a kerberized cluster?
Any help would be appreciated.
Regards,
Namit Kabra
Note: Here are the references that I tried:
1. It may be possible to run oozie using curl command on the Kereberized
cluster by using: oozie jobs -auth KERBEROS -oozie
https://[redacted-server-same]:11443/oozie (Reference:
https://community.cloudera.com/t5/Batch-Processing-and-Workflow/oozie-Authentication-exception-GSSException-No-valid/td-p/66172
)
2. https://oozie.apache.org/docs/4.3.1/ENG_Custom_Authentication.html
3.
https://ambari.apache.org/1.2.5/installing-hadoop-using-ambari/content/ambari-kerb-2-3-4.html
( Configuring Secure Oozie)
4.
https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cdh_sg_oozie_server_kerberos.html
(Configuring Kerberos Authentication for the Oozie Server for Cloudera)
5. https://oozie.apache.org/docs/3.3.2/AG_Install.html (Setting Up Oozie
with HTTPS (SSL))
6. https://oozie.apache.org/docs/3.2.0-incubating/DG_CommandLineTool.html
(Oozie CLI Utilities)
Re: Oozie on Kerberos
Posted by Andras Salamon <as...@cloudera.com.INVALID>.
Hi,
1. What is the error message? If you use SSL, you have a different port
(11443 by default).
2. I think you need to check the Oozie logs if it does not start the
mapreduce/yarn job.
Best,
Sala
On Fri, Mar 8, 2019 at 5:38 PM Namit Kabra <na...@gmail.com> wrote:
> HI Andras,
>
> Thank you for your response. I was not even expecting that someone would
> reply to me.
> I have the following specific questions:
> 1. How do I initiate a oozie job on a kerberos system? The following
> command does not work:
> oozie job -oozie http://localhost:11000/oozie -config job.properties –run
> 2. My oozie job is not triggering the MapReduce job (may be due to lack of
> permissions). How do I give those permissions so that MapReduce job gets
> triggered? What can be other reasons for the MapReduce to not get
> triggered? The Yarn log (on Ambari) does not show me anything.
>
> Thank you for your help,
> Regards,
> Namit Kabra
>
>
>
> On Wed, Feb 27, 2019 at 7:30 PM Andras Piros <an...@apache.org> wrote:
>
> > Hi Namit,
> >
> > for Oozie latest 5.1.0, you can try the following:
> >
> > 1. setting up proper Hadoop configuration
> > <https://oozie.apache.org/docs/5.1.0/DG_ActionAuthentication.html>,
> > also while
> > installing
> > <
> >
> https://oozie.apache.org/docs/5.1.0/AG_Install.html#Oozie_Hadoop_Authentication_Configuration
> > >
> > 2. server setup if you need SSL
> > <
> >
> https://oozie.apache.org/docs/5.1.0/AG_Install.html#Setting_Up_Oozie_with_HTTPS_SSL
> > >
> > 3. action-level Hadoop authentication setup
> > <https://oozie.apache.org/docs/5.1.0/DG_ActionAuthentication.html>.
> > Here's an example
> > <
> >
> https://github.com/apache/oozie/blob/master/examples/src/main/apps/hive2/workflow.xml.security
> > >
> > 4. CLI authentication
> > <
> >
> https://oozie.apache.org/docs/5.1.0/DG_CommandLineTool.html#Authentication
> > >
> >
> > Hope that helps,
> >
> > Andras
> >
> > On Wed, Feb 27, 2019 at 10:03 AM Namit Kabra <na...@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > I am searching for some documentation on how to run a mapreduce job
> using
> > > oozie on a kerberized cluster?
> > > Any help would be appreciated.
> > >
> > > Regards,
> > > Namit Kabra
> > >
> > > Note: Here are the references that I tried:
> > >
> > > 1. It may be possible to run oozie using curl command on the
> > Kereberized
> > > cluster by using: oozie jobs -auth KERBEROS -oozie
> > > https://[redacted-server-same]:11443/oozie (Reference:
> > >
> > >
> >
> https://community.cloudera.com/t5/Batch-Processing-and-Workflow/oozie-Authentication-exception-GSSException-No-valid/td-p/66172
> > > )
> > > 2.
> https://oozie.apache.org/docs/4.3.1/ENG_Custom_Authentication.html
> > > 3.
> > >
> > >
> >
> https://ambari.apache.org/1.2.5/installing-hadoop-using-ambari/content/ambari-kerb-2-3-4.html
> > > ( Configuring Secure Oozie)
> > > 4.
> > >
> > >
> >
> https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cdh_sg_oozie_server_kerberos.html
> > > (Configuring Kerberos Authentication for the Oozie Server for
> > Cloudera)
> > > 5. https://oozie.apache.org/docs/3.3.2/AG_Install.html (Setting Up
> > > Oozie
> > > with HTTPS (SSL))
> > > 6.
> > > https://oozie.apache.org/docs/3.2.0-incubating/DG_CommandLineTool.html
> > > (Oozie CLI Utilities)
> > >
> >
>
Re: Oozie on Kerberos
Posted by Namit Kabra <na...@gmail.com>.
HI Andras,
Thank you for your response. I was not even expecting that someone would
reply to me.
I have the following specific questions:
1. How do I initiate a oozie job on a kerberos system? The following
command does not work:
oozie job -oozie http://localhost:11000/oozie -config job.properties –run
2. My oozie job is not triggering the MapReduce job (may be due to lack of
permissions). How do I give those permissions so that MapReduce job gets
triggered? What can be other reasons for the MapReduce to not get
triggered? The Yarn log (on Ambari) does not show me anything.
Thank you for your help,
Regards,
Namit Kabra
On Wed, Feb 27, 2019 at 7:30 PM Andras Piros <an...@apache.org> wrote:
> Hi Namit,
>
> for Oozie latest 5.1.0, you can try the following:
>
> 1. setting up proper Hadoop configuration
> <https://oozie.apache.org/docs/5.1.0/DG_ActionAuthentication.html>,
> also while
> installing
> <
> https://oozie.apache.org/docs/5.1.0/AG_Install.html#Oozie_Hadoop_Authentication_Configuration
> >
> 2. server setup if you need SSL
> <
> https://oozie.apache.org/docs/5.1.0/AG_Install.html#Setting_Up_Oozie_with_HTTPS_SSL
> >
> 3. action-level Hadoop authentication setup
> <https://oozie.apache.org/docs/5.1.0/DG_ActionAuthentication.html>.
> Here's an example
> <
> https://github.com/apache/oozie/blob/master/examples/src/main/apps/hive2/workflow.xml.security
> >
> 4. CLI authentication
> <
> https://oozie.apache.org/docs/5.1.0/DG_CommandLineTool.html#Authentication
> >
>
> Hope that helps,
>
> Andras
>
> On Wed, Feb 27, 2019 at 10:03 AM Namit Kabra <na...@gmail.com> wrote:
>
> > Hi,
> >
> > I am searching for some documentation on how to run a mapreduce job using
> > oozie on a kerberized cluster?
> > Any help would be appreciated.
> >
> > Regards,
> > Namit Kabra
> >
> > Note: Here are the references that I tried:
> >
> > 1. It may be possible to run oozie using curl command on the
> Kereberized
> > cluster by using: oozie jobs -auth KERBEROS -oozie
> > https://[redacted-server-same]:11443/oozie (Reference:
> >
> >
> https://community.cloudera.com/t5/Batch-Processing-and-Workflow/oozie-Authentication-exception-GSSException-No-valid/td-p/66172
> > )
> > 2. https://oozie.apache.org/docs/4.3.1/ENG_Custom_Authentication.html
> > 3.
> >
> >
> https://ambari.apache.org/1.2.5/installing-hadoop-using-ambari/content/ambari-kerb-2-3-4.html
> > ( Configuring Secure Oozie)
> > 4.
> >
> >
> https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cdh_sg_oozie_server_kerberos.html
> > (Configuring Kerberos Authentication for the Oozie Server for
> Cloudera)
> > 5. https://oozie.apache.org/docs/3.3.2/AG_Install.html (Setting Up
> > Oozie
> > with HTTPS (SSL))
> > 6.
> > https://oozie.apache.org/docs/3.2.0-incubating/DG_CommandLineTool.html
> > (Oozie CLI Utilities)
> >
>
Re: Oozie on Kerberos
Posted by Andras Piros <an...@apache.org>.
Hi Namit,
for Oozie latest 5.1.0, you can try the following:
1. setting up proper Hadoop configuration
<https://oozie.apache.org/docs/5.1.0/DG_ActionAuthentication.html>,
also while
installing
<https://oozie.apache.org/docs/5.1.0/AG_Install.html#Oozie_Hadoop_Authentication_Configuration>
2. server setup if you need SSL
<https://oozie.apache.org/docs/5.1.0/AG_Install.html#Setting_Up_Oozie_with_HTTPS_SSL>
3. action-level Hadoop authentication setup
<https://oozie.apache.org/docs/5.1.0/DG_ActionAuthentication.html>.
Here's an example
<https://github.com/apache/oozie/blob/master/examples/src/main/apps/hive2/workflow.xml.security>
4. CLI authentication
<https://oozie.apache.org/docs/5.1.0/DG_CommandLineTool.html#Authentication>
Hope that helps,
Andras
On Wed, Feb 27, 2019 at 10:03 AM Namit Kabra <na...@gmail.com> wrote:
> Hi,
>
> I am searching for some documentation on how to run a mapreduce job using
> oozie on a kerberized cluster?
> Any help would be appreciated.
>
> Regards,
> Namit Kabra
>
> Note: Here are the references that I tried:
>
> 1. It may be possible to run oozie using curl command on the Kereberized
> cluster by using: oozie jobs -auth KERBEROS -oozie
> https://[redacted-server-same]:11443/oozie (Reference:
>
> https://community.cloudera.com/t5/Batch-Processing-and-Workflow/oozie-Authentication-exception-GSSException-No-valid/td-p/66172
> )
> 2. https://oozie.apache.org/docs/4.3.1/ENG_Custom_Authentication.html
> 3.
>
> https://ambari.apache.org/1.2.5/installing-hadoop-using-ambari/content/ambari-kerb-2-3-4.html
> ( Configuring Secure Oozie)
> 4.
>
> https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cdh_sg_oozie_server_kerberos.html
> (Configuring Kerberos Authentication for the Oozie Server for Cloudera)
> 5. https://oozie.apache.org/docs/3.3.2/AG_Install.html (Setting Up
> Oozie
> with HTTPS (SSL))
> 6.
> https://oozie.apache.org/docs/3.2.0-incubating/DG_CommandLineTool.html
> (Oozie CLI Utilities)
>