You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by "Vishwakarma, Chhaya" <Ch...@Teradata.com> on 2015/06/04 11:45:50 UTC
Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Hi All,
I have written a Oozie workflow which has a Sqoop action to pull data from Teradata and load into Hive table.
I have kerberos enabled hadoop cluster.
When I run Oozie workflwo I get below error
35521 [Thread-34] INFO org.apache.sqoop.hive.HiveImport -
35522 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
Heart beat
Heart beat
98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Delegation Token can be issued only with kerberos or web authentication
98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:6280)
I referred beelow link and made required changes as suggested in the linkhttps://prodlife.wordpress.com/2013/11/22/using-oozie-in-kerberized-cluster/
I'm missing anything here ?
please suggest
Re: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Posted by Abraham Elmahrek <ab...@cloudera.com>.
We recently found https://issues.apache.org/jira/browse/SQOOP-2400 which
might be the root cause of this issue.
-Abe
On Mon, Jun 15, 2015 at 5:59 AM, Zhou, Richard <ri...@intel.com>
wrote:
> Actually, there is a little complicated in a Kerberized cluster. Would
> you share more details about your environment for us to better understand?
> such as config file, log file and workflow for Oozie and Sqoop etc.
>
>
>
> BTW:
>
> Have you ever tested Sqoop 1 in the command line? The command “kinit” is
> required.
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Teradata.com]
> *Sent:* Thursday, June 11, 2015 9:10 PM
>
> *To:* user@sqoop.apache.org
> *Subject:* RE: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
>
>
>
> Sqoop 1.4
>
>
>
> *From:* Zhou, Richard [mailto:richard.zhou@intel.com
> <ri...@intel.com>]
> *Sent:* Friday, June 05, 2015 6:41 AM
> *To:* user@sqoop.apache.org
> *Subject:* RE: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
>
>
>
> Hey,
>
>
>
> Which version of Sqoop are you using?
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Teradata.com
> <Ch...@Teradata.com>]
> *Sent:* Thursday, June 04, 2015 5:46 PM
> *To:* user@sqoop.apache.org
> *Subject:* Oozie-sqoop action fails in Kerberos enabled hadoop cluster
>
>
>
> Hi All,
>
> I have written a Oozie workflow which has a Sqoop action to pull data from
> Teradata and load into Hive table.
>
> I have kerberos enabled hadoop cluster.
>
> When I run Oozie workflwo I get below error
>
> 35521 [Thread-34] INFO org.apache.sqoop.hive.HiveImport -
>
> 35522 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Logging
> initialized using configuration in
> file:/etc/hive/conf.dist/hive-log4j.properties
>
> Heart beat
>
> Heart beat
>
> 98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Delegation Token can be issued only with kerberos or web authentication
>
> 98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:6280)
>
>
>
> I referred beelow link and made required changes as suggested in the link
> https://prodlife.wordpress.com/2013/11/22/using-oozie-in-kerberized-cluster/
>
> I'm missing anything here ?
>
> please suggest
>
>
>
RE: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Posted by "Zhou, Richard" <ri...@intel.com>.
Actually, there is a little complicated in a Kerberized cluster. Would you share more details about your environment for us to better understand? such as config file, log file and workflow for Oozie and Sqoop etc.
BTW:
Have you ever tested Sqoop 1 in the command line? The command "kinit" is required.
Regards
Richard
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Teradata.com]
Sent: Thursday, June 11, 2015 9:10 PM
To: user@sqoop.apache.org
Subject: RE: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Sqoop 1.4
From: Zhou, Richard [mailto:richard.zhou@intel.com]
Sent: Friday, June 05, 2015 6:41 AM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: RE: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Hey,
Which version of Sqoop are you using?
Regards
Richard
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Teradata.com]
Sent: Thursday, June 04, 2015 5:46 PM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Hi All,
I have written a Oozie workflow which has a Sqoop action to pull data from Teradata and load into Hive table.
I have kerberos enabled hadoop cluster.
When I run Oozie workflwo I get below error
35521 [Thread-34] INFO org.apache.sqoop.hive.HiveImport -
35522 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
Heart beat
Heart beat
98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Delegation Token can be issued only with kerberos or web authentication
98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:6280)
I referred beelow link and made required changes as suggested in the linkhttps://prodlife.wordpress.com/2013/11/22/using-oozie-in-kerberized-cluster/
I'm missing anything here ?
please suggest
RE: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Posted by "Vishwakarma, Chhaya" <Ch...@Teradata.com>.
Sqoop 1.4
From: Zhou, Richard [mailto:richard.zhou@intel.com]
Sent: Friday, June 05, 2015 6:41 AM
To: user@sqoop.apache.org
Subject: RE: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Hey,
Which version of Sqoop are you using?
Regards
Richard
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Teradata.com]
Sent: Thursday, June 04, 2015 5:46 PM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Hi All,
I have written a Oozie workflow which has a Sqoop action to pull data from Teradata and load into Hive table.
I have kerberos enabled hadoop cluster.
When I run Oozie workflwo I get below error
35521 [Thread-34] INFO org.apache.sqoop.hive.HiveImport -
35522 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
Heart beat
Heart beat
98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Delegation Token can be issued only with kerberos or web authentication
98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:6280)
I referred beelow link and made required changes as suggested in the linkhttps://prodlife.wordpress.com/2013/11/22/using-oozie-in-kerberized-cluster/
I'm missing anything here ?
please suggest
RE: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Posted by "Zhou, Richard" <ri...@intel.com>.
Hey,
Which version of Sqoop are you using?
Regards
Richard
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Teradata.com]
Sent: Thursday, June 04, 2015 5:46 PM
To: user@sqoop.apache.org
Subject: Oozie-sqoop action fails in Kerberos enabled hadoop cluster
Hi All,
I have written a Oozie workflow which has a Sqoop action to pull data from Teradata and load into Hive table.
I have kerberos enabled hadoop cluster.
When I run Oozie workflwo I get below error
35521 [Thread-34] INFO org.apache.sqoop.hive.HiveImport -
35522 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
Heart beat
Heart beat
98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Delegation Token can be issued only with kerberos or web authentication
98703 [Thread-34] INFO org.apache.sqoop.hive.HiveImport - at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:6280)
I referred beelow link and made required changes as suggested in the linkhttps://prodlife.wordpress.com/2013/11/22/using-oozie-in-kerberized-cluster/
I'm missing anything here ?
please suggest