You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by amit kumar <ak...@gmail.com> on 2015/05/04 23:42:15 UTC
Unable to move files on Hive/Hdfs
While moving the data from hive/hdfs we get below error,
Please suggest on this.
Moving data to:
hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
Failed with exception Unable to move
sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
235-1/-ext-10002 to destination
hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
10000
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS
Write: 489704 SUCCESS
Total MapReduce CPU Time Spent: 5 seconds 830 msec
Error (1). Execution Failed.
2015-05-04 10:03:13 ERROR (1) in run_hive
Thanks,
Re: Unable to move files on Hive/Hdfs
Posted by "Moore, Douglas" <Do...@thinkbiganalytics.com>.
I've seen where hdfs:///tmp permissions were not compatible with the user running hive.
To fix that look at changing the perms on hdfs:///tmp
- Douglas
From: amit kumar <ak...@gmail.com>>
Reply-To: <us...@hive.apache.org>>
Date: Tue, 5 May 2015 04:48:58 +0530
To: <us...@hive.apache.org>>
Subject: Re: Unable to move files on Hive/Hdfs
Doug,
Do i need any changes in configuration or else to resolve this issue.
Thanks
On Tue, May 5, 2015 at 4:46 AM, amit kumar <ak...@gmail.com>> wrote:
Do you have any suggestion to resolve this issue,
I am looking for a resolution.
On Tue, May 5, 2015 at 4:42 AM, Moore, Douglas <Do...@thinkbiganalytics.com>> wrote:
Yep, permission problem. Weird though it seems to be moving a file within the same dir.
Thanks for the update!
- Douglas
From: amit kumar <ak...@gmail.com>>
Reply-To: <us...@hive.apache.org>>
Date: Tue, 5 May 2015 04:40:18 +0530
To: <us...@hive.apache.org>>
Subject: Re: Unable to move files on Hive/Hdfs
Hi Doug,
I have use CDH 5.2.1
I performed the below task, and getting the error, but after rolling back the below changes issue has been resolved itself.
Disable ACLs on Name Nodes
Set Enable Access Control Lists = False
Save Changes
Restart Hadoop Cluster
Thanks,
On Tue, May 5, 2015 at 4:36 AM, amit kumar <ak...@gmail.com>> wrote:
Hi Doug,
I have use CDH 5.2.1
Disable ACLs on Name Nodes
Set Enable Access Control Lists = False
Save Changes
Restart Hadoop Cluster
Stack trace:
2015-05-04 10:38:18,820 INFO [main]: exec.Task (SessionState.java:printInfo(537)) - Moving data to: hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000 from hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
2015-05-04 10:38:18,857 ERROR [main]: exec.Task (SessionState.java:printError(546)) - Failed with exception Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
at org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269)
at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89)
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false.
at org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553)
After rolling those same changes out, the problem resolved itself.
On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <Do...@thinkbiganalytics.com>> wrote:
Hi Amit,
We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2 release. There are lots of reasons for this though.
Can you provide more details about the stack trace and version so we can compare?
For our problem we've seen some relief with SET hive.metastore.client.socket.timeout=60s
but the problem still happens from time to time.
Thanks,
Douglas
From: amit kumar <ak...@gmail.com>>
Reply-To: <us...@hive.apache.org>>
Date: Tue, 5 May 2015 03:12:15 +0530
To: <us...@hive.apache.org>>
Subject: Unable to move files on Hive/Hdfs
While moving the data from hive/hdfs we get below error,
Please suggest on this.
Moving data to: hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
Failed with exception Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
235-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
10000
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS Write: 489704 SUCCESS
Total MapReduce CPU Time Spent: 5 seconds 830 msec
Error (1). Execution Failed.
2015-05-04 10:03:13 ERROR (1) in run_hive
Thanks,
Re: Unable to move files on Hive/Hdfs
Posted by amit kumar <ak...@gmail.com>.
Doug,
Do i need any changes in configuration or else to resolve this issue.
Thanks
On Tue, May 5, 2015 at 4:46 AM, amit kumar <ak...@gmail.com> wrote:
> Do you have any suggestion to resolve this issue,
>
> I am looking for a resolution.
>
> On Tue, May 5, 2015 at 4:42 AM, Moore, Douglas <
> Douglas.Moore@thinkbiganalytics.com> wrote:
>
>> Yep, permission problem. Weird though it seems to be moving a file
>> within the same dir.
>>
>> Thanks for the update!
>>
>> - Douglas
>> From: amit kumar <ak...@gmail.com>
>> Reply-To: <us...@hive.apache.org>
>> Date: Tue, 5 May 2015 04:40:18 +0530
>> To: <us...@hive.apache.org>
>> Subject: Re: Unable to move files on Hive/Hdfs
>>
>> Hi Doug,
>>
>> I have use CDH 5.2.1
>>
>> I performed the below task, and getting the error, but after rolling
>> back the below changes issue has been resolved itself.
>>
>> Disable ACLs on Name Nodes
>> Set Enable Access Control Lists = False
>> Save Changes
>> Restart Hadoop Cluster
>>
>>
>> Thanks,
>>
>>
>> On Tue, May 5, 2015 at 4:36 AM, amit kumar <ak...@gmail.com> wrote:
>>
>>> Hi Doug,
>>>
>>> I have use CDH 5.2.1
>>>
>>> Disable ACLs on Name Nodes
>>>
>>>
>>> Set Enable Access Control Lists = False
>>>
>>> Save Changes
>>>
>>> Restart Hadoop Cluster
>>>
>>>
>>>
>>> Stack trace:
>>>
>>> 2015-05-04 10:38:18,820 INFO [main]: exec.Task
>>> (SessionState.java:printInfo(537)) - Moving data to:
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>> from
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>>>
>>> 2015-05-04 10:38:18,857 ERROR [main]: exec.Task
>>> (SessionState.java:printError(546)) - Failed with exception Unable to move
>>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>>> to destination
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>>
>>> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
>>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>>> to destination
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>>
>>> at
>>> org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269)
>>>
>>> at
>>> org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89)
>>>
>>> at
>>> org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200)
>>>
>>> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>>>
>>> at
>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
>>>
>>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
>>>
>>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
>>>
>>> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
>>>
>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
>>>
>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
>>>
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
>>>
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
>>>
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
>>>
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
>>>
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
>>>
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
>>>
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
>>>
>>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
>>>
>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>>>
>>> Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL
>>> operation has been rejected. Support for ACLs has been disabled by setting
>>> dfs.namenode.acls.enabled to false.
>>>
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553)
>>>
>>>
>>> After rolling those same changes out, the problem resolved itself.
>>>
>>>
>>> On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <
>>> Douglas.Moore@thinkbiganalytics.com> wrote:
>>>
>>>> Hi Amit,
>>>>
>>>> We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2
>>>> release. There are lots of reasons for this though.
>>>> Can you provide more details about the stack trace and version so we
>>>> can compare?
>>>>
>>>> For our problem we've seen some relief with SET
>>>> hive.metastore.client.socket.timeout=60s
>>>> but the problem still happens from time to time.
>>>>
>>>> Thanks,
>>>> Douglas
>>>>
>>>> From: amit kumar <ak...@gmail.com>
>>>> Reply-To: <us...@hive.apache.org>
>>>> Date: Tue, 5 May 2015 03:12:15 +0530
>>>> To: <us...@hive.apache.org>
>>>> Subject: Unable to move files on Hive/Hdfs
>>>>
>>>> While moving the data from hive/hdfs we get below error,
>>>>
>>>> Please suggest on this.
>>>>
>>>> Moving data to:
>>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
>>>> Failed with exception Unable to move
>>>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
>>>> 235-1/-ext-10002 to destination
>>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
>>>> 10000
>>>> FAILED: Execution Error, return code 1 from
>>>> org.apache.hadoop.hive.ql.exec.MoveTask
>>>> MapReduce Jobs Launched:
>>>> Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS
>>>> Write: 489704 SUCCESS
>>>> Total MapReduce CPU Time Spent: 5 seconds 830 msec
>>>> Error (1). Execution Failed.
>>>> 2015-05-04 10:03:13 ERROR (1) in run_hive
>>>>
>>>> Thanks,
>>>>
>>>
>>>
>>
>
Re: Unable to move files on Hive/Hdfs
Posted by amit kumar <ak...@gmail.com>.
Do you have any suggestion to resolve this issue,
I am looking for a resolution.
On Tue, May 5, 2015 at 4:42 AM, Moore, Douglas <
Douglas.Moore@thinkbiganalytics.com> wrote:
> Yep, permission problem. Weird though it seems to be moving a file
> within the same dir.
>
> Thanks for the update!
>
> - Douglas
> From: amit kumar <ak...@gmail.com>
> Reply-To: <us...@hive.apache.org>
> Date: Tue, 5 May 2015 04:40:18 +0530
> To: <us...@hive.apache.org>
> Subject: Re: Unable to move files on Hive/Hdfs
>
> Hi Doug,
>
> I have use CDH 5.2.1
>
> I performed the below task, and getting the error, but after rolling
> back the below changes issue has been resolved itself.
>
> Disable ACLs on Name Nodes
> Set Enable Access Control Lists = False
> Save Changes
> Restart Hadoop Cluster
>
>
> Thanks,
>
>
> On Tue, May 5, 2015 at 4:36 AM, amit kumar <ak...@gmail.com> wrote:
>
>> Hi Doug,
>>
>> I have use CDH 5.2.1
>>
>> Disable ACLs on Name Nodes
>>
>>
>> Set Enable Access Control Lists = False
>>
>> Save Changes
>>
>> Restart Hadoop Cluster
>>
>>
>>
>> Stack trace:
>>
>> 2015-05-04 10:38:18,820 INFO [main]: exec.Task
>> (SessionState.java:printInfo(537)) - Moving data to:
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>> from
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>>
>> 2015-05-04 10:38:18,857 ERROR [main]: exec.Task
>> (SessionState.java:printError(546)) - Failed with exception Unable to move
>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>> to destination
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>
>> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>> to destination
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>
>> at
>> org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269)
>>
>> at
>> org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89)
>>
>> at
>> org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200)
>>
>> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>>
>> at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
>>
>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
>>
>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
>>
>> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
>>
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
>>
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
>>
>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
>>
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:606)
>>
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>>
>> Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL
>> operation has been rejected. Support for ACLs has been disabled by setting
>> dfs.namenode.acls.enabled to false.
>>
>> at
>> org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553)
>>
>>
>> After rolling those same changes out, the problem resolved itself.
>>
>>
>> On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <
>> Douglas.Moore@thinkbiganalytics.com> wrote:
>>
>>> Hi Amit,
>>>
>>> We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2
>>> release. There are lots of reasons for this though.
>>> Can you provide more details about the stack trace and version so we can
>>> compare?
>>>
>>> For our problem we've seen some relief with SET
>>> hive.metastore.client.socket.timeout=60s
>>> but the problem still happens from time to time.
>>>
>>> Thanks,
>>> Douglas
>>>
>>> From: amit kumar <ak...@gmail.com>
>>> Reply-To: <us...@hive.apache.org>
>>> Date: Tue, 5 May 2015 03:12:15 +0530
>>> To: <us...@hive.apache.org>
>>> Subject: Unable to move files on Hive/Hdfs
>>>
>>> While moving the data from hive/hdfs we get below error,
>>>
>>> Please suggest on this.
>>>
>>> Moving data to:
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
>>> Failed with exception Unable to move
>>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
>>> 235-1/-ext-10002 to destination
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
>>> 10000
>>> FAILED: Execution Error, return code 1 from
>>> org.apache.hadoop.hive.ql.exec.MoveTask
>>> MapReduce Jobs Launched:
>>> Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS
>>> Write: 489704 SUCCESS
>>> Total MapReduce CPU Time Spent: 5 seconds 830 msec
>>> Error (1). Execution Failed.
>>> 2015-05-04 10:03:13 ERROR (1) in run_hive
>>>
>>> Thanks,
>>>
>>
>>
>
Re: Unable to move files on Hive/Hdfs
Posted by "Moore, Douglas" <Do...@thinkbiganalytics.com>.
Yep, permission problem. Weird though it seems to be moving a file within the same dir.
Thanks for the update!
- Douglas
From: amit kumar <ak...@gmail.com>>
Reply-To: <us...@hive.apache.org>>
Date: Tue, 5 May 2015 04:40:18 +0530
To: <us...@hive.apache.org>>
Subject: Re: Unable to move files on Hive/Hdfs
Hi Doug,
I have use CDH 5.2.1
I performed the below task, and getting the error, but after rolling back the below changes issue has been resolved itself.
Disable ACLs on Name Nodes
Set Enable Access Control Lists = False
Save Changes
Restart Hadoop Cluster
Thanks,
On Tue, May 5, 2015 at 4:36 AM, amit kumar <ak...@gmail.com>> wrote:
Hi Doug,
I have use CDH 5.2.1
Disable ACLs on Name Nodes
Set Enable Access Control Lists = False
Save Changes
Restart Hadoop Cluster
Stack trace:
2015-05-04 10:38:18,820 INFO [main]: exec.Task (SessionState.java:printInfo(537)) - Moving data to: hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000 from hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
2015-05-04 10:38:18,857 ERROR [main]: exec.Task (SessionState.java:printError(546)) - Failed with exception Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
at org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269)
at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89)
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false.
at org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553)
After rolling those same changes out, the problem resolved itself.
On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <Do...@thinkbiganalytics.com>> wrote:
Hi Amit,
We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2 release. There are lots of reasons for this though.
Can you provide more details about the stack trace and version so we can compare?
For our problem we've seen some relief with SET hive.metastore.client.socket.timeout=60s
but the problem still happens from time to time.
Thanks,
Douglas
From: amit kumar <ak...@gmail.com>>
Reply-To: <us...@hive.apache.org>>
Date: Tue, 5 May 2015 03:12:15 +0530
To: <us...@hive.apache.org>>
Subject: Unable to move files on Hive/Hdfs
While moving the data from hive/hdfs we get below error,
Please suggest on this.
Moving data to: hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
Failed with exception Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
235-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
10000
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS Write: 489704 SUCCESS
Total MapReduce CPU Time Spent: 5 seconds 830 msec
Error (1). Execution Failed.
2015-05-04 10:03:13 ERROR (1) in run_hive
Thanks,
Re: Unable to move files on Hive/Hdfs
Posted by amit kumar <ak...@gmail.com>.
Hi Doug,
I have use CDH 5.2.1
I performed the below task, and getting the error, but after rolling back
the below changes issue has been resolved itself.
Disable ACLs on Name Nodes
Set Enable Access Control Lists = False
Save Changes
Restart Hadoop Cluster
Thanks,
On Tue, May 5, 2015 at 4:36 AM, amit kumar <ak...@gmail.com> wrote:
> Hi Doug,
>
> I have use CDH 5.2.1
>
> Disable ACLs on Name Nodes
>
>
> Set Enable Access Control Lists = False
>
> Save Changes
>
> Restart Hadoop Cluster
>
>
>
> Stack trace:
>
> 2015-05-04 10:38:18,820 INFO [main]: exec.Task
> (SessionState.java:printInfo(537)) - Moving data to:
> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
> from
> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>
> 2015-05-04 10:38:18,857 ERROR [main]: exec.Task
> (SessionState.java:printError(546)) - Failed with exception Unable to move
> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
> to destination
> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
> to destination
> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>
> at
> org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269)
>
> at
> org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89)
>
> at
> org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200)
>
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
>
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
>
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
>
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
>
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
>
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
>
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
>
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
>
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
>
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
>
> at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
>
> at
> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
>
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
>
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
>
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>
> Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL operation
> has been rejected. Support for ACLs has been disabled by setting
> dfs.namenode.acls.enabled to false.
>
> at
> org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553)
>
>
> After rolling those same changes out, the problem resolved itself.
>
>
> On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <
> Douglas.Moore@thinkbiganalytics.com> wrote:
>
>> Hi Amit,
>>
>> We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2 release.
>> There are lots of reasons for this though.
>> Can you provide more details about the stack trace and version so we can
>> compare?
>>
>> For our problem we've seen some relief with SET
>> hive.metastore.client.socket.timeout=60s
>> but the problem still happens from time to time.
>>
>> Thanks,
>> Douglas
>>
>> From: amit kumar <ak...@gmail.com>
>> Reply-To: <us...@hive.apache.org>
>> Date: Tue, 5 May 2015 03:12:15 +0530
>> To: <us...@hive.apache.org>
>> Subject: Unable to move files on Hive/Hdfs
>>
>> While moving the data from hive/hdfs we get below error,
>>
>> Please suggest on this.
>>
>> Moving data to:
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
>> Failed with exception Unable to move
>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
>> 235-1/-ext-10002 to destination
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
>> 10000
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.MoveTask
>> MapReduce Jobs Launched:
>> Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS
>> Write: 489704 SUCCESS
>> Total MapReduce CPU Time Spent: 5 seconds 830 msec
>> Error (1). Execution Failed.
>> 2015-05-04 10:03:13 ERROR (1) in run_hive
>>
>> Thanks,
>>
>
>
Re: Unable to move files on Hive/Hdfs
Posted by amit kumar <ak...@gmail.com>.
Hi Doug,
I have use CDH 5.2.1
Disable ACLs on Name Nodes
Set Enable Access Control Lists = False
Save Changes
Restart Hadoop Cluster
Stack trace:
2015-05-04 10:38:18,820 INFO [main]: exec.Task
(SessionState.java:printInfo(537)) - Moving data to:
hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
from
hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
2015-05-04 10:38:18,857 ERROR [main]: exec.Task
(SessionState.java:printError(546)) - Failed with exception Unable to move
sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
to destination
hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
to destination
hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
at
org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269)
at
org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89)
at
org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
at
org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
at
org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL operation
has been rejected. Support for ACLs has been disabled by setting
dfs.namenode.acls.enabled to false.
at
org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553)
After rolling those same changes out, the problem resolved itself.
On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <
Douglas.Moore@thinkbiganalytics.com> wrote:
> Hi Amit,
>
> We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2 release.
> There are lots of reasons for this though.
> Can you provide more details about the stack trace and version so we can
> compare?
>
> For our problem we've seen some relief with SET
> hive.metastore.client.socket.timeout=60s
> but the problem still happens from time to time.
>
> Thanks,
> Douglas
>
> From: amit kumar <ak...@gmail.com>
> Reply-To: <us...@hive.apache.org>
> Date: Tue, 5 May 2015 03:12:15 +0530
> To: <us...@hive.apache.org>
> Subject: Unable to move files on Hive/Hdfs
>
> While moving the data from hive/hdfs we get below error,
>
> Please suggest on this.
>
> Moving data to:
> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
> Failed with exception Unable to move
> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
> 235-1/-ext-10002 to destination
> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
> 10000
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask
> MapReduce Jobs Launched:
> Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS
> Write: 489704 SUCCESS
> Total MapReduce CPU Time Spent: 5 seconds 830 msec
> Error (1). Execution Failed.
> 2015-05-04 10:03:13 ERROR (1) in run_hive
>
> Thanks,
>
Re: Unable to move files on Hive/Hdfs
Posted by "Moore, Douglas" <Do...@thinkbiganalytics.com>.
Hi Amit,
We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2 release. There are lots of reasons for this though.
Can you provide more details about the stack trace and version so we can compare?
For our problem we've seen some relief with SET hive.metastore.client.socket.timeout=60s
but the problem still happens from time to time.
Thanks,
Douglas
From: amit kumar <ak...@gmail.com>>
Reply-To: <us...@hive.apache.org>>
Date: Tue, 5 May 2015 03:12:15 +0530
To: <us...@hive.apache.org>>
Subject: Unable to move files on Hive/Hdfs
While moving the data from hive/hdfs we get below error,
Please suggest on this.
Moving data to: hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
Failed with exception Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
235-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
10000
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS Write: 489704 SUCCESS
Total MapReduce CPU Time Spent: 5 seconds 830 msec
Error (1). Execution Failed.
2015-05-04 10:03:13 ERROR (1) in run_hive
Thanks,