You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by ipshita chatterji <si...@gmail.com> on 2011/08/25 10:30:15 UTC
How to delete a file in Pigscript
Hi Folks,
I want to delete a file "xyz.tmp" from my hdfs location below:
hdfs://MASTER/user/test/xyz.tmp
I have embedded the following statement in my pigscript:
--a.pig
fs -rmr 'xyz.tmp';
Everytime I execute the script, I get following error:
2011-08-24 22:00:44,722 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 6000: Output Location Validation Failed for:
'hdfs://MASTER/user/test/xyz.tmp More info to follow:
and the file is not deleted.
But when I execute this command from grunt shell, it is deleted
successfully.
Can anyone suggest why is it so? How to delete it from pig script?
Thanks ,
Ipshita
Re: How to delete a file in Pigscript
Posted by Daniel Dai <da...@hortonworks.com>.
The error message is not generated by "fs -rmr". It should be
generated by "store" in the case the output file already exists.
Daniel
On Thu, Aug 25, 2011 at 2:59 AM, ipshita chatterji <si...@gmail.com> wrote:
> Hi Marek,
> I use the following command to invoke the pig script.
>
> pig -x mapreduce a.pig
>
> and the script has following:
>
> fs -rmr 'xyz.tmp';
>
> Thanks,
> Ipshita
>
> On Thu, Aug 25, 2011 at 2:36 PM, Marek Miglinski <mm...@seven.com>wrote:
>
>> Hi Ipshita,
>>
>> As you already working on hdfs file system, you don't need 'ls' command,
>> use:
>> rmf 'path';
>>
>> "rmf" is equivalent to rm -f in unix, that is, it does not error out if the
>> indicated file/directory is not present to
>> remove.
>>
>> I assume, if pig was started with -x local param, then rmf command will
>> target local folder ... ?
>>
>>
>> Sincerely,
>> Marek M.
>> ________________________________________
>> From: ipshita chatterji [sigmarekha@gmail.com]
>> Sent: Thursday, August 25, 2011 11:30 AM
>> To: user@pig.apache.org
>> Subject: How to delete a file in Pigscript
>>
>> Hi Folks,
>> I want to delete a file "xyz.tmp" from my hdfs location below:
>>
>> hdfs://MASTER/user/test/xyz.tmp
>>
>> I have embedded the following statement in my pigscript:
>>
>> --a.pig
>>
>> fs -rmr 'xyz.tmp';
>>
>> Everytime I execute the script, I get following error:
>> 2011-08-24 22:00:44,722 [main] ERROR org.apache.pig.tools.grunt.Grunt -
>> ERROR 6000: Output Location Validation Failed for:
>> 'hdfs://MASTER/user/test/xyz.tmp More info to follow:
>>
>> and the file is not deleted.
>>
>> But when I execute this command from grunt shell, it is deleted
>> successfully.
>>
>> Can anyone suggest why is it so? How to delete it from pig script?
>>
>> Thanks ,
>> Ipshita
>>
>
Re: How to delete a file in Pigscript
Posted by ipshita chatterji <si...@gmail.com>.
Hi Marek,
I use the following command to invoke the pig script.
pig -x mapreduce a.pig
and the script has following:
fs -rmr 'xyz.tmp';
Thanks,
Ipshita
On Thu, Aug 25, 2011 at 2:36 PM, Marek Miglinski <mm...@seven.com>wrote:
> Hi Ipshita,
>
> As you already working on hdfs file system, you don't need 'ls' command,
> use:
> rmf 'path';
>
> "rmf" is equivalent to rm -f in unix, that is, it does not error out if the
> indicated file/directory is not present to
> remove.
>
> I assume, if pig was started with -x local param, then rmf command will
> target local folder ... ?
>
>
> Sincerely,
> Marek M.
> ________________________________________
> From: ipshita chatterji [sigmarekha@gmail.com]
> Sent: Thursday, August 25, 2011 11:30 AM
> To: user@pig.apache.org
> Subject: How to delete a file in Pigscript
>
> Hi Folks,
> I want to delete a file "xyz.tmp" from my hdfs location below:
>
> hdfs://MASTER/user/test/xyz.tmp
>
> I have embedded the following statement in my pigscript:
>
> --a.pig
>
> fs -rmr 'xyz.tmp';
>
> Everytime I execute the script, I get following error:
> 2011-08-24 22:00:44,722 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 6000: Output Location Validation Failed for:
> 'hdfs://MASTER/user/test/xyz.tmp More info to follow:
>
> and the file is not deleted.
>
> But when I execute this command from grunt shell, it is deleted
> successfully.
>
> Can anyone suggest why is it so? How to delete it from pig script?
>
> Thanks ,
> Ipshita
>
RE: How to delete a file in Pigscript
Posted by Marek Miglinski <mm...@seven.com>.
Hi Ipshita,
As you already working on hdfs file system, you don't need 'ls' command, use:
rmf 'path';
"rmf" is equivalent to rm -f in unix, that is, it does not error out if the indicated file/directory is not present to
remove.
I assume, if pig was started with -x local param, then rmf command will target local folder ... ?
Sincerely,
Marek M.
________________________________________
From: ipshita chatterji [sigmarekha@gmail.com]
Sent: Thursday, August 25, 2011 11:30 AM
To: user@pig.apache.org
Subject: How to delete a file in Pigscript
Hi Folks,
I want to delete a file "xyz.tmp" from my hdfs location below:
hdfs://MASTER/user/test/xyz.tmp
I have embedded the following statement in my pigscript:
--a.pig
fs -rmr 'xyz.tmp';
Everytime I execute the script, I get following error:
2011-08-24 22:00:44,722 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 6000: Output Location Validation Failed for:
'hdfs://MASTER/user/test/xyz.tmp More info to follow:
and the file is not deleted.
But when I execute this command from grunt shell, it is deleted
successfully.
Can anyone suggest why is it so? How to delete it from pig script?
Thanks ,
Ipshita