You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Adrian Liu <ad...@yahoo-inc.com> on 2012/06/26 09:56:21 UTC
Can I remove a folder on HDFS when decommissioning a data node?
Hi,
I tried to decommission a datanode, and then found that the active nodes don't have enough space to store the replicated blocks on the decommissioning node. So can I remove a big folder when the decommissioning is in progress? Will this causes some fatal errors? Thanks in advance.
Adrian Liu
adrianl@yahoo-inc.com
Re: Can I remove a folder on HDFS when decommissioning a data node?
Posted by Nitin Pawar <ni...@gmail.com>.
use -skipTrash while removing that way it wont be stored in .Trash
On Tue, Jun 26, 2012 at 8:21 PM, Michael Segel <mi...@hotmail.com>wrote:
> Hi,
>
> Yes you can remove a file while there is a node or node(s) being
> decommissioned.
>
> I wonder if there's a way to manually clear out the .trash which may also
> give you more space.
>
> On Jun 26, 2012, at 2:56 AM, Adrian Liu wrote:
>
> > Hi,
> >
> > I tried to decommission a datanode, and then found that the active nodes
> don't have enough space to store the replicated blocks on the
> decommissioning node. So can I remove a big folder when the
> decommissioning is in progress? Will this causes some fatal errors?
> Thanks in advance.
> >
> > Adrian Liu
> > adrianl@yahoo-inc.com
> >
> >
> >
> >
>
>
--
Nitin Pawar
Re: Can I remove a folder on HDFS when decommissioning a data node?
Posted by Harsh J <ha...@cloudera.com>.
Filed https://issues.apache.org/jira/browse/HDFS-3567
On Tue, Jun 26, 2012 at 9:05 PM, Michael Segel
<mi...@hotmail.com> wrote:
> Well, there you go... care to open a Jira for it?
> It would be very helpful... The OP's question presents a very good use case.
>
> Thx
>
> -Mike
>
> On Jun 26, 2012, at 10:00 AM, Harsh J wrote:
>
>> Hi,
>>
>> On Tue, Jun 26, 2012 at 8:21 PM, Michael Segel
>> <mi...@hotmail.com> wrote:
>>> Hi,
>>>
>>> Yes you can remove a file while there is a node or node(s) being decommissioned.
>>>
>>> I wonder if there's a way to manually clear out the .trash which may also give you more space.
>>
>> There's no way to do that across all users presently (for each user,
>> -expunge works though), other than running a manual rm (-skipTrash)
>> command for the .Trash files under each /user/* directory. I suppose
>> we can add in an admin command to clear out all trash manually
>> (forcing, other than relying on the periodic auto-emptier thread).
>>
>> --
>> Harsh J
>>
>
--
Harsh J
Re: Can I remove a folder on HDFS when decommissioning a data node?
Posted by Michael Segel <mi...@hotmail.com>.
Well, there you go... care to open a Jira for it?
It would be very helpful... The OP's question presents a very good use case.
Thx
-Mike
On Jun 26, 2012, at 10:00 AM, Harsh J wrote:
> Hi,
>
> On Tue, Jun 26, 2012 at 8:21 PM, Michael Segel
> <mi...@hotmail.com> wrote:
>> Hi,
>>
>> Yes you can remove a file while there is a node or node(s) being decommissioned.
>>
>> I wonder if there's a way to manually clear out the .trash which may also give you more space.
>
> There's no way to do that across all users presently (for each user,
> -expunge works though), other than running a manual rm (-skipTrash)
> command for the .Trash files under each /user/* directory. I suppose
> we can add in an admin command to clear out all trash manually
> (forcing, other than relying on the periodic auto-emptier thread).
>
> --
> Harsh J
>
Re: Can I remove a folder on HDFS when decommissioning a data node?
Posted by Harsh J <ha...@cloudera.com>.
Hi,
On Tue, Jun 26, 2012 at 8:21 PM, Michael Segel
<mi...@hotmail.com> wrote:
> Hi,
>
> Yes you can remove a file while there is a node or node(s) being decommissioned.
>
> I wonder if there's a way to manually clear out the .trash which may also give you more space.
There's no way to do that across all users presently (for each user,
-expunge works though), other than running a manual rm (-skipTrash)
command for the .Trash files under each /user/* directory. I suppose
we can add in an admin command to clear out all trash manually
(forcing, other than relying on the periodic auto-emptier thread).
--
Harsh J
Re: Can I remove a folder on HDFS when decommissioning a data node?
Posted by Michael Segel <mi...@hotmail.com>.
Hi,
Yes you can remove a file while there is a node or node(s) being decommissioned.
I wonder if there's a way to manually clear out the .trash which may also give you more space.
On Jun 26, 2012, at 2:56 AM, Adrian Liu wrote:
> Hi,
>
> I tried to decommission a datanode, and then found that the active nodes don't have enough space to store the replicated blocks on the decommissioning node. So can I remove a big folder when the decommissioning is in progress? Will this causes some fatal errors? Thanks in advance.
>
> Adrian Liu
> adrianl@yahoo-inc.com
>
>
>
>