You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Robert J Berger <rb...@runa.com> on 2011/08/09 10:34:14 UTC
Is it ok to manually delta ~hadoop/mapred/local/taskTracker/archive/*
Looks like I have something not configured particularly well so that mapred/local/taskTracker/archive is a local filesystem and its filling things up.
Is it ok to delete mapred/local/taskTracker/archive/* at the Unix level? Or is there some other way to force that to be deleted. I can't really restart my hadoop cluster just to fix this right n ow. I'm running hadoop 0.20.1.
Thanks!
Rob
__________________
Robert J Berger - CTO
Runa Inc.
+1 408-838-8896
http://blog.ibd.com
Re: Is it ok to manually delta ~hadoop/mapred/local/taskTracker/archive/*
Posted by Robert J Berger <rb...@runa.com>.
Great, I can restart the tasktracker with no issue. Glad I didn't manually delete the files! I'll give your suggestions a try.
We're (still) working on migrating to the latest and greatest versions of hadoop/hbase. (Have to robustly backup our HBase first. We're almost there)
Thanks for the help!
Rob
On Aug 9, 2011, at 8:45 AM, Owen O'Malley wrote:
> On Tue, Aug 9, 2011 at 8:34 AM, Robert J Berger <rb...@runa.com> wrote:
>
>> Looks like I have something not configured particularly well so that
>> mapred/local/taskTracker/archive is a local filesystem and its filling
>> things up.
>>
>
> Configure the size of the distributed cache on each node using
> local.cache.size, which defaults to 10gb.
>
>
>> Is it ok to delete mapred/local/taskTracker/archive/* at the Unix level? Or
>> is there some other way to force that to be deleted.
>
>
> If you restart the task tracker, I believe it will delete it. You shouldn't
> delete it behind the scenes, because you'll cause failures for any running
> tasks and confuse the task tracker with what it has stored.
>
>
>> I can't really restart my hadoop cluster just to fix this right n ow. I'm
>> running hadoop 0.20.1.
>
>
> I'd highly recommend you upgrade to 0.20.203.0.
>
> -- Owen
__________________
Robert J Berger - CTO
Runa Inc.
+1 408-838-8896
http://blog.ibd.com
Re: Is it ok to manually delta ~hadoop/mapred/local/taskTracker/archive/*
Posted by Owen O'Malley <ow...@hortonworks.com>.
On Tue, Aug 9, 2011 at 8:34 AM, Robert J Berger <rb...@runa.com> wrote:
> Looks like I have something not configured particularly well so that
> mapred/local/taskTracker/archive is a local filesystem and its filling
> things up.
>
Configure the size of the distributed cache on each node using
local.cache.size, which defaults to 10gb.
> Is it ok to delete mapred/local/taskTracker/archive/* at the Unix level? Or
> is there some other way to force that to be deleted.
If you restart the task tracker, I believe it will delete it. You shouldn't
delete it behind the scenes, because you'll cause failures for any running
tasks and confuse the task tracker with what it has stored.
> I can't really restart my hadoop cluster just to fix this right n ow. I'm
> running hadoop 0.20.1.
I'd highly recommend you upgrade to 0.20.203.0.
-- Owen