You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Zhao Hong <zh...@gmail.com> on 2012/09/10 05:11:06 UTC
how to clean {mapred.local.dir}/taskTracker
Hi all
How to clean {mapred.local.dir}/taskTracker ？ It's about 32 G and I must
restart the cluser to clean it .
Thanks & Best Regards
hong
Re: how to clean {mapred.local.dir}/taskTracker
Posted by Zhao Hong <zh...@gmail.com>.
Hi Harsh,
I looked up the directories. I do set the keep.failed.task.files=true
but It seems that my cluster stores all task files whatever failed or not.
Some completed job that have no any task failed are still in the directory.
Thanks.
Hong
On Mon, Sep 10, 2012 at 12:39 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi Zhao,
>
> The space should ideally be cleared up by itself. Can you inspect the
> directories to see which tasks seem to have left data over (they may
> have specifically done so via keep.failed.task.files=true) and if any
> of them are still running?
>
> On Mon, Sep 10, 2012 at 9:00 AM, Zhao Hong <zh...@gmail.com> wrote:
> > Thanks for your immediately reply . Are there some other ways that do
> not
> > interrupt the running of TaskTracker?
> >
> > On Mon, Sep 10, 2012 at 11:25 AM, Harsh J <ha...@cloudera.com> wrote:
> >
> >> You could just restart the specific tasktracker that has filled this
> >> directory; there's startup parts in the TT that clean up directories
> >> before beginning to serve out.
> >>
> >> On Mon, Sep 10, 2012 at 8:41 AM, Zhao Hong <zh...@gmail.com>
> wrote:
> >> > Hi all
> >> >
> >> > How to clean {mapred.local.dir}/taskTracker ？ It's about 32 G and I
> >> must
> >> > restart the cluser to clean it .
> >> >
> >> > Thanks & Best Regards
> >> >
> >> > hong
> >>
> >>
> >>
> >> --
> >> Harsh J
> >>
>
>
>
> --
> Harsh J
>
Re: how to clean {mapred.local.dir}/taskTracker
Posted by Harsh J <ha...@cloudera.com>.
Hi Zhao,
The space should ideally be cleared up by itself. Can you inspect the
directories to see which tasks seem to have left data over (they may
have specifically done so via keep.failed.task.files=true) and if any
of them are still running?
On Mon, Sep 10, 2012 at 9:00 AM, Zhao Hong <zh...@gmail.com> wrote:
> Thanks for your immediately reply . Are there some other ways that do not
> interrupt the running of TaskTracker?
>
> On Mon, Sep 10, 2012 at 11:25 AM, Harsh J <ha...@cloudera.com> wrote:
>
>> You could just restart the specific tasktracker that has filled this
>> directory; there's startup parts in the TT that clean up directories
>> before beginning to serve out.
>>
>> On Mon, Sep 10, 2012 at 8:41 AM, Zhao Hong <zh...@gmail.com> wrote:
>> > Hi all
>> >
>> > How to clean {mapred.local.dir}/taskTracker ？ It's about 32 G and I
>> must
>> > restart the cluser to clean it .
>> >
>> > Thanks & Best Regards
>> >
>> > hong
>>
>>
>>
>> --
>> Harsh J
>>
--
Harsh J
Re: how to clean {mapred.local.dir}/taskTracker
Posted by Zhao Hong <zh...@gmail.com>.
Thanks for your immediately reply . Are there some other ways that do not
interrupt the running of TaskTracker?
On Mon, Sep 10, 2012 at 11:25 AM, Harsh J <ha...@cloudera.com> wrote:
> You could just restart the specific tasktracker that has filled this
> directory; there's startup parts in the TT that clean up directories
> before beginning to serve out.
>
> On Mon, Sep 10, 2012 at 8:41 AM, Zhao Hong <zh...@gmail.com> wrote:
> > Hi all
> >
> > How to clean {mapred.local.dir}/taskTracker ？ It's about 32 G and I
> must
> > restart the cluser to clean it .
> >
> > Thanks & Best Regards
> >
> > hong
>
>
>
> --
> Harsh J
>
Re: how to clean {mapred.local.dir}/taskTracker
Posted by Harsh J <ha...@cloudera.com>.
You could just restart the specific tasktracker that has filled this
directory; there's startup parts in the TT that clean up directories
before beginning to serve out.
On Mon, Sep 10, 2012 at 8:41 AM, Zhao Hong <zh...@gmail.com> wrote:
> Hi all
>
> How to clean {mapred.local.dir}/taskTracker ？ It's about 32 G and I must
> restart the cluser to clean it .
>
> Thanks & Best Regards
>
> hong
--
Harsh J