You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Xibin Liu <xi...@gmail.com> on 2013/01/24 08:21:08 UTC
hdfs du periodicity and hdfs not respond at that time
hi all,
I found hdfs du periodicity(one hour), and because my disk is big, the
smallest one is 15T, so when hdfs exec du, datanode will not respond for
about 3 minuts because of io loading, this cause a lot of problem, anybody
knows why hdfs doing this and how to disable it?
--
Thanks & Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Xibin Liu <xi...@gmail.com>.
I'm using ext3 and use df instead of du is a good way to solve this
problem, thanks you all
2013/1/24 Harsh J <ha...@cloudera.com>
> I missed the periodicity part of your question. Unfortunately the "du"
> refresh interval is hard-coded today, although the "df" interval is
> configurable. Perhaps this is a bug - I filed
> https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
>
> Also, your problem reminded me of a similar issue my team and I faced
> once before, and
>
> http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
> helped us temporarily there. Perhaps it may help you as well, its a
> good tip.
>
> On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <xi...@gmail.com> wrote:
> > Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> > dont't know how to solve this problem, 3 minutes not respond once an hour
> > is a big problem for me, any clue for this?
> >
> >
> > 2013/1/24 Harsh J <ha...@cloudera.com>
> >>
> >> Hi,
> >>
> >> HDFS does this to estimate space reports. Perhaps the discussion here
> >> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
> >>
> >> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
> >> wrote:
> >> > hi all,
> >> > I found hdfs du periodicity(one hour), and because my disk is big, the
> >> > smallest one is 15T, so when hdfs exec du, datanode will not respond
> for
> >> > about 3 minuts because of io loading, this cause a lot of problem,
> >> > anybody
> >> > knows why hdfs doing this and how to disable it?
> >> >
> >> > --
> >> > Thanks & Regards
> >> > Xibin Liu
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
> >
> >
> >
> >
> > --
> > Thanks & Best Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>
--
Thanks & Best Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Xibin Liu <xi...@gmail.com>.
I'm using ext3 and use df instead of du is a good way to solve this
problem, thanks you all
2013/1/24 Harsh J <ha...@cloudera.com>
> I missed the periodicity part of your question. Unfortunately the "du"
> refresh interval is hard-coded today, although the "df" interval is
> configurable. Perhaps this is a bug - I filed
> https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
>
> Also, your problem reminded me of a similar issue my team and I faced
> once before, and
>
> http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
> helped us temporarily there. Perhaps it may help you as well, its a
> good tip.
>
> On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <xi...@gmail.com> wrote:
> > Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> > dont't know how to solve this problem, 3 minutes not respond once an hour
> > is a big problem for me, any clue for this?
> >
> >
> > 2013/1/24 Harsh J <ha...@cloudera.com>
> >>
> >> Hi,
> >>
> >> HDFS does this to estimate space reports. Perhaps the discussion here
> >> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
> >>
> >> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
> >> wrote:
> >> > hi all,
> >> > I found hdfs du periodicity(one hour), and because my disk is big, the
> >> > smallest one is 15T, so when hdfs exec du, datanode will not respond
> for
> >> > about 3 minuts because of io loading, this cause a lot of problem,
> >> > anybody
> >> > knows why hdfs doing this and how to disable it?
> >> >
> >> > --
> >> > Thanks & Regards
> >> > Xibin Liu
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
> >
> >
> >
> >
> > --
> > Thanks & Best Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>
--
Thanks & Best Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Xibin Liu <xi...@gmail.com>.
I'm using ext3 and use df instead of du is a good way to solve this
problem, thanks you all
2013/1/24 Harsh J <ha...@cloudera.com>
> I missed the periodicity part of your question. Unfortunately the "du"
> refresh interval is hard-coded today, although the "df" interval is
> configurable. Perhaps this is a bug - I filed
> https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
>
> Also, your problem reminded me of a similar issue my team and I faced
> once before, and
>
> http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
> helped us temporarily there. Perhaps it may help you as well, its a
> good tip.
>
> On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <xi...@gmail.com> wrote:
> > Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> > dont't know how to solve this problem, 3 minutes not respond once an hour
> > is a big problem for me, any clue for this?
> >
> >
> > 2013/1/24 Harsh J <ha...@cloudera.com>
> >>
> >> Hi,
> >>
> >> HDFS does this to estimate space reports. Perhaps the discussion here
> >> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
> >>
> >> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
> >> wrote:
> >> > hi all,
> >> > I found hdfs du periodicity(one hour), and because my disk is big, the
> >> > smallest one is 15T, so when hdfs exec du, datanode will not respond
> for
> >> > about 3 minuts because of io loading, this cause a lot of problem,
> >> > anybody
> >> > knows why hdfs doing this and how to disable it?
> >> >
> >> > --
> >> > Thanks & Regards
> >> > Xibin Liu
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
> >
> >
> >
> >
> > --
> > Thanks & Best Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>
--
Thanks & Best Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Xibin Liu <xi...@gmail.com>.
I'm using ext3 and use df instead of du is a good way to solve this
problem, thanks you all
2013/1/24 Harsh J <ha...@cloudera.com>
> I missed the periodicity part of your question. Unfortunately the "du"
> refresh interval is hard-coded today, although the "df" interval is
> configurable. Perhaps this is a bug - I filed
> https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
>
> Also, your problem reminded me of a similar issue my team and I faced
> once before, and
>
> http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
> helped us temporarily there. Perhaps it may help you as well, its a
> good tip.
>
> On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <xi...@gmail.com> wrote:
> > Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> > dont't know how to solve this problem, 3 minutes not respond once an hour
> > is a big problem for me, any clue for this?
> >
> >
> > 2013/1/24 Harsh J <ha...@cloudera.com>
> >>
> >> Hi,
> >>
> >> HDFS does this to estimate space reports. Perhaps the discussion here
> >> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
> >>
> >> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
> >> wrote:
> >> > hi all,
> >> > I found hdfs du periodicity(one hour), and because my disk is big, the
> >> > smallest one is 15T, so when hdfs exec du, datanode will not respond
> for
> >> > about 3 minuts because of io loading, this cause a lot of problem,
> >> > anybody
> >> > knows why hdfs doing this and how to disable it?
> >> >
> >> > --
> >> > Thanks & Regards
> >> > Xibin Liu
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
> >
> >
> >
> >
> > --
> > Thanks & Best Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>
--
Thanks & Best Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Harsh J <ha...@cloudera.com>.
I missed the periodicity part of your question. Unfortunately the "du"
refresh interval is hard-coded today, although the "df" interval is
configurable. Perhaps this is a bug - I filed
https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
Also, your problem reminded me of a similar issue my team and I faced
once before, and
http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
helped us temporarily there. Perhaps it may help you as well, its a
good tip.
On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <xi...@gmail.com> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <ha...@cloudera.com>
>>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> > anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks & Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>
--
Harsh J
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Chris Embree <ce...@gmail.com>.
What type of FS are you using under HDFS? XFS, ext3, ext4? The type and
configuration of the underlying FS will impact performance.
Most notably, ext3 has a lock-up effect when flushing disk cache.
On Thu, Jan 24, 2013 at 2:54 AM, Xibin Liu <xi...@gmail.com> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <ha...@cloudera.com>
>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks & Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>
>
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Harsh J <ha...@cloudera.com>.
I missed the periodicity part of your question. Unfortunately the "du"
refresh interval is hard-coded today, although the "df" interval is
configurable. Perhaps this is a bug - I filed
https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
Also, your problem reminded me of a similar issue my team and I faced
once before, and
http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
helped us temporarily there. Perhaps it may help you as well, its a
good tip.
On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <xi...@gmail.com> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <ha...@cloudera.com>
>>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> > anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks & Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>
--
Harsh J
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Harsh J <ha...@cloudera.com>.
I missed the periodicity part of your question. Unfortunately the "du"
refresh interval is hard-coded today, although the "df" interval is
configurable. Perhaps this is a bug - I filed
https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
Also, your problem reminded me of a similar issue my team and I faced
once before, and
http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
helped us temporarily there. Perhaps it may help you as well, its a
good tip.
On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <xi...@gmail.com> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <ha...@cloudera.com>
>>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> > anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks & Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>
--
Harsh J
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Chris Embree <ce...@gmail.com>.
What type of FS are you using under HDFS? XFS, ext3, ext4? The type and
configuration of the underlying FS will impact performance.
Most notably, ext3 has a lock-up effect when flushing disk cache.
On Thu, Jan 24, 2013 at 2:54 AM, Xibin Liu <xi...@gmail.com> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <ha...@cloudera.com>
>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks & Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>
>
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Chris Embree <ce...@gmail.com>.
What type of FS are you using under HDFS? XFS, ext3, ext4? The type and
configuration of the underlying FS will impact performance.
Most notably, ext3 has a lock-up effect when flushing disk cache.
On Thu, Jan 24, 2013 at 2:54 AM, Xibin Liu <xi...@gmail.com> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <ha...@cloudera.com>
>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks & Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>
>
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Harsh J <ha...@cloudera.com>.
I missed the periodicity part of your question. Unfortunately the "du"
refresh interval is hard-coded today, although the "df" interval is
configurable. Perhaps this is a bug - I filed
https://issues.apache.org/jira/browse/HADOOP-9241 to make it configurable.
Also, your problem reminded me of a similar issue my team and I faced
once before, and
http://allthingshadoop.com/2011/05/20/faster-datanodes-with-less-wait-io-using-df-instead-of-du/
helped us temporarily there. Perhaps it may help you as well, its a
good tip.
On Thu, Jan 24, 2013 at 1:24 PM, Xibin Liu <xi...@gmail.com> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <ha...@cloudera.com>
>>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> > anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks & Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>
--
Harsh J
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Chris Embree <ce...@gmail.com>.
What type of FS are you using under HDFS? XFS, ext3, ext4? The type and
configuration of the underlying FS will impact performance.
Most notably, ext3 has a lock-up effect when flushing disk cache.
On Thu, Jan 24, 2013 at 2:54 AM, Xibin Liu <xi...@gmail.com> wrote:
> Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
> dont't know how to solve this problem, 3 minutes not respond once an hour
> is a big problem for me, any clue for this?
>
>
> 2013/1/24 Harsh J <ha...@cloudera.com>
>
>> Hi,
>>
>> HDFS does this to estimate space reports. Perhaps the discussion here
>> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>>
>> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
>> wrote:
>> > hi all,
>> > I found hdfs du periodicity(one hour), and because my disk is big, the
>> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
>> > about 3 minuts because of io loading, this cause a lot of problem,
>> anybody
>> > knows why hdfs doing this and how to disable it?
>> >
>> > --
>> > Thanks & Regards
>> > Xibin Liu
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Best Regards
> Xibin Liu
>
>
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Xibin Liu <xi...@gmail.com>.
Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
dont't know how to solve this problem, 3 minutes not respond once an hour
is a big problem for me, any clue for this?
2013/1/24 Harsh J <ha...@cloudera.com>
> Hi,
>
> HDFS does this to estimate space reports. Perhaps the discussion here
> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>
> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
> wrote:
> > hi all,
> > I found hdfs du periodicity(one hour), and because my disk is big, the
> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
> > about 3 minuts because of io loading, this cause a lot of problem,
> anybody
> > knows why hdfs doing this and how to disable it?
> >
> > --
> > Thanks & Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>
--
Thanks & Best Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Xibin Liu <xi...@gmail.com>.
Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
dont't know how to solve this problem, 3 minutes not respond once an hour
is a big problem for me, any clue for this?
2013/1/24 Harsh J <ha...@cloudera.com>
> Hi,
>
> HDFS does this to estimate space reports. Perhaps the discussion here
> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>
> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
> wrote:
> > hi all,
> > I found hdfs du periodicity(one hour), and because my disk is big, the
> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
> > about 3 minuts because of io loading, this cause a lot of problem,
> anybody
> > knows why hdfs doing this and how to disable it?
> >
> > --
> > Thanks & Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>
--
Thanks & Best Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Xibin Liu <xi...@gmail.com>.
Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
dont't know how to solve this problem, 3 minutes not respond once an hour
is a big problem for me, any clue for this?
2013/1/24 Harsh J <ha...@cloudera.com>
> Hi,
>
> HDFS does this to estimate space reports. Perhaps the discussion here
> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>
> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
> wrote:
> > hi all,
> > I found hdfs du periodicity(one hour), and because my disk is big, the
> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
> > about 3 minuts because of io loading, this cause a lot of problem,
> anybody
> > knows why hdfs doing this and how to disable it?
> >
> > --
> > Thanks & Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>
--
Thanks & Best Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Xibin Liu <xi...@gmail.com>.
Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
dont't know how to solve this problem, 3 minutes not respond once an hour
is a big problem for me, any clue for this?
2013/1/24 Harsh J <ha...@cloudera.com>
> Hi,
>
> HDFS does this to estimate space reports. Perhaps the discussion here
> may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
>
> On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com>
> wrote:
> > hi all,
> > I found hdfs du periodicity(one hour), and because my disk is big, the
> > smallest one is 15T, so when hdfs exec du, datanode will not respond for
> > about 3 minuts because of io loading, this cause a lot of problem,
> anybody
> > knows why hdfs doing this and how to disable it?
> >
> > --
> > Thanks & Regards
> > Xibin Liu
> >
>
>
>
> --
> Harsh J
>
--
Thanks & Best Regards
Xibin Liu
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Harsh J <ha...@cloudera.com>.
Hi,
HDFS does this to estimate space reports. Perhaps the discussion here
may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com> wrote:
> hi all,
> I found hdfs du periodicity(one hour), and because my disk is big, the
> smallest one is 15T, so when hdfs exec du, datanode will not respond for
> about 3 minuts because of io loading, this cause a lot of problem, anybody
> knows why hdfs doing this and how to disable it?
>
> --
> Thanks & Regards
> Xibin Liu
>
--
Harsh J
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Harsh J <ha...@cloudera.com>.
Hi,
HDFS does this to estimate space reports. Perhaps the discussion here
may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com> wrote:
> hi all,
> I found hdfs du periodicity(one hour), and because my disk is big, the
> smallest one is 15T, so when hdfs exec du, datanode will not respond for
> about 3 minuts because of io loading, this cause a lot of problem, anybody
> knows why hdfs doing this and how to disable it?
>
> --
> Thanks & Regards
> Xibin Liu
>
--
Harsh J
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Harsh J <ha...@cloudera.com>.
Hi,
HDFS does this to estimate space reports. Perhaps the discussion here
may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com> wrote:
> hi all,
> I found hdfs du periodicity(one hour), and because my disk is big, the
> smallest one is 15T, so when hdfs exec du, datanode will not respond for
> about 3 minuts because of io loading, this cause a lot of problem, anybody
> knows why hdfs doing this and how to disable it?
>
> --
> Thanks & Regards
> Xibin Liu
>
--
Harsh J
Re: hdfs du periodicity and hdfs not respond at that time
Posted by Harsh J <ha...@cloudera.com>.
Hi,
HDFS does this to estimate space reports. Perhaps the discussion here
may help you: http://search-hadoop.com/m/LLBgUiH0Bg2
On Thu, Jan 24, 2013 at 12:51 PM, Xibin Liu <xi...@gmail.com> wrote:
> hi all,
> I found hdfs du periodicity(one hour), and because my disk is big, the
> smallest one is 15T, so when hdfs exec du, datanode will not respond for
> about 3 minuts because of io loading, this cause a lot of problem, anybody
> knows why hdfs doing this and how to disable it?
>
> --
> Thanks & Regards
> Xibin Liu
>
--
Harsh J