You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by 风雨无阻 <23...@qq.com> on 2014/07/14 10:10:10 UTC
Block should be additionally replicated on 1 more rack(s)
HI all:
After the cluster configuration rack awareness,run " hadoop fsck / "
A lot of the following error occurred:
Replica placement policy is violated for blk_-1267324897180563985_11130670. Block should be additionally replicated on 1 more rack(s).
Online said "The reason is that three copies on the same rack" .
The solution is now:
hadoop dfs -setrep 4 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
sleep N
hadoop dfs -setrep 3 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
But the speed is very slow。
What is a better good way to make HDFS become healthy?
Thanks,
Ma Jian
回复: Block should be additionally replicated on 1 more rack(s)
Posted by 风雨无阻 <23...@qq.com>.
HI ,
I didn't try Hadoop rebalancer 。Because I remember rebalancer only considers disk load, and won't consider that data blocks which rack 。
I can try 。Thank you for your reply 。
------------------ 原始邮件 ------------------
发件人: "Yehia Elshater";<y....@gmail.com>;
发送时间: 2014年7月14日(星期一) 下午4:52
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Block should be additionally replicated on 1 more rack(s)
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <23...@qq.com> wrote:
HI all:
After the cluster configuration rack awareness,run " hadoop fsck / "
A lot of the following error occurred:
Replica placement policy is violated for blk_-1267324897180563985_11130670. Block should be additionally replicated on 1 more rack(s).
Online said "The reason is that three copies on the same rack" .
The solution is now:
hadoop dfs -setrep 4 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
sleep N
hadoop dfs -setrep 3 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
But the speed is very slow。
What is a better good way to make HDFS become healthy?
Thanks,
Ma Jian
回复: Block should be additionally replicated on 1 more rack(s)
Posted by 风雨无阻 <23...@qq.com>.
HI ,
I didn't try Hadoop rebalancer 。Because I remember rebalancer only considers disk load, and won't consider that data blocks which rack 。
I can try 。Thank you for your reply 。
------------------ 原始邮件 ------------------
发件人: "Yehia Elshater";<y....@gmail.com>;
发送时间: 2014年7月14日(星期一) 下午4:52
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Block should be additionally replicated on 1 more rack(s)
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <23...@qq.com> wrote:
HI all:
After the cluster configuration rack awareness,run " hadoop fsck / "
A lot of the following error occurred:
Replica placement policy is violated for blk_-1267324897180563985_11130670. Block should be additionally replicated on 1 more rack(s).
Online said "The reason is that three copies on the same rack" .
The solution is now:
hadoop dfs -setrep 4 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
sleep N
hadoop dfs -setrep 3 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
But the speed is very slow。
What is a better good way to make HDFS become healthy?
Thanks,
Ma Jian
回复: Block should be additionally replicated on 1 more rack(s)
Posted by 风雨无阻 <23...@qq.com>.
HI ,
I didn't try Hadoop rebalancer 。Because I remember rebalancer only considers disk load, and won't consider that data blocks which rack 。
I can try 。Thank you for your reply 。
------------------ 原始邮件 ------------------
发件人: "Yehia Elshater";<y....@gmail.com>;
发送时间: 2014年7月14日(星期一) 下午4:52
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Block should be additionally replicated on 1 more rack(s)
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <23...@qq.com> wrote:
HI all:
After the cluster configuration rack awareness,run " hadoop fsck / "
A lot of the following error occurred:
Replica placement policy is violated for blk_-1267324897180563985_11130670. Block should be additionally replicated on 1 more rack(s).
Online said "The reason is that three copies on the same rack" .
The solution is now:
hadoop dfs -setrep 4 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
sleep N
hadoop dfs -setrep 3 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
But the speed is very slow。
What is a better good way to make HDFS become healthy?
Thanks,
Ma Jian
回复: Block should be additionally replicated on 1 more rack(s)
Posted by 风雨无阻 <23...@qq.com>.
HI ,
I didn't try Hadoop rebalancer 。Because I remember rebalancer only considers disk load, and won't consider that data blocks which rack 。
I can try 。Thank you for your reply 。
------------------ 原始邮件 ------------------
发件人: "Yehia Elshater";<y....@gmail.com>;
发送时间: 2014年7月14日(星期一) 下午4:52
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Block should be additionally replicated on 1 more rack(s)
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <23...@qq.com> wrote:
HI all:
After the cluster configuration rack awareness,run " hadoop fsck / "
A lot of the following error occurred:
Replica placement policy is violated for blk_-1267324897180563985_11130670. Block should be additionally replicated on 1 more rack(s).
Online said "The reason is that three copies on the same rack" .
The solution is now:
hadoop dfs -setrep 4 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
sleep N
hadoop dfs -setrep 3 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
But the speed is very slow。
What is a better good way to make HDFS become healthy?
Thanks,
Ma Jian
Re: Block should be additionally replicated on 1 more rack(s)
Posted by Yehia Elshater <y....@gmail.com>.
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <23...@qq.com> wrote:
> HI all:
>
> After the cluster configuration rack awareness,run " hadoop fsck / "
> A lot of the following error occurred:
> Replica placement policy is violated for
> blk_-1267324897180563985_11130670. Block should be additionally replicated
> on 1 more rack(s).
>
> Online said "The reason is that three copies on the same rack" .
> The solution is now:
> hadoop dfs -setrep 4
> /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
> sleep N
> hadoop dfs -setrep 3
> /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
> But the speed is very slow。
>
> What is a better good way to make HDFS become healthy?
>
> Thanks,
> Ma Jian
>
Re: Block should be additionally replicated on 1 more rack(s)
Posted by Yehia Elshater <y....@gmail.com>.
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <23...@qq.com> wrote:
> HI all:
>
> After the cluster configuration rack awareness,run " hadoop fsck / "
> A lot of the following error occurred:
> Replica placement policy is violated for
> blk_-1267324897180563985_11130670. Block should be additionally replicated
> on 1 more rack(s).
>
> Online said "The reason is that three copies on the same rack" .
> The solution is now:
> hadoop dfs -setrep 4
> /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
> sleep N
> hadoop dfs -setrep 3
> /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
> But the speed is very slow。
>
> What is a better good way to make HDFS become healthy?
>
> Thanks,
> Ma Jian
>
Re: Block should be additionally replicated on 1 more rack(s)
Posted by Yehia Elshater <y....@gmail.com>.
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <23...@qq.com> wrote:
> HI all:
>
> After the cluster configuration rack awareness,run " hadoop fsck / "
> A lot of the following error occurred:
> Replica placement policy is violated for
> blk_-1267324897180563985_11130670. Block should be additionally replicated
> on 1 more rack(s).
>
> Online said "The reason is that three copies on the same rack" .
> The solution is now:
> hadoop dfs -setrep 4
> /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
> sleep N
> hadoop dfs -setrep 3
> /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
> But the speed is very slow。
>
> What is a better good way to make HDFS become healthy?
>
> Thanks,
> Ma Jian
>
Re: Block should be additionally replicated on 1 more rack(s)
Posted by Yehia Elshater <y....@gmail.com>.
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <23...@qq.com> wrote:
> HI all:
>
> After the cluster configuration rack awareness,run " hadoop fsck / "
> A lot of the following error occurred:
> Replica placement policy is violated for
> blk_-1267324897180563985_11130670. Block should be additionally replicated
> on 1 more rack(s).
>
> Online said "The reason is that three copies on the same rack" .
> The solution is now:
> hadoop dfs -setrep 4
> /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
> sleep N
> hadoop dfs -setrep 3
> /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
> But the speed is very slow。
>
> What is a better good way to make HDFS become healthy?
>
> Thanks,
> Ma Jian
>