Hadoop balancer

[Ryan LeCompte <le...@gmail.com>]

I've tried running the bin/hadoop balance command since I recently
added a new node to the Hadoop cluster. I noticed the following output
in the beginning:

08/12/03 10:26:35 INFO balancer.Balancer: Will move 10 GBbytes in this iteration
Dec 3, 2008 10:26:35 AM           0                 0 KB
2.67 GB              10 GB
08/12/03 10:26:36 WARN balancer.Balancer: Error moving block
-6653850537520285828 from 10.30.2.31:50010 to 10.30.2.23:50010 through
10.30.2.31:50010: block move is failed

However, subsequent iterations have succeeded. Is this harmless or is
there a possibility that data was corrupted during the re-balancing
attempt?

Thanks,
Ryan

Re: Hadoop balancer

[Alex Loddengaard <al...@cloudera.com>]

Have you tried running fsck?  fsck will tell you if you have corruption.

<http://hadoop.apache.org/core/docs/r0.17.2/hdfs_user_guide.html#Fsck>

Alex

On Wed, Dec 3, 2008 at 7:37 AM, Ryan LeCompte <le...@gmail.com> wrote:

> I've tried running the bin/hadoop balance command since I recently
> added a new node to the Hadoop cluster. I noticed the following output
> in the beginning:
>
> 08/12/03 10:26:35 INFO balancer.Balancer: Will move 10 GBbytes in this
> iteration
> Dec 3, 2008 10:26:35 AM           0                 0 KB
> 2.67 GB              10 GB
> 08/12/03 10:26:36 WARN balancer.Balancer: Error moving block
> -6653850537520285828 from 10.30.2.31:50010 to 10.30.2.23:50010 through
> 10.30.2.31:50010: block move is failed
>
> However, subsequent iterations have succeeded. Is this harmless or is
> there a possibility that data was corrupted during the re-balancing
> attempt?
>
> Thanks,
> Ryan
>