You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Tomasz Chmielewski <ma...@wpkg.org> on 2011/10/31 14:49:37 UTC
rebalancing data on disks?
Hi,
I have a HDFS cluster consisting of several hosts.
On each node, I add a new disk when the current capacity is close to full.
Right now, every server has more or less such distribution of data:
/dev/sdf 493G 468G 51M 100% /data1
/dev/sdg 493G 468G 51M 100% /data2
/dev/sdh 493G 103G 365G 22% /data3
/dev/sdi 493G 100G 368G 22% /data4
So, /dev/sdf and /dev/sdg almost 100% full, and there is lots of free
space on /dev/sdh and /dev/sdi.
Disks which are 100% full don't make monitoring very happy.
Is it possible to rebalance data on the disks on one HDFS server (or,
more servers)?
"hadoop balancer" will want to rebalance data between the servers, but
not between the disks on one server.
--
Tomasz Chmielewski
http://wpkg.org
Re: rebalancing data on disks?
Posted by Harsh J <ha...@cloudera.com>.
Which is why you need your HDFS shutdown before this is done.
See http://search-hadoop.com/m/fSof91EYbe9 and http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
On 31-Oct-2011, at 8:39 PM, Tomasz Chmielewski wrote:
> On 31.10.2011 15:37, Jeff Bean wrote:
>> IIUC you can move the data block files from one disk to another until they are evenly distributed. Give it a shot.
>
> Just like that? Using "mv" command?
>
> I.e. move /data2/hdfs/data/current/subdir44/* to /data3/hdfs/data/current/subdir44/
>
> ?
>
>
> It's not atomic, so at some point, /data3/ would contain a file which is not a full copy of the same file on /data2/; later, both /data2 and /data3 would contain identical file.
>
>
> --
> Tomasz Chmielewski
> http://wpkg.org
>
Re: rebalancing data on disks?
Posted by Tomasz Chmielewski <ma...@wpkg.org>.
On 31.10.2011 15:37, Jeff Bean wrote:
> IIUC you can move the data block files from one disk to another until they are evenly distributed. Give it a shot.
Just like that? Using "mv" command?
I.e. move /data2/hdfs/data/current/subdir44/* to
/data3/hdfs/data/current/subdir44/
?
It's not atomic, so at some point, /data3/ would contain a file which is
not a full copy of the same file on /data2/; later, both /data2 and
/data3 would contain identical file.
--
Tomasz Chmielewski
http://wpkg.org
Re: rebalancing data on disks?
Posted by Jeff Bean <jw...@cloudera.com>.
IIUC you can move the data block files from one disk to another until they are evenly distributed. Give it a shot.
Sent from my iPad
On Oct 31, 2011, at 6:49, Tomasz Chmielewski <ma...@wpkg.org> wrote:
> Hi,
>
> I have a HDFS cluster consisting of several hosts.
>
> On each node, I add a new disk when the current capacity is close to full.
>
> Right now, every server has more or less such distribution of data:
>
> /dev/sdf 493G 468G 51M 100% /data1
> /dev/sdg 493G 468G 51M 100% /data2
> /dev/sdh 493G 103G 365G 22% /data3
> /dev/sdi 493G 100G 368G 22% /data4
>
>
> So, /dev/sdf and /dev/sdg almost 100% full, and there is lots of free space on /dev/sdh and /dev/sdi.
>
> Disks which are 100% full don't make monitoring very happy.
>
> Is it possible to rebalance data on the disks on one HDFS server (or, more servers)?
>
>
> "hadoop balancer" will want to rebalance data between the servers, but not between the disks on one server.
>
>
> --
> Tomasz Chmielewski
> http://wpkg.org