You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by "Brian C. Huffman" <bh...@etinternational.com> on 2014/10/21 15:57:59 UTC
HDFS - Consolidate 2 small volumes into 1 large volume
All,
Is it possible to consolidate two small data volumes (500GB each) into a
larger data volume (3TB)?
I'm thinking that as long as the block file names and metadata are
unique, then I should be able to shut down the datanode and use
something like tar or rsync to copy the contents of each small volume to
the large volume. Will that work?
Thanks,
Brian
Re: HDFS - Consolidate 2 small volumes into 1 large volume
Posted by Azuryy Yu <az...@gmail.com>.
yes, you can.
stop the cluster,
change your hdfs-site.xml on your datanode. (dfs.datanode.dir) to the large
volume.
copy two small data volumes to the large volumes, which was configured on
above.
start cluster.
then you are done.
On Tue, Oct 21, 2014 at 9:57 PM, Brian C. Huffman <
bhuffman@etinternational.com> wrote:
> All,
>
> Is it possible to consolidate two small data volumes (500GB each) into a
> larger data volume (3TB)?
>
> I'm thinking that as long as the block file names and metadata are unique,
> then I should be able to shut down the datanode and use something like tar
> or rsync to copy the contents of each small volume to the large volume.
> Will that work?
>
> Thanks,
> Brian
>
>
Re: HDFS - Consolidate 2 small volumes into 1 large volume
Posted by Azuryy Yu <az...@gmail.com>.
yes, you can.
stop the cluster,
change your hdfs-site.xml on your datanode. (dfs.datanode.dir) to the large
volume.
copy two small data volumes to the large volumes, which was configured on
above.
start cluster.
then you are done.
On Tue, Oct 21, 2014 at 9:57 PM, Brian C. Huffman <
bhuffman@etinternational.com> wrote:
> All,
>
> Is it possible to consolidate two small data volumes (500GB each) into a
> larger data volume (3TB)?
>
> I'm thinking that as long as the block file names and metadata are unique,
> then I should be able to shut down the datanode and use something like tar
> or rsync to copy the contents of each small volume to the large volume.
> Will that work?
>
> Thanks,
> Brian
>
>
Re: HDFS - Consolidate 2 small volumes into 1 large volume
Posted by Azuryy Yu <az...@gmail.com>.
yes, you can.
stop the cluster,
change your hdfs-site.xml on your datanode. (dfs.datanode.dir) to the large
volume.
copy two small data volumes to the large volumes, which was configured on
above.
start cluster.
then you are done.
On Tue, Oct 21, 2014 at 9:57 PM, Brian C. Huffman <
bhuffman@etinternational.com> wrote:
> All,
>
> Is it possible to consolidate two small data volumes (500GB each) into a
> larger data volume (3TB)?
>
> I'm thinking that as long as the block file names and metadata are unique,
> then I should be able to shut down the datanode and use something like tar
> or rsync to copy the contents of each small volume to the large volume.
> Will that work?
>
> Thanks,
> Brian
>
>
Re: HDFS - Consolidate 2 small volumes into 1 large volume
Posted by Azuryy Yu <az...@gmail.com>.
yes, you can.
stop the cluster,
change your hdfs-site.xml on your datanode. (dfs.datanode.dir) to the large
volume.
copy two small data volumes to the large volumes, which was configured on
above.
start cluster.
then you are done.
On Tue, Oct 21, 2014 at 9:57 PM, Brian C. Huffman <
bhuffman@etinternational.com> wrote:
> All,
>
> Is it possible to consolidate two small data volumes (500GB each) into a
> larger data volume (3TB)?
>
> I'm thinking that as long as the block file names and metadata are unique,
> then I should be able to shut down the datanode and use something like tar
> or rsync to copy the contents of each small volume to the large volume.
> Will that work?
>
> Thanks,
> Brian
>
>