You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Xiaoyu Wang <wa...@jd.com> on 2015/06/16 10:28:55 UTC
FSImage from uncompress to compress change
Hi all!
My hadoop version is 2.0.0
In my hadoop configuration config the dfs.image.compress=false
Now the cluster has been running for a long time.
the fsimage size is growing. I want to compress the fsimage.
can I change the dfs.image.compress=true and
dfs.image.compression.codec=xx.Codec and restart the namenode?
the no compress fsimage can be read? the data will lost or not ?
Thanks!
Re: FSImage from uncompress to compress change
Posted by Yanbo Liang <yb...@gmail.com>.
As far as I know, HDFS get image compression information from image file
when loading fsimage.
So you can correctly load fsimage file even you set different compression
codec.
I strongly recommend to do these operations with the same version and run
"hdfs dfsadmin -saveNamespace" to save the new compacted fsimage to
decrease storage size.
2015-06-16 16:28 GMT+08:00 Xiaoyu Wang <wa...@jd.com>:
> Hi all!
> My hadoop version is 2.0.0
> In my hadoop configuration config the dfs.image.compress=false
> Now the cluster has been running for a long time.
> the fsimage size is growing. I want to compress the fsimage.
> can I change the dfs.image.compress=true and
> dfs.image.compression.codec=xx.Codec and restart the namenode?
> the no compress fsimage can be read? the data will lost or not ?
>
> Thanks!
>
Re: FSImage from uncompress to compress change
Posted by Yanbo Liang <yb...@gmail.com>.
As far as I know, HDFS get image compression information from image file
when loading fsimage.
So you can correctly load fsimage file even you set different compression
codec.
I strongly recommend to do these operations with the same version and run
"hdfs dfsadmin -saveNamespace" to save the new compacted fsimage to
decrease storage size.
2015-06-16 16:28 GMT+08:00 Xiaoyu Wang <wa...@jd.com>:
> Hi all!
> My hadoop version is 2.0.0
> In my hadoop configuration config the dfs.image.compress=false
> Now the cluster has been running for a long time.
> the fsimage size is growing. I want to compress the fsimage.
> can I change the dfs.image.compress=true and
> dfs.image.compression.codec=xx.Codec and restart the namenode?
> the no compress fsimage can be read? the data will lost or not ?
>
> Thanks!
>
Re: FSImage from uncompress to compress change
Posted by Yanbo Liang <yb...@gmail.com>.
As far as I know, HDFS get image compression information from image file
when loading fsimage.
So you can correctly load fsimage file even you set different compression
codec.
I strongly recommend to do these operations with the same version and run
"hdfs dfsadmin -saveNamespace" to save the new compacted fsimage to
decrease storage size.
2015-06-16 16:28 GMT+08:00 Xiaoyu Wang <wa...@jd.com>:
> Hi all!
> My hadoop version is 2.0.0
> In my hadoop configuration config the dfs.image.compress=false
> Now the cluster has been running for a long time.
> the fsimage size is growing. I want to compress the fsimage.
> can I change the dfs.image.compress=true and
> dfs.image.compression.codec=xx.Codec and restart the namenode?
> the no compress fsimage can be read? the data will lost or not ?
>
> Thanks!
>
Re: FSImage from uncompress to compress change
Posted by Yanbo Liang <yb...@gmail.com>.
As far as I know, HDFS get image compression information from image file
when loading fsimage.
So you can correctly load fsimage file even you set different compression
codec.
I strongly recommend to do these operations with the same version and run
"hdfs dfsadmin -saveNamespace" to save the new compacted fsimage to
decrease storage size.
2015-06-16 16:28 GMT+08:00 Xiaoyu Wang <wa...@jd.com>:
> Hi all!
> My hadoop version is 2.0.0
> In my hadoop configuration config the dfs.image.compress=false
> Now the cluster has been running for a long time.
> the fsimage size is growing. I want to compress the fsimage.
> can I change the dfs.image.compress=true and
> dfs.image.compression.codec=xx.Codec and restart the namenode?
> the no compress fsimage can be read? the data will lost or not ?
>
> Thanks!
>