You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by 조주일 <tj...@kgrid.co.kr> on 2014/06/30 07:50:10 UTC
How quickly can I increase the number of replicas?
My cluser version hdfs 2.2 stable ( 2 ha namenodes, 10 datanodes)
I was command bin/hdfs dfs -setrep -R 2 / ( replication 1 to 2 )
I found that HDFS is actually replicating the under replicated blocks but it works very slowly.
HDFS performs the replication about 1 block per second.
I have about 400000 under replicated blocks so it will take about 4 more days.
Is there any way to speed it up?
Re: How quickly can I increase the number of replicas?
Posted by Vikas Srivastava <vi...@apple.com>.
Hey,
You can set the below properties to speed up replication
<property>
<!-- 100Mbit/s -->
<name>dfs.balance.bandwidthPerSec</name>
<value>131072000</value>
</property>
<property>
<name>dfs.max-repl-streams</name>
<value>50</value>
</property>
Thanks
Vikas
On Jun 29, 2014, at 10:50 PM, 조주일 <tj...@kgrid.co.kr> wrote:
>
> My cluser version hdfs 2.2 stable ( 2 ha namenodes, 10 datanodes)
>
>
> I was command bin/hdfs dfs -setrep -R 2 / ( replication 1 to 2 )
>
>
> I found that HDFS is actually replicating the under replicated blocks but it works very slowly.
>
> HDFS performs the replication about 1 block per second.
>
> I have about 400000 under replicated blocks so it will take about 4 more days.
>
> Is there any way to speed it up?
>
Re: How quickly can I increase the number of replicas?
Posted by Vikas Srivastava <vi...@apple.com>.
Hey,
You can set the below properties to speed up replication
<property>
<!-- 100Mbit/s -->
<name>dfs.balance.bandwidthPerSec</name>
<value>131072000</value>
</property>
<property>
<name>dfs.max-repl-streams</name>
<value>50</value>
</property>
Thanks
Vikas
On Jun 29, 2014, at 10:50 PM, 조주일 <tj...@kgrid.co.kr> wrote:
>
> My cluser version hdfs 2.2 stable ( 2 ha namenodes, 10 datanodes)
>
>
> I was command bin/hdfs dfs -setrep -R 2 / ( replication 1 to 2 )
>
>
> I found that HDFS is actually replicating the under replicated blocks but it works very slowly.
>
> HDFS performs the replication about 1 block per second.
>
> I have about 400000 under replicated blocks so it will take about 4 more days.
>
> Is there any way to speed it up?
>
Re: How quickly can I increase the number of replicas?
Posted by Vikas Srivastava <vi...@apple.com>.
Hey,
You can set the below properties to speed up replication
<property>
<!-- 100Mbit/s -->
<name>dfs.balance.bandwidthPerSec</name>
<value>131072000</value>
</property>
<property>
<name>dfs.max-repl-streams</name>
<value>50</value>
</property>
Thanks
Vikas
On Jun 29, 2014, at 10:50 PM, 조주일 <tj...@kgrid.co.kr> wrote:
>
> My cluser version hdfs 2.2 stable ( 2 ha namenodes, 10 datanodes)
>
>
> I was command bin/hdfs dfs -setrep -R 2 / ( replication 1 to 2 )
>
>
> I found that HDFS is actually replicating the under replicated blocks but it works very slowly.
>
> HDFS performs the replication about 1 block per second.
>
> I have about 400000 under replicated blocks so it will take about 4 more days.
>
> Is there any way to speed it up?
>
Re: How quickly can I increase the number of replicas?
Posted by Vikas Srivastava <vi...@apple.com>.
Hey,
You can set the below properties to speed up replication
<property>
<!-- 100Mbit/s -->
<name>dfs.balance.bandwidthPerSec</name>
<value>131072000</value>
</property>
<property>
<name>dfs.max-repl-streams</name>
<value>50</value>
</property>
Thanks
Vikas
On Jun 29, 2014, at 10:50 PM, 조주일 <tj...@kgrid.co.kr> wrote:
>
> My cluser version hdfs 2.2 stable ( 2 ha namenodes, 10 datanodes)
>
>
> I was command bin/hdfs dfs -setrep -R 2 / ( replication 1 to 2 )
>
>
> I found that HDFS is actually replicating the under replicated blocks but it works very slowly.
>
> HDFS performs the replication about 1 block per second.
>
> I have about 400000 under replicated blocks so it will take about 4 more days.
>
> Is there any way to speed it up?
>