You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "eric baldeschwieler (JIRA)" <ji...@apache.org> on 2006/11/02 09:37:17 UTC
[jira] Commented: (HADOOP-659) Boost the priority of re-replicating
blocks that are far from their replication target
[ http://issues.apache.org/jira/browse/HADOOP-659?page=comments#action_12446507 ]
eric baldeschwieler commented on HADOOP-659:
--------------------------------------------
why not maintain a total order?
> Boost the priority of re-replicating blocks that are far from their replication target
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-659
> URL: http://issues.apache.org/jira/browse/HADOOP-659
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.7.2
> Reporter: Konstantin Shvachko
> Assigned To: Konstantin Shvachko
>
> I see two types of replications that should be accelerated compared to all others.
> 1. Blocks that have only one remaining copy (but are required to have higher replication).
> 2. Blocks that have less than 1/3 of their replicas in place.
> The latter occurs when map/reduce sets replication of certain files to 10, and we want
> it happen fast to achieve better performance on the tasks.
> So I think we should distinguish two major groups of under-replicated blocks:
> first-priority (having only 1 copy or less than 1/3 of required replicas), and the rest.
> The name-node places first-priority blocks into the beginning of the neededReplication
> list, and the rest are placed at the end. That way the first-priority blocks will be replicated
> first and then the others.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira