[jira] [Commented] (HDFS-6133) Make Balancer support exclude specified path

["Tsz Wo Nicholas Sze (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290892#comment-14290892 ] 

Tsz Wo Nicholas Sze commented on HDFS-6133:
-------------------------------------------

Thanks for working on this.  The pinning idea is very interesting!

A replica is pinned only if it is stored in a favored node.  In a write pipeline, it could possibly be some datanodes are favored nodes but some are not.  So the new parameters in DataTransferProtocol.writeBlock should be similar to storageType and targetStorageTypes, i.e. a new pinning parameter to the next datanodes and another new parameter targetPinnings for the downstream datanodes.
{code}
   public void writeBlock(final ExtendedBlock blk,
       final StorageType storageType, 
+      final boolean pinning,
       final Token<BlockTokenIdentifier> blockToken,
       final String clientName,
       final DatanodeInfo[] targets,
       final StorageType[] targetStorageTypes, 
+      final boolean[] targetPinnings,
       final DatanodeInfo source,
       final BlockConstructionStage stage,
       final int pipelineSize,
{code}


> Make Balancer support exclude specified path
> --------------------------------------------
>
>                 Key: HDFS-6133
>                 URL: https://issues.apache.org/jira/browse/HDFS-6133
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer & mover, namenode
>            Reporter: zhaoyunjiong
>            Assignee: zhaoyunjiong
>         Attachments: HDFS-6133-1.patch, HDFS-6133-2.patch, HDFS-6133-3.patch, HDFS-6133-4.patch, HDFS-6133-5.patch, HDFS-6133-6.patch, HDFS-6133.patch
>
>
> Currently, run Balancer will destroying Regionserver's data locality.
> If getBlocks could exclude blocks belongs to files which have specific path prefix, like "/hbase", then we can run Balancer without destroying Regionserver's data locality.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)