[jira] [Commented] (HDFS-7692) DataStorage#addStorageLocations(...) should support MultiThread to speedup the upgrade of block pool at multi storage directories.

["Lei (Eddy) Xu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594136#comment-14594136 ] 

Lei (Eddy) Xu commented on HDFS-7692:
-------------------------------------

Hi, [~guoleitao].  Thanks a lot for updating this. It looks good after addressing the comments.

One question I have: why does not create {{ExecutorService}} and collect {{successVolumes}} within {{DataStorage#addStorageLocation}}? The interface might look simpler (less modification) in that way.

One minor nits:  
{code}
      e.printStackTrace();
{code}

Can we replace it with {{log.warn(e)}} or {{log.error(e)}}?

> DataStorage#addStorageLocations(...) should support MultiThread to speedup the upgrade of block pool at multi storage directories.
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-7692
>                 URL: https://issues.apache.org/jira/browse/HDFS-7692
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.5.2
>            Reporter: Leitao Guo
>            Assignee: Leitao Guo
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-7692.01.patch, HDFS-7692.02.patch
>
>
> {code:title=DataStorage#addStorageLocations(...)|borderStyle=solid}
>     for (StorageLocation dataDir : dataDirs) {
>       File root = dataDir.getFile();
>      ... ...
>         bpStorage.recoverTransitionRead(datanode, nsInfo, bpDataDirs, startOpt);
>         addBlockPoolStorage(bpid, bpStorage);
>     ... ...
>       successVolumes.add(dataDir);
>     }
> {code}
> In the above code the storage directories will be analyzed one by one, which is really time consuming when upgrading HDFS with datanodes have dozens of large volumes.  MultiThread dataDirs analyzing should be supported here to speedup upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)