You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2017/12/16 02:30:06 UTC
[jira] [Commented] (HDFS-12093) [READ] Share remoteFS between
ProvidedReplica instances.
[ https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293587#comment-16293587 ]
Hudson commented on HDFS-12093:
-------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13391 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13391/])
HDFS-12093. [READ] Share remoteFS between ProvidedReplica instances. (cdouglas: rev 2407c9b93aabb021b76c802b19c928fb6cbb0a85)
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FinalizedProvidedReplica.java
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ProvidedReplica.java
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ReplicaBuilder.java
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ProvidedVolumeImpl.java
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestProvidedReplicaImpl.java
> [READ] Share remoteFS between ProvidedReplica instances.
> --------------------------------------------------------
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ewan Higgs
> Assignee: Virajith Jalaparti
> Attachments: HDFS-12093-HDFS-9806.001.patch, HDFS-12093-HDFS-9806.002.patch
>
>
> When a Datanode comes online using Provided storage, it fills the {{ReplicaMap}} with the known replicas. With Provided Storage, this includes {{ProvidedReplica}} instances. Each of these objects, in their constructor, will construct an FileSystem using the Service Provider. This can result in contacting the remote file system and checking that the credentials are correct and that the data is there. For large systems this is a prohibitively expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org