You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Colin Patrick McCabe (JIRA)" <ji...@apache.org> on 2014/09/30 02:03:07 UTC

[jira] [Created] (HDFS-7168) Use excludedNodes consistently in DFSOutputStream

Colin Patrick McCabe created HDFS-7168:
------------------------------------------

             Summary: Use excludedNodes consistently in DFSOutputStream
                 Key: HDFS-7168
                 URL: https://issues.apache.org/jira/browse/HDFS-7168
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Colin Patrick McCabe
            Assignee: Colin Patrick McCabe


We currently have two separate collections of excluded nodes in the {{DFSOutputStream#DataStreamer}}.  One is {{DFSOutputStream#failed}}; another is {{DFSOutputStream#excludedNodes}}.  Both of these collections just deal with blacklisting nodes that we have found to be bad.  We should just use excludedNodes for both.

We also should make this a per-DFSOutputStream variable, rather than being per-DataStreamer.  We don't need to forget all this information whenever a DataStreamer is torn down.  Since {{DFSOutputStream#excludedNodes}} is a Guava cache, nodes will expire out of it once enough time elapses, so they will not be permanently blacklisted.

We should also remove {{DFSOutputStream#setTestFilename}}, since it is no longer needed now that we can safely rename streams that are open for write.  And {{DFSOutputStream#getBlock}} should be synchronized.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)