You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Yi Liu (JIRA)" <ji...@apache.org> on 2014/12/30 07:15:13 UTC

[jira] [Created] (YARN-2996) Refine some fs operations in FileSystemRMStateStore to improve performance

Yi Liu created YARN-2996:
----------------------------

             Summary: Refine some fs operations in FileSystemRMStateStore to improve performance
                 Key: YARN-2996
                 URL: https://issues.apache.org/jira/browse/YARN-2996
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: resourcemanager
            Reporter: Yi Liu
            Assignee: Yi Liu


In {{FileSystemRMStateStore}}, we can refine some fs operations to improve performance:
*1.* There are several places invoke {{fs.exists}}, then {{fs.getFileStatus}}, we can merge them to save one RPC call
{code}
if (fs.exists(versionNodePath)) {
    FileStatus status = fs.getFileStatus(versionNodePath);
{code}

*2.*
{code}
protected void updateFile(Path outputPath, byte[] data) throws Exception {
  Path newPath = new Path(outputPath.getParent(), outputPath.getName() + ".new");
  // use writeFile to make sure .new file is created atomically
  writeFile(newPath, data);
  replaceFile(newPath, outputPath);
}
{code}
The {{updateFile}} is not good too, it write file to _output\_file_.tmp, then rename to _output\_file_.new, then rename it to _output\_file_, we can reduce one rename operation.

Also there is one unnecessary import, we can remove it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)