[jira] [Commented] (HADOOP-11262) Enable YARN to use S3A

["Lei (Eddy) Xu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-11262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986384#comment-14986384 ] 

Lei (Eddy) Xu commented on HADOOP-11262:
----------------------------------------

[~PieterReuse], thanks a lot for working on this. Could you update the code to obey [Hadoop coding style|https://wiki.apache.org/hadoop/CodeReviewChecklist] (i.e., 2 space indent and putting {{@override}} below comments.).  I think that we do not need {{@Override}} for {{@Before/@After/@Test}} methods.

{code}     try {
       // instruct FS to verify checksum through the FileContext:
-      fc.setVerifyChecksum(true, path);
       out.write(data, 0, data.length);
     } finally {
       out.close();
+      fc.setVerifyChecksum(true, path);
     }
{code}

Could you help me to understand why you move {{setVerifyChecksum}} after {{close()}}? Wouldn't the checksum not being set in the write op?

{code}
+    private String readStringFromFile(Path path){
+        FSDataInputStream in = null;
+        try {
+            in = fs.open(path);
+            long bytesLen = fs.getFileStatus(path).getLen();
+            byte[] buffer = new byte[(int)bytesLen];
+            IOUtils.readFully(in, buffer, 0, buffer.length);
+            return new String(buffer);
+        } catch (IOException e ){
+            throw new RuntimeException("Failed to read from [" + path + "]", e);
+        } finally {
+            IOUtils.closeStream(in);
+        }
+    }
{code}
Could you use {{try-with-resource}} in JDK7 for this? I see you have already used it in the other functions.

[~stevel@apache.org] and [~cnauroth], could you please provide some color on this JIRA? You have much more experiences on YARN. 

> Enable YARN to use S3A 
> -----------------------
>
>                 Key: HADOOP-11262
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11262
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Thomas Demoor
>            Assignee: Pieter Reuse
>              Labels: amazon, s3
>         Attachments: HADOOP-11262-2.patch, HADOOP-11262-3.patch, HADOOP-11262-4.patch, HADOOP-11262-5.patch, HADOOP-11262-6.patch, HADOOP-11262-7.patch, HADOOP-11262.patch
>
>
> Uses DelegateToFileSystem to expose S3A as an AbstractFileSystem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)