You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "shimingfei (JIRA)" <ji...@apache.org> on 2016/06/23 08:13:16 UTC
[jira] [Comment Edited] (HADOOP-12756) Incorporate Aliyun OSS file
system implementation
[ https://issues.apache.org/jira/browse/HADOOP-12756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15346018#comment-15346018 ]
shimingfei edited comment on HADOOP-12756 at 6/23/16 8:12 AM:
--------------------------------------------------------------
[~drankye] Thanks for your suggestions! I have modified the code according to your comments:
1. Done
2. Done
3. The check is used to make sure that when a file is deleted, the parent directory still exists. for example, when a file named "/temp/tests/test0" is deleted, and if parent directory "/temp/tests/" doesn't exist, then it will be created as parent directory
4. We keep the method name not changed, it is used to check whether the path is valid, if "/temp/tests" exists and the size is not zero, then "/temp/tests/test0" is not a valid file name.
6. we don't have strong intentions to do it currently, because most of the logic can not be reused, and many parameters are needed for the utility functions
was (Author: shimingfei):
[~drankye] Thanks for your suggestions! I have modified the code according to your comments:
1. Done
2. Done
3. The check is used to make sure that when a file is deleted, the parent directory still exists. for example, when a file named "/temp/tests/test0" is deleted, and if parent directory "/temp/tests/" doesn't exist, then it will be created as parent directory
4. We keep the method name not changed, it is used to check whether the path is valid, if "/temp/tests" exists and the size is not zero, then "/temp/tests/test0" is not a valid file name.
6. we don't have strong intentions to do it currently, because most of the logic can not be reused, and many parameters are needed for the utility functions
Since the functionality and testing are done, I suggest merging this first, and Aliyun will do the optimization work in following PRs
> Incorporate Aliyun OSS file system implementation
> -------------------------------------------------
>
> Key: HADOOP-12756
> URL: https://issues.apache.org/jira/browse/HADOOP-12756
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Affects Versions: 2.8.0
> Reporter: shimingfei
> Assignee: shimingfei
> Attachments: HADOOP-12756-v02.patch, HADOOP-12756.003.patch, HADOOP-12756.004.patch, HADOOP-12756.005.patch, HCFS User manual.md, OSS integration.pdf, OSS integration.pdf
>
>
> Aliyun OSS is widely used among China’s cloud users, but currently it is not easy to access data laid on OSS storage from user’s Hadoop/Spark application, because of no original support for OSS in Hadoop.
> This work aims to integrate Aliyun OSS with Hadoop. By simple configuration, Spark/Hadoop applications can read/write data from OSS without any code change. Narrowing the gap between user’s APP and data storage, like what have been done for S3 in Hadoop
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org