[jira] Updated: (HADOOP-574) want FileSystem implementation for Amazon S3

["Tom White (JIRA)" <ji...@apache.org>]

     [ http://issues.apache.org/jira/browse/HADOOP-574?page=all ]

Tom White updated HADOOP-574:
-----------------------------

               Status: Patch Available  (was: Open)
    Affects Version/s: 0.9.0

Please find attached a patch for an S3 filesystem for review and discussion. It is not yet complete so I am not expecting it to be committed.
There are a number of TODO comments in the code that list the remaining work items. These mainly concern corner cases or tidying up work - all the basic file system operations work.

The code has been exercised by unit tests. To get these to run you need to fill in S3 access codes in the test hadoop-site.xml. (This raises an interesting question of how to have these run as automated tests.)

I have not tried running Hadoop (or something like DfsShell) with this filesystem. We probably need to wait until HADOOP-571 is done.

Looking forward to your feedback!

> want FileSystem implementation for Amazon S3
> --------------------------------------------
>
>                 Key: HADOOP-574
>                 URL: http://issues.apache.org/jira/browse/HADOOP-574
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 0.9.0
>            Reporter: Doug Cutting
>
> An S3-based Hadoop FileSystem would make a great addition to Hadoop.
> It would facillitate use of Hadoop on Amazon's EC2 computing grid, as discussed here:
> http://www.mail-archive.com/hadoop-user@lucene.apache.org/msg00318.html
> This is related to HADOOP-571, which would make Hadoop's FileSystem considerably easier to extend.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira