You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2015/03/06 19:22:45 UTC

[Hadoop Wiki] Trivial Update of "AmazonS3" by SteveLoughran

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "AmazonS3" page has been changed by SteveLoughran:
https://wiki.apache.org/hadoop/AmazonS3?action=diff&rev1=17&rev2=18

Comment:
cut the recommendation to use a version of hadoop >0.10 for s3: to work

  Note that by using S3 as an input you lose the data locality optimization, which may be significant. The general best practise is to copy in data using `distcp` at the start of a workflow, then copy it out at the end, using the transient HDFS in between.
  
  = History =
-  * The S3 block filesystem was introduced in Hadoop 0.10.0 ([[http://issues.apache.org/jira/browse/HADOOP-574|HADOOP-574]]), but this had a few bugs so you should use Hadoop 0.10.1 or later.
+  * The S3 block filesystem was introduced in Hadoop 0.10.0 ([[http://issues.apache.org/jira/browse/HADOOP-574|HADOOP-574]]).
   * The S3 native filesystem was introduced in Hadoop 0.18.0 ([[http://issues.apache.org/jira/browse/HADOOP-930|HADOOP-930]]) and rename support was added in Hadoop 0.19.0 ([[https://issues.apache.org/jira/browse/HADOOP-3361|HADOOP-3361]]).
   * The S3A filesystem was introduced in Hadoop 2.6.0. Some issues were found and fixed for later Hadoop versions[[https://issues.apache.org/jira/browse/HADOOP-11571|HADOOP-11571]], so Hadoop-2.6.0's support of s3a must be considered an incomplete replacement for the s3n FS.