You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2007/09/06 22:15:56 UTC

[Lucene-hadoop Wiki] Update of "AmazonS3" by TobyDiPasquale

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by TobyDiPasquale:
http://wiki.apache.org/lucene-hadoop/AmazonS3

------------------------------------------------------------------------------
  
  S3 support was introduced in Hadoop 0.10.0 ([http://issues.apache.org/jira/browse/HADOOP-574 HADOOP-574]),
  but this had a few bugs so you should use Hadoop 0.10.1 or later.
+ 
+ = Notes on S3 Support in Hadoop =
+ 
+ The current implementation of support for Amazon S3 in Hadoop is set up such that it uses S3 as a backing store for a block-based filesystem, very similar to the way HDFS is implemented. It '''does not''' currently allow you to access any "regular" S3 keys but instead requires you to set aside an S3 bucket for its block-based format. There is a request in for the ability to access "regular" S3 files with Hadoop ([https://issues.apache.org/jira/browse/HADOOP-930 HADOOP-930]), but as yet this is unimplemented.
  
  = Setting up hadoop to use S3 as a replacement for HDFS =