[Hadoop Wiki] Update of "HCFS" by JayVyas

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "HCFS" page has been changed by JayVyas:
https://wiki.apache.org/hadoop/HCFS?action=diff&rev1=22&rev2=23

  = Filesystem Compatibility with Apache Hadoop =
+ [[https://wiki.apache.org/hadoop/HCFS/Progress|See this link for Community Progress and Participation on these topics]]
  
- [[ https://wiki.apache.org/hadoop/HCFS/Progress | See this link for Community Progress and Participation on these topics  ]]
- 
- Apache Hadoop is built on a distributed filesystem, HDFS, ''Hadoop Distributed File System'', capable of storing tens of Petabytes of data. This filesystem is designed to work with Apache Hadoop from the ground up, with location aware block placement, integration with the Hadoop tools and both explicit and implicit testing. 
+ Apache Hadoop is built on a distributed filesystem, HDFS, ''Hadoop Distributed File System'', capable of storing tens of Petabytes of data. This filesystem is designed to work with Apache Hadoop from the ground up, with location aware block placement, integration with the Hadoop tools and both explicit and implicit testing.
  
  Apache Hadoop also works with other filesystems, the platform specific "local" filesystem, [[BlobStore|Blobstores]] such as Amazon S3 and Azure storage, as well as  alternative distributed filesystems.
  
@@ -13, +12 @@

   1. The filesystem looks like a "native" filesystem, and is accessed as a local FS, perhaps with some filesystem-specific means of telling the MapReduce layer which TaskTracker is closest to the data.
   1. The filesystem provides an implementation of the `org.apache.hadoop.fs.FileSystem` class (and in Hadoop v2, in implementation of the `FileContext` class}
  
- Implementing the `FileSystem` class ensures that there is an API for applications such as MapReduce, Apache HBase, Apache Giraph and others can use -including third-party applications as well as code running in a MapReduce job that wishes to read or write data. 
+ Implementing the `FileSystem` class ensures that there is an API for applications such as MapReduce, Apache HBase, Apache Giraph and others can use -including third-party applications as well as code running in a MapReduce job that wishes to read or write data.
  
  The selection of which filesystem to use comes from the URI scheme used to refer to it -the prefix `hdfs:` on any file path means that it refers to an HDFS filesystem; `file:` to the local filesystem, `s3:` to Amazon S3, `ftp:` FTP, `swift:` OpenStackSwift, ...etc.
  
  There are other filesystems that provide explicit integration with Hadoop through the relevant Java JAR files, native binaries and configuration parameters needed to add a new schema to Hadoop, such as `fat32:`
  
- All providers of filesystem plugins do their utmost to make their filesystems are compatible with Apache Hadoop. Ambiguities in the Hadoop APIs do not help here -as a lot of the expectations of Hadoop applications are set not by the `FileSystem` API, but the behavior of HDFS itself -which makes it harder to distinguish "bug" from "feature" in the behavior of HDFS. 
+ All providers of filesystem plugins do their utmost to make their filesystems are compatible with Apache Hadoop. Ambiguities in the Hadoop APIs do not help here -as a lot of the expectations of Hadoop applications are set not by the `FileSystem` API, but the behavior of HDFS itself -which makes it harder to distinguish "bug" from "feature" in the behavior of HDFS.
  
- The Hadoop developers are (as of April 2013), attempting to [[https://issues.apache.org/jira/browse/HADOOP-9371|define the Semantics of the Hadoop FileSystem more rigorously]] as well as adding [[https://issues.apache.org/jira/browse/HADOOP-9258|better test coverage for the filesystem APIs]]. This will ensure that we can keep the filesystem implementations that ship with Hadoop -HDFS itself, and those classes that connect to other filesystems, currently `s3:`, `s3n:`, `file:`, `ftp:`, `webhdfs`- consistent with each other, and compatible with existing applications. 
+ The Hadoop developers are (as of April 2013), attempting to [[https://issues.apache.org/jira/browse/HADOOP-9371|define the Semantics of the Hadoop FileSystem more rigorously]] as well as adding [[https://issues.apache.org/jira/browse/HADOOP-9258|better test coverage for the filesystem APIs]]. This will ensure that we can keep the filesystem implementations that ship with Hadoop -HDFS itself, and those classes that connect to other filesystems, currently `s3:`, `s3n:`, `file:`, `ftp:`, `webhdfs`- consistent with each other, and compatible with existing applications.
  
- This formalisation of the API will also benefit anyone who wishes to to provide a library that lets Hadoop applications work with their FileSystem -such people have been very constructive in helping define the FileSystem APIs more rigorously. 
+ This formalisation of the API will also benefit anyone who wishes to to provide a library that lets Hadoop applications work with their FileSystem -such people have been very constructive in helping define the FileSystem APIs more rigorously.
- 
  
  The list below contains links to information about some of the additional FileSystems and Object Stores for Apache Hadoop
  
  Known contributors actively commited to working on HCFS initiative... (Add your organization here) !
  
-  * [[https://github.com/apache/hadoop-hdfs | HDFS ]]
+  * [[https://github.com/apache/hadoop-hdfs|HDFS]]
-  * [[https://forge.gluster.org/hadoop | GlusterFS]]
+  * [[https://forge.gluster.org/hadoop|GlusterFS]]
-  * [[http://www.omnibond.com/orangefs/docs/v_2_8_8/index.htm#Hadoop_Client.htm | OrangeFS ]]
+  * [[http://www.omnibond.com/orangefs/docs/v_2_8_8/index.htm#Hadoop_Client.htm|OrangeFS]]
-  * [[https://github.com/steveloughran/Hadoop-and-Swift-integration/ | SwiftFS ]]
+  * [[https://github.com/steveloughran/Hadoop-and-Swift-integration/|SwiftFS]]
-  * [[http://www.gridgain.com/media/in-memory-hdfs-readme.pdf | GridGain ]]
+  * [[http://www.gridgain.com/media/in-memory-hdfs-readme.pdf|GridGain]]
  
  Other known members of the HCFS community:
  
-  /* Alphabetical order, no endorsements, please */.
+  . /* Alphabetical order, no endorsements, please */.  */
-  * [[https://issues.apache.org/jira/browse/HADOOP-9629 | Windows Azure Blob Storage]]
+  * [[https://issues.apache.org/jira/browse/HADOOP-9629|Windows Azure Blob Storage]]
-  * [[http://www.datastax.com/dev/blog/cassandra-file-system-design | CassandraFS]]
+  * [[http://www.datastax.com/dev/blog/cassandra-file-system-design|CassandraFS]]
-  * [[http://ceph.com/docs/next/cephfs/hadoop/ | CephFS ]]
+  * [[https://ceph.com/docs/master/cephfs/hadoop/|CephFS]]
-  * [[http://www.cleversafe.com/news-reviews/cleversafe-press-releases/2012-press-releases/cleversafe-first-to-deliver-breakthrough-capabilities-for-combined-storage-and-massive-computation | CleverSafe Object Store ]]
+  * [[http://www.cleversafe.com/news-reviews/cleversafe-press-releases/2012-press-releases/cleversafe-first-to-deliver-breakthrough-capabilities-for-combined-storage-and-massive-computation|CleverSafe Object Store]]
-  * [[https://developers.google.com/hadoop/google-cloud-storage-connector | Google Cloud Storage Connector ]]
+  * [[https://developers.google.com/hadoop/google-cloud-storage-connector|Google Cloud Storage Connector]]
-  * [[http://newsroom.intel.com/community/intel_newsroom/blog/2013/06/12/intel-expands-software-portfolio-for-big-data-solutions | Lustre ]]
+  * [[http://newsroom.intel.com/community/intel_newsroom/blog/2013/06/12/intel-expands-software-portfolio-for-big-data-solutions|Lustre]]
-  * [[http://answers.mapr.com/questions/116/is-mapr-wire-compatible-or-api-compatible-with-hadoop-0202 | MapR FileSystem]]
+  * [[http://answers.mapr.com/questions/116/is-mapr-wire-compatible-or-api-compatible-with-hadoop-0202|MapR FileSystem]]
-  * [[https://github.com/quantcast/qfs/wiki/Migration-Guide | Quantcast File System ]]
+  * [[https://github.com/quantcast/qfs/wiki/Migration-Guide|Quantcast File System]]
-  * [[http://www.symantec.com/enterprise-solution-for-hadoop | Symtantec Veritas Cluster File System]]
+  * [[http://www.symantec.com/enterprise-solution-for-hadoop|Symtantec Veritas Cluster File System]]
  
  Even if the filesystem is supported by a library for tight integration with Apache Hadoop, it may behave differently from what Hadoop and applications expect: this is something to explore with the supplier of the filesystem.