You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-commits@hadoop.apache.org by to...@apache.org on 2011/05/19 21:05:31 UTC
svn commit: r1125057 - in /hadoop/hdfs/trunk: CHANGES.txt
src/docs/src/documentation/content/xdocs/hftp.xml
src/docs/src/documentation/content/xdocs/site.xml
Author: todd
Date: Thu May 19 19:05:30 2011
New Revision: 1125057
URL: http://svn.apache.org/viewvc?rev=1125057&view=rev
Log:
HDFS-1957. Add documentation for HFTP. Contributed by Ari Rabkin.
Added:
hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml
Modified:
hadoop/hdfs/trunk/CHANGES.txt
hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml
Modified: hadoop/hdfs/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/CHANGES.txt?rev=1125057&r1=1125056&r2=1125057&view=diff
==============================================================================
--- hadoop/hdfs/trunk/CHANGES.txt (original)
+++ hadoop/hdfs/trunk/CHANGES.txt Thu May 19 19:05:30 2011
@@ -848,6 +848,8 @@ Release 0.22.0 - Unreleased
HDFS-1947. DFSClient should use mapreduce.task.attempt.id. (eli)
+ HDFS-1957. Add documentation for HFTP. (Ari Rabkin via todd)
+
OPTIMIZATIONS
HDFS-1140. Speedup INode.getPathComponents. (Dmytro Molkov via shv)
Added: hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml
URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml?rev=1125057&view=auto
==============================================================================
--- hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml (added)
+++ hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml Thu May 19 19:05:30 2011
@@ -0,0 +1,69 @@
+<?xml version="1.0"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" "http://forrest.apache.org/dtd/document-v20.dtd">
+
+
+<document>
+
+ <header>
+ <title> HFTP Guide</title>
+ </header>
+
+ <body>
+ <section>
+ <title> Introduction </title>
+ <p> HFTP is a Hadoop filesystem implementation that lets you read data from a remote Hadoop HDFS cluster.
+ The reads are done via HTTP, and data is sourced from DataNodes.
+ HFTP is a read-only filesystem, and will throw exceptions if you try to use it to write data or modify
+ the filesystem state.</p>
+
+ <p>HFTP is primarily useful if you have multiple HDFS clusters with different versions and you need to move data from one to another. HFTP is wire-compatible even between different versions of HDFS. For example, you can do things like:
+ <code>hadoop distcp -i hftp://sourceFS:50070/src hdfs://destFS:50070/dest</code>. Note that HFTP is read-only so the destination must be an HDFS filesystem. (Also, in this example, the <code>distcp</code> should be run using the configuraton of the new filesystem.)</p>
+
+ <p>An extension, HSFTP, uses HTTPS by default. This means that data will be encrypted in transit.</p>
+ </section>
+
+ <section>
+ <title>Implementation</title>
+ <p>The code for HFTP lives in the Java class <code>org.apache.hadoop.hdfs.HftpFileSystem</code>. Likewise,
+ HSFTP is implemented in <code>org.apache.hadoop.hdfs.HsftpFileSystem</code>.
+ </p>
+ </section>
+
+ <section>
+ <title> Configuration Options </title>
+ <table>
+ <tr>
+ <th>Name</th>
+ <th>Description</th>
+ </tr>
+ <tr>
+ <td>dfs.hftp.https.port</td>
+ <td>the HTTPS port on the remote cluster. If not set, HFTP will fall back on
+ <code>dfs.https.port</code>.</td>
+ </tr>
+ <tr>
+ <td>hdfs.service.host_<strong>ip:port</strong></td>
+ <td>Specifies the service name (for the security subsystem) associated with the HFTP filesystem
+ running at <strong>ip:port.</strong></td>
+ </tr>
+ </table>
+ </section>
+ </body>
+</document>
Modified: hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml
URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml?rev=1125057&r1=1125056&r2=1125057&view=diff
==============================================================================
--- hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml (original)
+++ hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml Thu May 19 19:05:30 2011
@@ -44,6 +44,7 @@ See http://forrest.apache.org/docs/linki
<hdfs_imageviewer label="Offline Image Viewer" href="hdfs_imageviewer.html" />
<hdfs_editsviewer label="Offline Edits Viewer" href="hdfs_editsviewer.html" />
<hdfsproxy label="HDFS Proxy" href="hdfsproxy.html"/>
+ <hftp label="HFTP" href="hftp.html"/>
<faultinject_framework label="Fault Injection" href="faultinject_framework.html" />
<hdfs_libhdfs label="C API libhdfs" href="libhdfs.html" />
</docs>