You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by el...@apache.org on 2013/12/03 17:47:29 UTC
[1/2] git commit: ACCUMULO-1946 Note in the README about
dfs.datanode.synconclose,
with a slight amendment to hadoop version application.
Updated Branches:
refs/heads/1.5.1-SNAPSHOT cd96f85e0 -> 747c7cdbd
ACCUMULO-1946 Note in the README about dfs.datanode.synconclose, with a slight amendment to hadoop version application.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/47403ba4
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/47403ba4
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/47403ba4
Branch: refs/heads/1.5.1-SNAPSHOT
Commit: 47403ba4f5d552cabb94bc710e050dc114bcc922
Parents: 1b3d071
Author: Josh Elser <el...@apache.org>
Authored: Mon Dec 2 12:43:58 2013 -0500
Committer: Josh Elser <el...@apache.org>
Committed: Tue Dec 3 11:41:36 2013 -0500
----------------------------------------------------------------------
README | 5 +++++
1 file changed, 5 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/47403ba4/README
----------------------------------------------------------------------
diff --git a/README b/README
index 878bbe3..bc469fc 100644
--- a/README
+++ b/README
@@ -73,6 +73,11 @@ You will need to have hadoop installed and configured on your system.
Accumulo 1.4.5-SNAPSHOT has been tested with hadoop version 0.20.2 and
0.20.203.0.
+Additionally, it is strongly recommended that you enable 'dfs.datanode.synconclose'
+(only available in Apache Hadoop >=1.1.1 or >=0.23) in your hdfs-site.xml configuration
+file to ensure that, in the face of unexpected power loss to a datanode, files are
+wholly synced to disk.
+
The example accumulo configuration files are placed in directories based on the
memory footprint for the accumulo processes. If you are using native libraries
for you tablet server in-memory map, then you can use the files in "native-standalone".
[2/2] git commit: Merge branch '1.4.5-SNAPSHOT' into 1.5.1-SNAPSHOT
Posted by el...@apache.org.
Merge branch '1.4.5-SNAPSHOT' into 1.5.1-SNAPSHOT
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/747c7cdb
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/747c7cdb
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/747c7cdb
Branch: refs/heads/1.5.1-SNAPSHOT
Commit: 747c7cdbd38f33d5c611dded1d04b3adbe848cd8
Parents: cd96f85 47403ba
Author: Josh Elser <el...@apache.org>
Authored: Tue Dec 3 11:46:31 2013 -0500
Committer: Josh Elser <el...@apache.org>
Committed: Tue Dec 3 11:46:31 2013 -0500
----------------------------------------------------------------------
README | 5 +++++
1 file changed, 5 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/747c7cdb/README
----------------------------------------------------------------------
diff --cc README
index ab6f3fc,bc469fc..a100077
--- a/README
+++ b/README
@@@ -76,28 -69,15 +76,33 @@@ the machines in the cluster and that ha
found in the same location on every machine in the cluster. You will need to
have password-less ssh set up as described in the hadoop documentation.
-You will need to have hadoop installed and configured on your system.
-Accumulo 1.4.5-SNAPSHOT has been tested with hadoop version 0.20.2 and
-0.20.203.0.
+You will need to have hadoop installed and configured on your system. Accumulo
+1.5.0 has been tested with hadoop version 1.0.4. To avoid data loss,
+you must enable HDFS durable sync. How you enable this depends on your version
+of Hadoop. Please consult the table below for information regarding your version.
+If you need to set the coniguration, please be sure to restart HDFS. See
+ACCUMULO-623 for more information.
+
+HADOOP RELEASE VERSION SYNC NAME DEFAULT
+Apache Hadoop 0.20.205 dfs.support.append false
+Apache Hadoop 0.23.x dfs.support.append true
+Apache Hadoop 1.0.x dfs.support.append false
+Apache Hadoop 1.1.x dfs.durable.sync true
+Apache Hadoop 2.0.0-2.0.4 dfs.support.append true
+Cloudera CDH 3u0-3u3 ???? true
+Cloudera CDH 3u4 dfs.support.append true
+Hortonworks HDP `1.0 dfs.support.append false
+Hortonworks HDP `1.1 dfs.support.append false
+
+Additionally, it is strongly recommended that you enable 'dfs.datanode.synconclose'
+in your hdfs-site.xml configuration file to ensure that, in the face of unexpected
+power loss to a datanode, files are wholly synced to disk.
+ Additionally, it is strongly recommended that you enable 'dfs.datanode.synconclose'
+ (only available in Apache Hadoop >=1.1.1 or >=0.23) in your hdfs-site.xml configuration
+ file to ensure that, in the face of unexpected power loss to a datanode, files are
+ wholly synced to disk.
+
The example accumulo configuration files are placed in directories based on the
memory footprint for the accumulo processes. If you are using native libraries
for you tablet server in-memory map, then you can use the files in "native-standalone".