You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by el...@apache.org on 2015/05/01 22:40:13 UTC
[1/3] accumulo git commit: ACCUMULO-3766 Add some documentation about
using FQDN in config files
Repository: accumulo
Updated Branches:
refs/heads/1.7 fae7a8224 -> 2436eec23
refs/heads/master e25146d0d -> e7fc66caa
ACCUMULO-3766 Add some documentation about using FQDN in config files
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2436eec2
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2436eec2
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2436eec2
Branch: refs/heads/1.7
Commit: 2436eec237dc23608e91cae56777a323e2f77295
Parents: fae7a82
Author: Josh Elser <el...@apache.org>
Authored: Fri May 1 16:39:24 2015 -0400
Committer: Josh Elser <el...@apache.org>
Committed: Fri May 1 16:39:24 2015 -0400
----------------------------------------------------------------------
docs/src/main/asciidoc/chapters/administration.txt | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2436eec2/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt
index 857d948..cbe7ed2 100644
--- a/docs/src/main/asciidoc/chapters/administration.txt
+++ b/docs/src/main/asciidoc/chapters/administration.txt
@@ -238,6 +238,23 @@ to load Hadoop from multiple potential locations depending on how Hadoop was ins
recommended that `general.classpaths` is defined and limited to only the necessary jars to prevent
extra jars from being unintentionally loaded into Accumulo processes.
+==== Hostnames in configuration files
+
+Accumulo has a number of configuration files which can contain references to other hosts in your
+network. All of the "host" configuration files for Accumulo (+gc+, +masters+, +slaves+, +monitor+,
++tracers+) as well as +instance.volumes+ in accumulo-site.xml must contain some host reference.
+
+While IP address, short hostnames, or fully qualified domain names (FQDN) are all technically valid, it
+is good practice to always use FQDNs for both Accumulo and other processes in your Hadoop cluster.
+Failing to consistently use FQDNs can have unexpected consequences in how Accumulo uses the FileSystem.
+
+A common way for this problem can be observed is via applications that use Bulk Ingest. The Accumulo
+Master coordinates moving the input files to Bulk Ingest to an Accumulo-managed directory. However,
+Accumulo cannot safely move files across different Hadoop FileSystems. This is problematic because
+Accumulo also cannot make reliable assertions across what is the same FileSystem which is specified
+with different names. Naively, while 127.0.0.1:8020 might be a valid identifier for an HDFS instance,
+Accumulo identifies +localhost:8020+ as a different HDFS instance than +127.0.0.1:8020+.
+
==== Deploy Configuration
Copy the masters, slaves, accumulo-env.sh, and if necessary, accumulo-site.xml
[3/3] accumulo git commit: Merge branch '1.7'
Posted by el...@apache.org.
Merge branch '1.7'
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/e7fc66ca
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/e7fc66ca
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/e7fc66ca
Branch: refs/heads/master
Commit: e7fc66caa9fbe20d35eaf3f2a9082c5119bbcf83
Parents: e25146d 2436eec
Author: Josh Elser <el...@apache.org>
Authored: Fri May 1 16:40:03 2015 -0400
Committer: Josh Elser <el...@apache.org>
Committed: Fri May 1 16:40:03 2015 -0400
----------------------------------------------------------------------
docs/src/main/asciidoc/chapters/administration.txt | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
----------------------------------------------------------------------
[2/3] accumulo git commit: ACCUMULO-3766 Add some documentation about
using FQDN in config files
Posted by el...@apache.org.
ACCUMULO-3766 Add some documentation about using FQDN in config files
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2436eec2
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2436eec2
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2436eec2
Branch: refs/heads/master
Commit: 2436eec237dc23608e91cae56777a323e2f77295
Parents: fae7a82
Author: Josh Elser <el...@apache.org>
Authored: Fri May 1 16:39:24 2015 -0400
Committer: Josh Elser <el...@apache.org>
Committed: Fri May 1 16:39:24 2015 -0400
----------------------------------------------------------------------
docs/src/main/asciidoc/chapters/administration.txt | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2436eec2/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt
index 857d948..cbe7ed2 100644
--- a/docs/src/main/asciidoc/chapters/administration.txt
+++ b/docs/src/main/asciidoc/chapters/administration.txt
@@ -238,6 +238,23 @@ to load Hadoop from multiple potential locations depending on how Hadoop was ins
recommended that `general.classpaths` is defined and limited to only the necessary jars to prevent
extra jars from being unintentionally loaded into Accumulo processes.
+==== Hostnames in configuration files
+
+Accumulo has a number of configuration files which can contain references to other hosts in your
+network. All of the "host" configuration files for Accumulo (+gc+, +masters+, +slaves+, +monitor+,
++tracers+) as well as +instance.volumes+ in accumulo-site.xml must contain some host reference.
+
+While IP address, short hostnames, or fully qualified domain names (FQDN) are all technically valid, it
+is good practice to always use FQDNs for both Accumulo and other processes in your Hadoop cluster.
+Failing to consistently use FQDNs can have unexpected consequences in how Accumulo uses the FileSystem.
+
+A common way for this problem can be observed is via applications that use Bulk Ingest. The Accumulo
+Master coordinates moving the input files to Bulk Ingest to an Accumulo-managed directory. However,
+Accumulo cannot safely move files across different Hadoop FileSystems. This is problematic because
+Accumulo also cannot make reliable assertions across what is the same FileSystem which is specified
+with different names. Naively, while 127.0.0.1:8020 might be a valid identifier for an HDFS instance,
+Accumulo identifies +localhost:8020+ as a different HDFS instance than +127.0.0.1:8020+.
+
==== Deploy Configuration
Copy the masters, slaves, accumulo-env.sh, and if necessary, accumulo-site.xml