You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by yl...@apache.org on 2015/01/21 02:52:19 UTC
hadoop git commit: HDFS-7623. Add htrace configuration properties to
core-default.xml and update user doc about how to enable htrace. (yliu)
Repository: hadoop
Updated Branches:
refs/heads/branch-2 610aa71a1 -> 3f37354da
HDFS-7623. Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace. (yliu)
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/3f37354d
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/3f37354d
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/3f37354d
Branch: refs/heads/branch-2
Commit: 3f37354daae7703779b767b22e5328dc13fa65c6
Parents: 610aa71
Author: yliu <yl...@apache.org>
Authored: Wed Jan 21 00:57:51 2015 +0800
Committer: yliu <yl...@apache.org>
Committed: Wed Jan 21 00:57:51 2015 +0800
----------------------------------------------------------------------
.../src/main/resources/core-default.xml | 24 +++++++++++++++++++
.../hadoop-common/src/site/apt/Tracing.apt.vm | 25 +++++++++++++++-----
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt | 3 +++
3 files changed, 46 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/3f37354d/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml b/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
index 4ba2e5a..4a9c3da 100644
--- a/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
+++ b/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
@@ -1646,4 +1646,28 @@ for ldap providers in the same way as above does.
</description>
</property>
+<property>
+ <name>hadoop.htrace.sampler</name>
+ <value>NeverSampler</value>
+ <description>
+ Configure the samplers for HTrace, the value can be NeverSampler,
+ AlwaysSampler or ProbabilitySampler. NeverSampler: HTrace is OFF
+ for all spans; AlwaysSampler: HTrace is ON for all spans;
+ ProbabilitySampler: HTrace is ON for some percentage% of top-level
+ spans.
+ </description>
+</property>
+
+<property>
+ <name>hadoop.htrace.spanreceiver.classes</name>
+ <value></value>
+ <description>
+ A comma separated list of the fully-qualified class name of classes
+ implementing SpanReceiver. The tracing system works by collecting
+ information in structs called 'Spans'. It is up to you to choose
+ how you want to receive this information by implementing the
+ SpanReceiver interface.
+ </description>
+</property>
+
</configuration>
http://git-wip-us.apache.org/repos/asf/hadoop/blob/3f37354d/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm b/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm
index 9eda220..f04da33 100644
--- a/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm
+++ b/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm
@@ -16,19 +16,32 @@
---
${maven.build.timestamp}
-Enabling Dapper-like Tracing in HDFS
+Enabling Dapper-like Tracing in Hadoop
%{toc|section=1|fromDepth=0}
-* {Dapper-like Tracing in HDFS}
+* {Dapper-like Tracing in Hadoop}
** HTrace
{{{https://issues.apache.org/jira/browse/HDFS-5274}HDFS-5274}}
added support for tracing requests through HDFS,
- using the open source tracing library, {{{https://github.com/cloudera/htrace}HTrace}}.
+ using the open source tracing library, {{{https://git-wip-us.apache.org/repos/asf/incubator-htrace.git}Apache HTrace}}.
Setting up tracing is quite simple, however it requires some very minor changes to your client code.
+** Samplers
+ Configure the samplers in <<<core-site.xml>>> property: <<<hadoop.htrace.sampler>>>.
+ The value can be NeverSampler, AlwaysSampler or ProbabilitySampler. NeverSampler: HTrace is OFF
+ for all spans; AlwaysSampler: HTrace is ON for all spans; ProbabilitySampler: HTrace is ON for
+ some percentage% of top-level spans.
+
++----
+ <property>
+ <name>hadoop.htrace.sampler</name>
+ <value>NeverSampler</value>
+ </property>
++----
+
** SpanReceivers
The tracing system works by collecting information in structs called 'Spans'.
@@ -42,7 +55,7 @@ public void receiveSpan(Span span);
Configure what SpanReceivers you'd like to use
by putting a comma separated list of the fully-qualified class name of
classes implementing SpanReceiver
- in <<<hdfs-site.xml>>> property: <<<hadoop.htrace.spanreceiver.classes>>>.
+ in <<<core-site.xml>>> property: <<<hadoop.htrace.spanreceiver.classes>>>.
+----
<property>
@@ -83,11 +96,11 @@ public void receiveSpan(Span span);
$ git clone https://github.com/cloudera/htrace
$ cd htrace/htrace-zipkin
$ mvn compile assembly:single
- $ cp target/htrace-zipkin-*-jar-with-dependencies.jar $HADOOP_HOME/share/hadoop/hdfs/lib/
+ $ cp target/htrace-zipkin-*-jar-with-dependencies.jar $HADOOP_HOME/share/hadoop/common/lib/
+----
The sample configuration for <<<ZipkinSpanReceiver>>> is shown below.
- By adding these to <<<hdfs-site.xml>>> of NameNode and DataNodes,
+ By adding these to <<<core-site.xml>>> of NameNode and DataNodes,
<<<ZipkinSpanReceiver>>> is initialized on the startup.
You also need this configuration on the client node in addition to the servers.
http://git-wip-us.apache.org/repos/asf/hadoop/blob/3f37354d/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
index dc78530..53a23fd 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
+++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
@@ -258,6 +258,9 @@ Release 2.7.0 - UNRELEASED
HDFS-7640. Print NFS Client in the NFS log. (Brandon Li via wheat9)
+ HDFS-7623. Add htrace configuration properties to core-default.xml and
+ update user doc about how to enable htrace. (yliu)
+
OPTIMIZATIONS
HDFS-7454. Reduce memory footprint for AclEntries in NameNode.