You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by el...@apache.org on 2015/08/28 05:33:58 UTC
[1/9] accumulo git commit: ACCUMULO-3971 Add more tracing doc to user
manual
Repository: accumulo
Updated Branches:
refs/heads/1.7 d77863e04 -> 1761200a9
refs/heads/master b38371d9a -> 948d8497f
ACCUMULO-3971 Add more tracing doc to user manual
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/cf5ef46f
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/cf5ef46f
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/cf5ef46f
Branch: refs/heads/1.7
Commit: cf5ef46f3a450c50aa804445d4a82fbabb18dacd
Parents: da484a8
Author: Dylan Hutchison <dh...@mit.edu>
Authored: Sun Aug 23 19:50:24 2015 -0400
Committer: Dylan Hutchison <dh...@mit.edu>
Committed: Sun Aug 23 21:25:10 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 52 +++++++++++++++++++-
1 file changed, 50 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/cf5ef46f/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt
index f5f16bb..1373f89 100644
--- a/docs/src/main/asciidoc/chapters/administration.txt
+++ b/docs/src/main/asciidoc/chapters/administration.txt
@@ -306,6 +306,7 @@ A KeyStore can also be stored in HDFS, which will make the KeyStore readily avai
all Accumulo servers. If the local filesystem is used, be aware that each Accumulo server
will expect the KeyStore in the same location.
+[[ClientConfiguration]]
==== Client Configuration
In version 1.6.0, Accumulo includes a new type of configuration file known as a client
@@ -536,6 +537,7 @@ To collect traces, Accumulo needs at least one server listed in
from clients and writes them to the +trace+ table. The Accumulo
user that the tracer connects to Accumulo with can be configured with
the following properties
+(see the <<configuration,Configuration>> section for setting Accumulo server properties)
trace.user
trace.token.property.password
@@ -610,6 +612,47 @@ Hadoop to send traces to ZooTraceClient is
The accumulo-core, accumulo-tracer, accumulo-fate and libthrift
jars must also be placed on Hadoop's classpath.
+===== Adding additional SpanReceivers
+https://github.com/openzipkin/zipkin[Zipkin]
+has a SpanReceiver supported by HTrace and popularized by Twitter
+that users looking for a more graphical trace display may opt to use.
+The following steps configure Accumulo to use org.apache.htrace.impl.ZipkinSpanReceiver
+in addition to the Accumulo's default ZooTraceClient, and they serve as a template
+for adding any SpanReceiver to Accumulo:
+
+1. Add the Jar containing the ZipkinSpanReceiver class file to the
++$ACCUMULO_HOME/lib/+. It is critical that the Jar is placed in
++lib/+ and NOT in +lib/ext/+ so that the new SpanReceiver class
+is visible to the same class loader of htrace-core.
+
+2. Add the following to +$ACCUMULO_HOME/conf/accumulo-site.xml+:
+
+ <property>
+ <name>trace.span.receivers</name>
+ <value>org.apache.accumulo.tracer.ZooTraceClient,org.apache.htrace.impl.ZipkinSpanReceiver</value>
+ </property>
+
+3. Restart your Accumulo tablet servers.
+
+In order to use ZipkinSpanReceiver from a client as well as the Accumulo server,
+
+1. Ensure your client can see the ZipkinSpanReceiver class at runtime. For Maven projects,
+this is easily done by adding to your client's pom.xml (taking care to specify a good version)
+
+ <dependency>
+ <groupId>org.apache.htrace</groupId>
+ <artifactId>htrace-zipkin</artifactId>
+ <version>3.1.0-incubating</version>
+ <scope>runtime</scope>
+ </dependency>
+
+2. Add the following to your ClientConfiguration
+(see the <<ClientConfiguration>> section)
+
+ trace.span.receivers=org.apache.accumulo.tracer.ZooTraceClient,org.apache.htrace.impl.ZipkinSpanReceiver
+
+3. Instrument your client as in the next section.
+
==== Instrumenting a Client
Tracing can be used to measure a client operation, such as a scan, as
the operation traverses the distributed system. To enable tracing for
@@ -637,8 +680,13 @@ for (Entry entry : scanner) {
}
scope.close();
-Additionally, the user can create additional Spans within a Trace.
-The sampler for the trace should only be specified with the first span, and subsequent spans will be collected depending on whether that first span was sampled.
+The user can create additional Spans within a Trace.
+
+The sampler (such as +Sampler.ALWAYS+) for the trace should only be specified with a top-level span,
+and subsequent spans will be collected depending on whether that first span was sampled.
+Don't forget to specify a Sampler at the top-level span
+because the default Sampler only samples when part of a pre-existing trace,
+which will never occur in a client that never specifies a Sampler.
[source,java]
TraceScope scope = Trace.startSpan("Client Update", Sampler.ALWAYS);
[7/9] accumulo git commit: Merge remote-tracking branch
'dhutchis/ACCUMULO-3971' into 1.7
Posted by el...@apache.org.
Merge remote-tracking branch 'dhutchis/ACCUMULO-3971' into 1.7
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/1761200a
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/1761200a
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/1761200a
Branch: refs/heads/master
Commit: 1761200a9cbea16e2a803e9c07a7917fb6ce455c
Parents: d77863e 2e644f2
Author: Josh Elser <el...@apache.org>
Authored: Thu Aug 27 23:29:38 2015 -0400
Committer: Josh Elser <el...@apache.org>
Committed: Thu Aug 27 23:29:38 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 126 ++++++++++++++++++-
1 file changed, 124 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
[2/9] accumulo git commit: ACCUMULO-3971 Add more tracing doc to user
manual
Posted by el...@apache.org.
ACCUMULO-3971 Add more tracing doc to user manual
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/cf5ef46f
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/cf5ef46f
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/cf5ef46f
Branch: refs/heads/master
Commit: cf5ef46f3a450c50aa804445d4a82fbabb18dacd
Parents: da484a8
Author: Dylan Hutchison <dh...@mit.edu>
Authored: Sun Aug 23 19:50:24 2015 -0400
Committer: Dylan Hutchison <dh...@mit.edu>
Committed: Sun Aug 23 21:25:10 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 52 +++++++++++++++++++-
1 file changed, 50 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/cf5ef46f/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt
index f5f16bb..1373f89 100644
--- a/docs/src/main/asciidoc/chapters/administration.txt
+++ b/docs/src/main/asciidoc/chapters/administration.txt
@@ -306,6 +306,7 @@ A KeyStore can also be stored in HDFS, which will make the KeyStore readily avai
all Accumulo servers. If the local filesystem is used, be aware that each Accumulo server
will expect the KeyStore in the same location.
+[[ClientConfiguration]]
==== Client Configuration
In version 1.6.0, Accumulo includes a new type of configuration file known as a client
@@ -536,6 +537,7 @@ To collect traces, Accumulo needs at least one server listed in
from clients and writes them to the +trace+ table. The Accumulo
user that the tracer connects to Accumulo with can be configured with
the following properties
+(see the <<configuration,Configuration>> section for setting Accumulo server properties)
trace.user
trace.token.property.password
@@ -610,6 +612,47 @@ Hadoop to send traces to ZooTraceClient is
The accumulo-core, accumulo-tracer, accumulo-fate and libthrift
jars must also be placed on Hadoop's classpath.
+===== Adding additional SpanReceivers
+https://github.com/openzipkin/zipkin[Zipkin]
+has a SpanReceiver supported by HTrace and popularized by Twitter
+that users looking for a more graphical trace display may opt to use.
+The following steps configure Accumulo to use org.apache.htrace.impl.ZipkinSpanReceiver
+in addition to the Accumulo's default ZooTraceClient, and they serve as a template
+for adding any SpanReceiver to Accumulo:
+
+1. Add the Jar containing the ZipkinSpanReceiver class file to the
++$ACCUMULO_HOME/lib/+. It is critical that the Jar is placed in
++lib/+ and NOT in +lib/ext/+ so that the new SpanReceiver class
+is visible to the same class loader of htrace-core.
+
+2. Add the following to +$ACCUMULO_HOME/conf/accumulo-site.xml+:
+
+ <property>
+ <name>trace.span.receivers</name>
+ <value>org.apache.accumulo.tracer.ZooTraceClient,org.apache.htrace.impl.ZipkinSpanReceiver</value>
+ </property>
+
+3. Restart your Accumulo tablet servers.
+
+In order to use ZipkinSpanReceiver from a client as well as the Accumulo server,
+
+1. Ensure your client can see the ZipkinSpanReceiver class at runtime. For Maven projects,
+this is easily done by adding to your client's pom.xml (taking care to specify a good version)
+
+ <dependency>
+ <groupId>org.apache.htrace</groupId>
+ <artifactId>htrace-zipkin</artifactId>
+ <version>3.1.0-incubating</version>
+ <scope>runtime</scope>
+ </dependency>
+
+2. Add the following to your ClientConfiguration
+(see the <<ClientConfiguration>> section)
+
+ trace.span.receivers=org.apache.accumulo.tracer.ZooTraceClient,org.apache.htrace.impl.ZipkinSpanReceiver
+
+3. Instrument your client as in the next section.
+
==== Instrumenting a Client
Tracing can be used to measure a client operation, such as a scan, as
the operation traverses the distributed system. To enable tracing for
@@ -637,8 +680,13 @@ for (Entry entry : scanner) {
}
scope.close();
-Additionally, the user can create additional Spans within a Trace.
-The sampler for the trace should only be specified with the first span, and subsequent spans will be collected depending on whether that first span was sampled.
+The user can create additional Spans within a Trace.
+
+The sampler (such as +Sampler.ALWAYS+) for the trace should only be specified with a top-level span,
+and subsequent spans will be collected depending on whether that first span was sampled.
+Don't forget to specify a Sampler at the top-level span
+because the default Sampler only samples when part of a pre-existing trace,
+which will never occur in a client that never specifies a Sampler.
[source,java]
TraceScope scope = Trace.startSpan("Client Update", Sampler.ALWAYS);
[3/9] accumulo git commit: ACCUMULO-3971 Add trace table format and
utilities to manual
Posted by el...@apache.org.
ACCUMULO-3971 Add trace table format and utilities to manual
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/16db7873
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/16db7873
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/16db7873
Branch: refs/heads/1.7
Commit: 16db78737a2d4438f34eb72a8c7f0e785f17a280
Parents: cf5ef46
Author: Dylan Hutchison <dh...@mit.edu>
Authored: Sun Aug 23 21:25:30 2015 -0400
Committer: Dylan Hutchison <dh...@mit.edu>
Committed: Sun Aug 23 21:26:03 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 61 ++++++++++++++++++++
.../shell/commands/ActiveScanIterator.java | 9 ++-
2 files changed, 65 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/16db7873/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt
index 1373f89..da98409 100644
--- a/docs/src/main/asciidoc/chapters/administration.txt
+++ b/docs/src/main/asciidoc/chapters/administration.txt
@@ -745,6 +745,67 @@ To view collected traces, use the "Recent Traces" link on the Monitor
UI. You can also programmatically access and print traces using the
+TraceDump+ class.
+===== Trace Table Format
+This section is for developers looking to use data recorded in the trace table
+directly, above and beyond the default services of the Accumulo monitor.
+Please note the trace table format and its supporting classes
+are not in the public API and may be subject to change in future versions.
+
+Each span received by a tracer's ZooTraceClient is recorded in the trace table
+in the form of three entries: span entries, index entries, and start time entries.
+Span entries record full span information,
+whereas index and start time entries provide indexing into span information
+useful for quickly finding spans by type or start time.
+
+Each entry is illustrated by a description and sample of data.
+In the description, a token in quotes is a String literal,
+whereas other other tokens are span variables.
+Parentheses group parts together, to distinguish colon characters inside the
+column family or qualifier from the colon that separates column family and qualifier.
+We use the format +row columnFamily:columnQualifier columnVisibility value+
+(omitting timestamp which records the time an entry is written to the trace table).
+
+Span entries take the following form:
+
+ traceId "span":(parentSpanId:spanId) [] spanBinaryEncoding
+ 63b318de80de96d1 span:4b8f66077df89de1:3778c6739afe4e1 [] %18;%09;...
+
+The parentSpanId is "" for the root span of a trace.
+The spanBinaryEncoding is a compact Apache Thrift encoding of the original Span object.
+This allows clients (and the Accumulo monitor) to recover all the details of the original Span
+at a later time, by scanning the trace table and decoding the value of span entries
+via +TraceFormatter.getRemoteSpan(entry)+.
+
+The trace table has a formatter class by default (org.apache.accumulo.tracer.TraceFormatter)
+that changes how span entries appear from the Accumulo shell.
+Normal scans to the trace table do not use this formatter representation;
+it exists only to make span entries easier to view inside the Accumulo shell.
+
+Index entries take the following form:
+
+ "idx":service:startTime description:sender [] traceId:elapsedTime
+ idx:tserver:14f3828f58b startScan:localhost [] 63b318de80de96d1:1
+
+The service and sender are set by the first call of each Accumulo process
+(and instrumented client processes) to +DistributedTrace.enable(...)+
+(the sender is autodetected if not specified).
+The description is specified in each span.
+Start time and the elapsed time (start - stop, 1 millisecond in the example above)
+are recorded in milliseconds as long values serialized to a string in hex.
+
+Start time entries take the following form:
+
+ "start":startTime "id":traceId [] spanBinaryEncoding
+ start:14f3828a351 id:63b318de80de96d1 [] %18;%09;...
+
+The following classes may be run from $ACCUMULO_HOME while Accumulo is running
+to provide insight into trace statistics. These require
+accumulo-trace-VERSION.jar to be provided on the Accumulo classpath
+(+$ACCUMULO_HOME/lib/ext+ is fine).
+
+ $ bin/accumulo org.apache.accumulo.tracer.TraceTableStats -u username -p password -i instancename
+ $ bin/accumulo org.apache.accumulo.tracer.TraceDump -u username -p password -i instancename -r
+
==== Tracing from the Shell
You can enable tracing for operations run from the shell by using the
+trace on+ and +trace off+ commands.
http://git-wip-us.apache.org/repos/asf/accumulo/blob/16db7873/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java
----------------------------------------------------------------------
diff --git a/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java b/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java
index 9f2e23b..1089d78 100644
--- a/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java
+++ b/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java
@@ -42,11 +42,10 @@ class ActiveScanIterator implements Iterator<String> {
final List<ActiveScan> asl = instanceOps.getActiveScans(tserver);
for (ActiveScan as : asl) {
- scans
- .add(String.format("%21s |%21s |%9s |%9s |%7s |%6s |%8s |%8s |%10s |%20s |%10s |%20s |%10s | %s", tserver, as.getClient(),
- Duration.format(as.getAge(), "", "-"), Duration.format(as.getLastContactTime(), "", "-"), as.getState(), as.getType(), as.getUser(),
- as.getTable(), as.getColumns(), as.getAuthorizations(), (as.getType() == ScanType.SINGLE ? as.getTablet() : "N/A"), as.getScanid(),
- as.getSsiList(), as.getSsio()));
+ scans.add(String.format("%21s |%21s |%9s |%9s |%7s |%6s |%8s |%8s |%10s |%20s |%10s |%20s |%10s | %s", tserver, as.getClient(),
+ Duration.format(as.getAge(), "", "-"), Duration.format(as.getLastContactTime(), "", "-"), as.getState(), as.getType(), as.getUser(),
+ as.getTable(), as.getColumns(), as.getAuthorizations(), (as.getType() == ScanType.SINGLE ? as.getTablet() : "N/A"), as.getScanid(),
+ as.getSsiList(), as.getSsio()));
}
} catch (Exception e) {
scans.add(tserver + " ERROR " + e.getMessage());
[8/9] accumulo git commit: Merge remote-tracking branch
'dhutchis/ACCUMULO-3971' into 1.7
Posted by el...@apache.org.
Merge remote-tracking branch 'dhutchis/ACCUMULO-3971' into 1.7
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/1761200a
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/1761200a
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/1761200a
Branch: refs/heads/1.7
Commit: 1761200a9cbea16e2a803e9c07a7917fb6ce455c
Parents: d77863e 2e644f2
Author: Josh Elser <el...@apache.org>
Authored: Thu Aug 27 23:29:38 2015 -0400
Committer: Josh Elser <el...@apache.org>
Committed: Thu Aug 27 23:29:38 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 126 ++++++++++++++++++-
1 file changed, 124 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
[5/9] accumulo git commit: ACCUMULO-3971 Fix wording and add other
Zipkin params
Posted by el...@apache.org.
ACCUMULO-3971 Fix wording and add other Zipkin params
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2e644f2e
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2e644f2e
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2e644f2e
Branch: refs/heads/1.7
Commit: 2e644f2e63bc208b83219862b40be6667994d805
Parents: 16db787
Author: Dylan Hutchison <dh...@mit.edu>
Authored: Mon Aug 24 19:17:06 2015 -0400
Committer: Dylan Hutchison <dh...@mit.edu>
Committed: Mon Aug 24 19:17:06 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 19 ++++++++++++++++---
1 file changed, 16 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2e644f2e/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt
index da98409..0a29711 100644
--- a/docs/src/main/asciidoc/chapters/administration.txt
+++ b/docs/src/main/asciidoc/chapters/administration.txt
@@ -616,7 +616,7 @@ jars must also be placed on Hadoop's classpath.
https://github.com/openzipkin/zipkin[Zipkin]
has a SpanReceiver supported by HTrace and popularized by Twitter
that users looking for a more graphical trace display may opt to use.
-The following steps configure Accumulo to use org.apache.htrace.impl.ZipkinSpanReceiver
+The following steps configure Accumulo to use +org.apache.htrace.impl.ZipkinSpanReceiver+
in addition to the Accumulo's default ZooTraceClient, and they serve as a template
for adding any SpanReceiver to Accumulo:
@@ -653,6 +653,19 @@ this is easily done by adding to your client's pom.xml (taking care to specify a
3. Instrument your client as in the next section.
+Your SpanReceiver may require additional properties, and if so these should likewise
+be placed in the ClientConfiguration (if applicable) and Accumulo's +accumulo-site.xml+.
+Two such properties for ZipkinSpanReceiver, listed with their default values, are
+
+ <property>
+ <name>trace.span.receiver.zipkin.collector-hostname</name>
+ <value>localhost</value>
+ </property>
+ <property>
+ <name>trace.span.receiver.zipkin.collector-port</name>
+ <value>9410</value>
+ </property>
+
==== Instrumenting a Client
Tracing can be used to measure a client operation, such as a scan, as
the operation traverses the distributed system. To enable tracing for
@@ -753,8 +766,8 @@ are not in the public API and may be subject to change in future versions.
Each span received by a tracer's ZooTraceClient is recorded in the trace table
in the form of three entries: span entries, index entries, and start time entries.
-Span entries record full span information,
-whereas index and start time entries provide indexing into span information
+Span and start time entries record full span information,
+whereas index entries provide indexing into span information
useful for quickly finding spans by type or start time.
Each entry is illustrated by a description and sample of data.
[6/9] accumulo git commit: ACCUMULO-3971 Fix wording and add other
Zipkin params
Posted by el...@apache.org.
ACCUMULO-3971 Fix wording and add other Zipkin params
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2e644f2e
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2e644f2e
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2e644f2e
Branch: refs/heads/master
Commit: 2e644f2e63bc208b83219862b40be6667994d805
Parents: 16db787
Author: Dylan Hutchison <dh...@mit.edu>
Authored: Mon Aug 24 19:17:06 2015 -0400
Committer: Dylan Hutchison <dh...@mit.edu>
Committed: Mon Aug 24 19:17:06 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 19 ++++++++++++++++---
1 file changed, 16 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2e644f2e/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt
index da98409..0a29711 100644
--- a/docs/src/main/asciidoc/chapters/administration.txt
+++ b/docs/src/main/asciidoc/chapters/administration.txt
@@ -616,7 +616,7 @@ jars must also be placed on Hadoop's classpath.
https://github.com/openzipkin/zipkin[Zipkin]
has a SpanReceiver supported by HTrace and popularized by Twitter
that users looking for a more graphical trace display may opt to use.
-The following steps configure Accumulo to use org.apache.htrace.impl.ZipkinSpanReceiver
+The following steps configure Accumulo to use +org.apache.htrace.impl.ZipkinSpanReceiver+
in addition to the Accumulo's default ZooTraceClient, and they serve as a template
for adding any SpanReceiver to Accumulo:
@@ -653,6 +653,19 @@ this is easily done by adding to your client's pom.xml (taking care to specify a
3. Instrument your client as in the next section.
+Your SpanReceiver may require additional properties, and if so these should likewise
+be placed in the ClientConfiguration (if applicable) and Accumulo's +accumulo-site.xml+.
+Two such properties for ZipkinSpanReceiver, listed with their default values, are
+
+ <property>
+ <name>trace.span.receiver.zipkin.collector-hostname</name>
+ <value>localhost</value>
+ </property>
+ <property>
+ <name>trace.span.receiver.zipkin.collector-port</name>
+ <value>9410</value>
+ </property>
+
==== Instrumenting a Client
Tracing can be used to measure a client operation, such as a scan, as
the operation traverses the distributed system. To enable tracing for
@@ -753,8 +766,8 @@ are not in the public API and may be subject to change in future versions.
Each span received by a tracer's ZooTraceClient is recorded in the trace table
in the form of three entries: span entries, index entries, and start time entries.
-Span entries record full span information,
-whereas index and start time entries provide indexing into span information
+Span and start time entries record full span information,
+whereas index entries provide indexing into span information
useful for quickly finding spans by type or start time.
Each entry is illustrated by a description and sample of data.
[9/9] accumulo git commit: Merge branch '1.7'
Posted by el...@apache.org.
Merge branch '1.7'
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/948d8497
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/948d8497
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/948d8497
Branch: refs/heads/master
Commit: 948d8497f5c33c5aa80fe3e1cab9378bafa55bbf
Parents: b38371d 1761200
Author: Josh Elser <el...@apache.org>
Authored: Thu Aug 27 23:33:22 2015 -0400
Committer: Josh Elser <el...@apache.org>
Committed: Thu Aug 27 23:33:22 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 126 ++++++++++++++++++-
1 file changed, 124 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/948d8497/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
[4/9] accumulo git commit: ACCUMULO-3971 Add trace table format and
utilities to manual
Posted by el...@apache.org.
ACCUMULO-3971 Add trace table format and utilities to manual
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/16db7873
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/16db7873
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/16db7873
Branch: refs/heads/master
Commit: 16db78737a2d4438f34eb72a8c7f0e785f17a280
Parents: cf5ef46
Author: Dylan Hutchison <dh...@mit.edu>
Authored: Sun Aug 23 21:25:30 2015 -0400
Committer: Dylan Hutchison <dh...@mit.edu>
Committed: Sun Aug 23 21:26:03 2015 -0400
----------------------------------------------------------------------
.../main/asciidoc/chapters/administration.txt | 61 ++++++++++++++++++++
.../shell/commands/ActiveScanIterator.java | 9 ++-
2 files changed, 65 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/16db7873/docs/src/main/asciidoc/chapters/administration.txt
----------------------------------------------------------------------
diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt
index 1373f89..da98409 100644
--- a/docs/src/main/asciidoc/chapters/administration.txt
+++ b/docs/src/main/asciidoc/chapters/administration.txt
@@ -745,6 +745,67 @@ To view collected traces, use the "Recent Traces" link on the Monitor
UI. You can also programmatically access and print traces using the
+TraceDump+ class.
+===== Trace Table Format
+This section is for developers looking to use data recorded in the trace table
+directly, above and beyond the default services of the Accumulo monitor.
+Please note the trace table format and its supporting classes
+are not in the public API and may be subject to change in future versions.
+
+Each span received by a tracer's ZooTraceClient is recorded in the trace table
+in the form of three entries: span entries, index entries, and start time entries.
+Span entries record full span information,
+whereas index and start time entries provide indexing into span information
+useful for quickly finding spans by type or start time.
+
+Each entry is illustrated by a description and sample of data.
+In the description, a token in quotes is a String literal,
+whereas other other tokens are span variables.
+Parentheses group parts together, to distinguish colon characters inside the
+column family or qualifier from the colon that separates column family and qualifier.
+We use the format +row columnFamily:columnQualifier columnVisibility value+
+(omitting timestamp which records the time an entry is written to the trace table).
+
+Span entries take the following form:
+
+ traceId "span":(parentSpanId:spanId) [] spanBinaryEncoding
+ 63b318de80de96d1 span:4b8f66077df89de1:3778c6739afe4e1 [] %18;%09;...
+
+The parentSpanId is "" for the root span of a trace.
+The spanBinaryEncoding is a compact Apache Thrift encoding of the original Span object.
+This allows clients (and the Accumulo monitor) to recover all the details of the original Span
+at a later time, by scanning the trace table and decoding the value of span entries
+via +TraceFormatter.getRemoteSpan(entry)+.
+
+The trace table has a formatter class by default (org.apache.accumulo.tracer.TraceFormatter)
+that changes how span entries appear from the Accumulo shell.
+Normal scans to the trace table do not use this formatter representation;
+it exists only to make span entries easier to view inside the Accumulo shell.
+
+Index entries take the following form:
+
+ "idx":service:startTime description:sender [] traceId:elapsedTime
+ idx:tserver:14f3828f58b startScan:localhost [] 63b318de80de96d1:1
+
+The service and sender are set by the first call of each Accumulo process
+(and instrumented client processes) to +DistributedTrace.enable(...)+
+(the sender is autodetected if not specified).
+The description is specified in each span.
+Start time and the elapsed time (start - stop, 1 millisecond in the example above)
+are recorded in milliseconds as long values serialized to a string in hex.
+
+Start time entries take the following form:
+
+ "start":startTime "id":traceId [] spanBinaryEncoding
+ start:14f3828a351 id:63b318de80de96d1 [] %18;%09;...
+
+The following classes may be run from $ACCUMULO_HOME while Accumulo is running
+to provide insight into trace statistics. These require
+accumulo-trace-VERSION.jar to be provided on the Accumulo classpath
+(+$ACCUMULO_HOME/lib/ext+ is fine).
+
+ $ bin/accumulo org.apache.accumulo.tracer.TraceTableStats -u username -p password -i instancename
+ $ bin/accumulo org.apache.accumulo.tracer.TraceDump -u username -p password -i instancename -r
+
==== Tracing from the Shell
You can enable tracing for operations run from the shell by using the
+trace on+ and +trace off+ commands.
http://git-wip-us.apache.org/repos/asf/accumulo/blob/16db7873/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java
----------------------------------------------------------------------
diff --git a/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java b/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java
index 9f2e23b..1089d78 100644
--- a/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java
+++ b/shell/src/main/java/org/apache/accumulo/shell/commands/ActiveScanIterator.java
@@ -42,11 +42,10 @@ class ActiveScanIterator implements Iterator<String> {
final List<ActiveScan> asl = instanceOps.getActiveScans(tserver);
for (ActiveScan as : asl) {
- scans
- .add(String.format("%21s |%21s |%9s |%9s |%7s |%6s |%8s |%8s |%10s |%20s |%10s |%20s |%10s | %s", tserver, as.getClient(),
- Duration.format(as.getAge(), "", "-"), Duration.format(as.getLastContactTime(), "", "-"), as.getState(), as.getType(), as.getUser(),
- as.getTable(), as.getColumns(), as.getAuthorizations(), (as.getType() == ScanType.SINGLE ? as.getTablet() : "N/A"), as.getScanid(),
- as.getSsiList(), as.getSsio()));
+ scans.add(String.format("%21s |%21s |%9s |%9s |%7s |%6s |%8s |%8s |%10s |%20s |%10s |%20s |%10s | %s", tserver, as.getClient(),
+ Duration.format(as.getAge(), "", "-"), Duration.format(as.getLastContactTime(), "", "-"), as.getState(), as.getType(), as.getUser(),
+ as.getTable(), as.getColumns(), as.getAuthorizations(), (as.getType() == ScanType.SINGLE ? as.getTablet() : "N/A"), as.getScanid(),
+ as.getSsiList(), as.getSsio()));
}
} catch (Exception e) {
scans.add(tserver + " ERROR " + e.getMessage());