You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/02/13 08:04:24 UTC
[GitHub] [hadoop] smengcl opened a new pull request #1846: HADOOP-15566.
Support Opentracing
smengcl opened a new pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846
Replacing HTrace with OpenTracing.
TODO:
1. Maybe remove the shim layer and use the OpenTracing calls directly -> this leads to more diff lines but should reduce the overhead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on issue #1846: HADOOP-15566. Support
Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-587430087
HADOOP-16824 is blocking this PR.
After HADOOP-16824 is committed, trigger the build and tests again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1846: HADOOP-15566.
Support Opentracing
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-588013386
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 7s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 1s | trunk passed |
| +1 :green_heart: | compile | 17m 50s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 13s | trunk passed |
| +1 :green_heart: | mvnsite | 5m 55s | trunk passed |
| +1 :green_heart: | shadedclient | 24m 38s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 4m 45s | trunk passed |
| +0 :ok: | spotbugs | 0m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +0 :ok: | findbugs | 0m 27s | branch/hadoop-project no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 27s | branch/hadoop-client-modules/hadoop-client-runtime no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 29s | branch/hadoop-client-modules/hadoop-client-check-invariants no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 35s | branch/hadoop-client-modules/hadoop-client-check-test-invariants no findbugs output file (findbugsXml.xml) |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 37s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvninstall | 0m 33s | hadoop-hdfs-client in the patch failed. |
| -1 :x: | compile | 0m 34s | root in the patch failed. |
| -1 :x: | cc | 0m 34s | root in the patch failed. |
| -1 :x: | javac | 0m 34s | root in the patch failed. |
| -0 :warning: | checkstyle | 0m 21s | The patch fails to run checkstyle in root |
| -1 :x: | mvnsite | 0m 12s | hadoop-common in the patch failed. |
| -1 :x: | mvnsite | 0m 37s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvnsite | 0m 34s | hadoop-hdfs-client in the patch failed. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 8s | The patch has no ill-formed XML file. |
| -1 :x: | shadedclient | 0m 35s | patch has errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 12s | hadoop-common in the patch failed. |
| -1 :x: | javadoc | 0m 35s | hadoop-hdfs-project_hadoop-hdfs generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) |
| -1 :x: | javadoc | 0m 27s | hadoop-hdfs-project_hadoop-hdfs-client generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) |
| +0 :ok: | findbugs | 0m 13s | hadoop-project has no data from findbugs |
| +0 :ok: | findbugs | 0m 15s | hadoop-client-modules/hadoop-client-check-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 15s | hadoop-client-modules/hadoop-client-check-test-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 13s | hadoop-client-modules/hadoop-client-runtime has no data from findbugs |
| -1 :x: | findbugs | 0m 12s | hadoop-common in the patch failed. |
| -1 :x: | findbugs | 0m 34s | hadoop-hdfs in the patch failed. |
| -1 :x: | findbugs | 0m 33s | hadoop-hdfs-client in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 12s | hadoop-project in the patch passed. |
| +1 :green_heart: | unit | 0m 12s | hadoop-client-check-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 13s | hadoop-client-check-test-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-runtime in the patch passed. |
| -1 :x: | unit | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | unit | 0m 37s | hadoop-hdfs in the patch failed. |
| -1 :x: | unit | 0m 34s | hadoop-hdfs-client in the patch failed. |
| +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. |
| | | 96m 51s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1846 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle cc prototool |
| uname | Linux 9d294225260d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / d6d7f8d |
| Default Java | 1.8.0_242 |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-compile-root.txt |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-compile-root.txt |
| javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-compile-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out//home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1846/out/maven-patch-checkstyle-root.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-common.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-findbugs-hadoop-common-project_hadoop-common.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/testReport/ |
| Max. process+thread count | 413 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-client-modules/hadoop-client-check-invariants hadoop-client-modules/hadoop-client-check-test-invariants hadoop-client-modules/hadoop-client-runtime hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/3/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r380999997
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/datatransfer.proto
##########
@@ -223,6 +225,7 @@ message ShortCircuitShmRequestProto {
// purely for logging / debugging purposes.
required string clientName = 1;
optional DataTransferTraceInfoProto traceInfo = 2;
+ optional bytes spanContext = 3;
Review comment:
The OT span context should be added into DataTransferTraceInfoProto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r382856741
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsTracer.java
##########
@@ -49,6 +49,7 @@ public static synchronized Tracer get(Configuration conf) {
@VisibleForTesting
public static synchronized void clear() {
Review comment:
If this method is no longer needed, let's remove it entirely.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r387319459
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/Span.java
##########
@@ -0,0 +1,54 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.tracing;
+
+import io.opentracing.SpanContext;
+import java.io.Closeable;
+
+public class Span implements Closeable {
Review comment:
Thanks Adam for the comment.
Yeah we would definitely do that if we are going to keep the shim. But now we are going to remove it, so the class won't exist anymore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on issue #1846: HADOOP-15566. Support
Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-597420552
> I created a branch "HADOOP-15566-OpenTracing" please raise a PR against that branch in the future. Thanks!
I've updated this PR's base branch to `HADOOP-15566-OpenTracing`. No need to close and raise a new PR :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] adamantal commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
adamantal commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r383182280
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/Span.java
##########
@@ -0,0 +1,54 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.tracing;
+
+import io.opentracing.SpanContext;
+import java.io.Closeable;
+
+public class Span implements Closeable {
Review comment:
Could you please add javadoc that describes the usage of this class?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r381030814
##########
File path: hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto
##########
@@ -91,6 +91,7 @@ message RpcRequestHeaderProto { // the header for the RpcRequest
optional RPCTraceInfoProto traceInfo = 6; // tracing info
optional RPCCallerContextProto callerContext = 7; // call context
optional int64 stateId = 8; // The last seen Global State ID
+ optional bytes spanContext = 9; // OpenTracing SpanContext
Review comment:
Done in b8c63f3f5d65e4d94caef98587d468bdbab78e5d.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r381000344
##########
File path: hadoop-project/pom.xml
##########
@@ -1196,6 +1196,16 @@
<artifactId>jsch</artifactId>
<version>0.1.54</version>
</dependency>
+ <dependency>
+ <groupId>io.opentracing</groupId>
+ <artifactId>opentracing-util</artifactId>
+ <version>0.31.0</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.hadoop.thirdparty</groupId>
+ <artifactId>hadoop-shaded-jaeger</artifactId>
+ <version>1.0.0-SNAPSHOT</version>
Review comment:
use ${hadoop-thirdparty-protobuf.version}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on issue #1846: HADOOP-15566. Support
Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-597360719
I created a branch "HADOOP-15566-OpenTracing" please raise a PR against that branch in the future. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r382861120
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/TraceUtils.java
##########
@@ -72,4 +97,71 @@ private String getInternal(String key) {
}
};
}
+
+ public static Tracer createAndRegisterTracer(String name) {
+ if (!GlobalTracer.isRegistered()) {
+ io.jaegertracing.Configuration config =
+ io.jaegertracing.Configuration.fromEnv(name);
+ Tracer tracer = config.getTracerBuilder().build();
+ GlobalTracer.register(tracer);
+ }
+
+ return GlobalTracer.get();
+ }
+
+ public static SpanContext byteStringToSpanContext(ByteString byteString) {
+ if (byteString == null || byteString.isEmpty()) {
+ LOG.debug("The provided serialized context was null or empty");
+ return null;
+ }
+
+ SpanContext context = null;
+ ByteArrayInputStream stream =
+ new ByteArrayInputStream(byteString.toByteArray());
+
+ try {
+ ObjectInputStream objStream = new ObjectInputStream(stream);
+ Map<String, String> carrier =
+ (Map<String, String>) objStream.readObject();
+
+ context = GlobalTracer.get().extract(Format.Builtin.TEXT_MAP,
+ new TextMapExtractAdapter(carrier));
+ } catch (Exception e) {
+ LOG.warn("Could not deserialize context {}", e);
+ }
+
+ return context;
+ }
+
+ public static ByteString spanContextToByteString(SpanContext context) {
+ if (context == null) {
+ LOG.debug("No SpanContext was provided");
+ return null;
+ }
+
+ Map<String, String> carrier = new HashMap<String, String>();
+ GlobalTracer.get().inject(context, Format.Builtin.TEXT_MAP,
+ new TextMapInjectAdapter(carrier));
+ if (carrier.isEmpty()) {
+ LOG.warn("SpanContext was not properly injected by the Tracer.");
+ return null;
+ }
+
+ ByteString byteString = null;
+ ByteArrayOutputStream stream = new ByteArrayOutputStream();
+
+ try {
+ ObjectOutputStream objStream = new ObjectOutputStream(stream);
+ objStream.writeObject(carrier);
+ objStream.flush();
+
+ byteString = ByteString.copyFrom(stream.toByteArray());
+ LOG.debug("SpanContext serialized, resulting byte length is {}",
+ byteString.size());
+ } catch (IOException e) {
+ LOG.warn("Could not serialize context {}", e);
Review comment:
missing a parameter here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r380999437
##########
File path: hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto
##########
@@ -91,6 +91,7 @@ message RpcRequestHeaderProto { // the header for the RpcRequest
optional RPCTraceInfoProto traceInfo = 6; // tracing info
optional RPCCallerContextProto callerContext = 7; // call context
optional int64 stateId = 8; // The last seen Global State ID
+ optional bytes spanContext = 9; // OpenTracing SpanContext
Review comment:
we should add the OT span context in the RPCTraceInfoProto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r382865557
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
##########
@@ -698,11 +698,14 @@ public void run() {
LOG.debug("Thread interrupted", e);
}
one = dataQueue.getFirst(); // regular data packet
- SpanId[] parents = one.getTraceParents();
+ SpanContext[] parents = one.getTraceParents();
if (parents.length > 0) {
+ // The original code stored multiple parents in the DFSPacket, and
+ // use them ALL here when creating a new Span. We only use the
+ // last one FOR NOW. Moreover, we don't activate the Span for now.
scope = dfsClient.getTracer().
- newScope("dataStreamer", parents[0]);
- scope.getSpan().setParents(parents);
+ newScope("dataStreamer", parents[0], false);
+ //scope.getSpan().setParents(parents);
Review comment:
i need to check if the concept matches but io.opentracing.Tracer offers a addReference() API to represent the casual relationship.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r382858893
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/TraceUtils.java
##########
@@ -72,4 +97,71 @@ private String getInternal(String key) {
}
};
}
+
+ public static Tracer createAndRegisterTracer(String name) {
+ if (!GlobalTracer.isRegistered()) {
+ io.jaegertracing.Configuration config =
+ io.jaegertracing.Configuration.fromEnv(name);
+ Tracer tracer = config.getTracerBuilder().build();
+ GlobalTracer.register(tracer);
+ }
+
+ return GlobalTracer.get();
+ }
+
+ public static SpanContext byteStringToSpanContext(ByteString byteString) {
+ if (byteString == null || byteString.isEmpty()) {
+ LOG.debug("The provided serialized context was null or empty");
+ return null;
+ }
+
+ SpanContext context = null;
+ ByteArrayInputStream stream =
+ new ByteArrayInputStream(byteString.toByteArray());
+
+ try {
+ ObjectInputStream objStream = new ObjectInputStream(stream);
+ Map<String, String> carrier =
+ (Map<String, String>) objStream.readObject();
+
+ context = GlobalTracer.get().extract(Format.Builtin.TEXT_MAP,
+ new TextMapExtractAdapter(carrier));
+ } catch (Exception e) {
+ LOG.warn("Could not deserialize context {}", e);
Review comment:
did you forget to log variable byteString?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1846: HADOOP-15566.
Support Opentracing
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-587478993
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 33s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 9s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 18m 52s | trunk passed |
| +1 :green_heart: | compile | 17m 4s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 11s | trunk passed |
| +1 :green_heart: | mvnsite | 6m 18s | trunk passed |
| +1 :green_heart: | shadedclient | 24m 54s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 5m 0s | trunk passed |
| +0 :ok: | spotbugs | 0m 33s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +0 :ok: | findbugs | 0m 32s | branch/hadoop-project no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 34s | branch/hadoop-client-modules/hadoop-client-runtime no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 34s | branch/hadoop-client-modules/hadoop-client-check-invariants no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 33s | branch/hadoop-client-modules/hadoop-client-check-test-invariants no findbugs output file (findbugsXml.xml) |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 12s | hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 36s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvninstall | 0m 33s | hadoop-hdfs-client in the patch failed. |
| -1 :x: | compile | 0m 36s | root in the patch failed. |
| -1 :x: | cc | 0m 36s | root in the patch failed. |
| -1 :x: | javac | 0m 36s | root in the patch failed. |
| -0 :warning: | checkstyle | 0m 20s | The patch fails to run checkstyle in root |
| -1 :x: | mvnsite | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | mvnsite | 0m 35s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvnsite | 0m 35s | hadoop-hdfs-client in the patch failed. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 8s | The patch has no ill-formed XML file. |
| -1 :x: | shadedclient | 0m 35s | patch has errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | javadoc | 0m 39s | hadoop-hdfs-project_hadoop-hdfs generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) |
| -1 :x: | javadoc | 0m 27s | hadoop-hdfs-project_hadoop-hdfs-client generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) |
| +0 :ok: | findbugs | 0m 13s | hadoop-project has no data from findbugs |
| +0 :ok: | findbugs | 0m 15s | hadoop-client-modules/hadoop-client-check-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 15s | hadoop-client-modules/hadoop-client-check-test-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 14s | hadoop-client-modules/hadoop-client-runtime has no data from findbugs |
| -1 :x: | findbugs | 0m 14s | hadoop-common in the patch failed. |
| -1 :x: | findbugs | 0m 36s | hadoop-hdfs in the patch failed. |
| -1 :x: | findbugs | 0m 34s | hadoop-hdfs-client in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 13s | hadoop-project in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-check-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-check-test-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-runtime in the patch passed. |
| -1 :x: | unit | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | unit | 0m 37s | hadoop-hdfs in the patch failed. |
| -1 :x: | unit | 0m 34s | hadoop-hdfs-client in the patch failed. |
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| | | 96m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1846 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle cc prototool |
| uname | Linux 4d502e21cc19 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / a562942 |
| Default Java | 1.8.0_242 |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-compile-root.txt |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-compile-root.txt |
| javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-compile-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out//home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1846/out/maven-patch-checkstyle-root.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-common.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-findbugs-hadoop-common-project_hadoop-common.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/testReport/ |
| Max. process+thread count | 412 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-client-modules/hadoop-client-check-invariants hadoop-client-modules/hadoop-client-check-test-invariants hadoop-client-modules/hadoop-client-runtime hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/2/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r381016157
##########
File path: hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto
##########
@@ -91,6 +91,7 @@ message RpcRequestHeaderProto { // the header for the RpcRequest
optional RPCTraceInfoProto traceInfo = 6; // tracing info
optional RPCCallerContextProto callerContext = 7; // call context
optional int64 stateId = 8; // The last seen Global State ID
+ optional bytes spanContext = 9; // OpenTracing SpanContext
Review comment:
I'm moving it to `RPCTraceInfoProto` instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r381016157
##########
File path: hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto
##########
@@ -91,6 +91,7 @@ message RpcRequestHeaderProto { // the header for the RpcRequest
optional RPCTraceInfoProto traceInfo = 6; // tracing info
optional RPCCallerContextProto callerContext = 7; // call context
optional int64 stateId = 8; // The last seen Global State ID
+ optional bytes spanContext = 9; // OpenTracing SpanContext
Review comment:
Moved to `RPCTraceInfoProto`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1846: HADOOP-15566.
Support Opentracing
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-603615342
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 33s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 1s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ HADOOP-15566-OpenTracing Compile Tests _ |
| +0 :ok: | mvndep | 1m 7s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 26s | HADOOP-15566-OpenTracing passed |
| +1 :green_heart: | compile | 27m 11s | HADOOP-15566-OpenTracing passed |
| +1 :green_heart: | checkstyle | 4m 31s | HADOOP-15566-OpenTracing passed |
| +1 :green_heart: | mvnsite | 8m 29s | HADOOP-15566-OpenTracing passed |
| +1 :green_heart: | shadedclient | 34m 7s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 5m 3s | HADOOP-15566-OpenTracing passed |
| +0 :ok: | spotbugs | 0m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +0 :ok: | findbugs | 0m 33s | branch/hadoop-project no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 34s | branch/hadoop-client-modules/hadoop-client-runtime no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 34s | branch/hadoop-client-modules/hadoop-client-check-invariants no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 35s | branch/hadoop-client-modules/hadoop-client-check-test-invariants no findbugs output file (findbugsXml.xml) |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 12s | hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 35s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvninstall | 0m 32s | hadoop-hdfs-client in the patch failed. |
| -1 :x: | compile | 0m 35s | root in the patch failed. |
| -1 :x: | cc | 0m 35s | root in the patch failed. |
| -1 :x: | javac | 0m 35s | root in the patch failed. |
| -0 :warning: | checkstyle | 0m 19s | The patch fails to run checkstyle in root |
| -1 :x: | mvnsite | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | mvnsite | 0m 37s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvnsite | 0m 34s | hadoop-hdfs-client in the patch failed. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 8s | The patch has no ill-formed XML file. |
| -1 :x: | shadedclient | 0m 34s | patch has errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | javadoc | 0m 39s | hadoop-hdfs-project_hadoop-hdfs generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) |
| -1 :x: | javadoc | 0m 26s | hadoop-hdfs-project_hadoop-hdfs-client generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) |
| +0 :ok: | findbugs | 0m 15s | hadoop-project has no data from findbugs |
| +0 :ok: | findbugs | 0m 14s | hadoop-client-modules/hadoop-client-check-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 15s | hadoop-client-modules/hadoop-client-check-test-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 15s | hadoop-client-modules/hadoop-client-runtime has no data from findbugs |
| -1 :x: | findbugs | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | findbugs | 0m 36s | hadoop-hdfs in the patch failed. |
| -1 :x: | findbugs | 0m 33s | hadoop-hdfs-client in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 13s | hadoop-project in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-check-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-check-test-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-runtime in the patch passed. |
| -1 :x: | unit | 0m 14s | hadoop-common in the patch failed. |
| -1 :x: | unit | 0m 36s | hadoop-hdfs in the patch failed. |
| -1 :x: | unit | 0m 33s | hadoop-hdfs-client in the patch failed. |
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| | | 120m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1846 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle cc prototool |
| uname | Linux 9e56d618c119 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | HADOOP-15566-OpenTracing / cf9cf83 |
| Default Java | 1.8.0_242 |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-compile-root.txt |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-compile-root.txt |
| javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-compile-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out//home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1846/out/maven-patch-checkstyle-root.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-common.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-findbugs-hadoop-common-project_hadoop-common.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/testReport/ |
| Max. process+thread count | 413 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-client-modules/hadoop-client-check-invariants hadoop-client-modules/hadoop-client-check-test-invariants hadoop-client-modules/hadoop-client-runtime hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/6/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r387400362
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/TraceUtils.java
##########
@@ -72,4 +97,71 @@ private String getInternal(String key) {
}
};
}
+
+ public static Tracer createAndRegisterTracer(String name) {
+ if (!GlobalTracer.isRegistered()) {
+ io.jaegertracing.Configuration config =
+ io.jaegertracing.Configuration.fromEnv(name);
+ Tracer tracer = config.getTracerBuilder().build();
+ GlobalTracer.register(tracer);
+ }
+
+ return GlobalTracer.get();
+ }
+
+ public static SpanContext byteStringToSpanContext(ByteString byteString) {
+ if (byteString == null || byteString.isEmpty()) {
+ LOG.debug("The provided serialized context was null or empty");
+ return null;
+ }
+
+ SpanContext context = null;
+ ByteArrayInputStream stream =
+ new ByteArrayInputStream(byteString.toByteArray());
+
+ try {
+ ObjectInputStream objStream = new ObjectInputStream(stream);
+ Map<String, String> carrier =
+ (Map<String, String>) objStream.readObject();
+
+ context = GlobalTracer.get().extract(Format.Builtin.TEXT_MAP,
+ new TextMapExtractAdapter(carrier));
+ } catch (Exception e) {
+ LOG.warn("Could not deserialize context {}", e);
+ }
+
+ return context;
+ }
+
+ public static ByteString spanContextToByteString(SpanContext context) {
+ if (context == null) {
+ LOG.debug("No SpanContext was provided");
+ return null;
+ }
+
+ Map<String, String> carrier = new HashMap<String, String>();
+ GlobalTracer.get().inject(context, Format.Builtin.TEXT_MAP,
+ new TextMapInjectAdapter(carrier));
+ if (carrier.isEmpty()) {
+ LOG.warn("SpanContext was not properly injected by the Tracer.");
+ return null;
+ }
+
+ ByteString byteString = null;
+ ByteArrayOutputStream stream = new ByteArrayOutputStream();
+
+ try {
+ ObjectOutputStream objStream = new ObjectOutputStream(stream);
Review comment:
Moreover, ObjectInputStream doesn't guarantee compatibility between a server and a client if they are on different version. Finally, I bet it is not optimal serialization format. We should find one that's as short as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r381030917
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/datatransfer.proto
##########
@@ -223,6 +225,7 @@ message ShortCircuitShmRequestProto {
// purely for logging / debugging purposes.
required string clientName = 1;
optional DataTransferTraceInfoProto traceInfo = 2;
+ optional bytes spanContext = 3;
Review comment:
Done in b8c63f3f5d65e4d94caef98587d468bdbab78e5d (same commit as above protobuf change).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1846: HADOOP-15566.
Support Opentracing
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-588020487
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 24s | trunk passed |
| +1 :green_heart: | compile | 18m 1s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 6s | trunk passed |
| +1 :green_heart: | mvnsite | 6m 1s | trunk passed |
| +1 :green_heart: | shadedclient | 24m 47s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 33s | hadoop-client-check-invariants in trunk failed. |
| -1 :x: | javadoc | 0m 33s | hadoop-client-check-test-invariants in trunk failed. |
| +0 :ok: | spotbugs | 0m 32s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +0 :ok: | findbugs | 0m 33s | branch/hadoop-project no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 32s | branch/hadoop-client-modules/hadoop-client-runtime no findbugs output file (findbugsXml.xml) |
| -1 :x: | findbugs | 0m 31s | hadoop-client-check-invariants in trunk failed. |
| -1 :x: | findbugs | 0m 32s | hadoop-client-check-test-invariants in trunk failed. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 12s | hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 37s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvninstall | 0m 34s | hadoop-hdfs-client in the patch failed. |
| -1 :x: | compile | 0m 36s | root in the patch failed. |
| -1 :x: | cc | 0m 36s | root in the patch failed. |
| -1 :x: | javac | 0m 36s | root in the patch failed. |
| -0 :warning: | checkstyle | 0m 19s | The patch fails to run checkstyle in root |
| -1 :x: | mvnsite | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | mvnsite | 0m 37s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvnsite | 0m 35s | hadoop-hdfs-client in the patch failed. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 9s | The patch has no ill-formed XML file. |
| -1 :x: | shadedclient | 0m 35s | patch has errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 13s | hadoop-common in the patch failed. |
| -1 :x: | javadoc | 0m 39s | hadoop-hdfs-project_hadoop-hdfs generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) |
| -1 :x: | javadoc | 0m 27s | hadoop-hdfs-project_hadoop-hdfs-client generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) |
| +0 :ok: | findbugs | 0m 14s | hadoop-project has no data from findbugs |
| +0 :ok: | findbugs | 0m 15s | hadoop-client-modules/hadoop-client-check-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 15s | hadoop-client-modules/hadoop-client-check-test-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 14s | hadoop-client-modules/hadoop-client-runtime has no data from findbugs |
| -1 :x: | findbugs | 0m 14s | hadoop-common in the patch failed. |
| -1 :x: | findbugs | 0m 35s | hadoop-hdfs in the patch failed. |
| -1 :x: | findbugs | 0m 34s | hadoop-hdfs-client in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 13s | hadoop-project in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-check-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-check-test-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 14s | hadoop-client-runtime in the patch passed. |
| -1 :x: | unit | 0m 14s | hadoop-common in the patch failed. |
| -1 :x: | unit | 0m 36s | hadoop-hdfs in the patch failed. |
| -1 :x: | unit | 0m 34s | hadoop-hdfs-client in the patch failed. |
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| | | 98m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1846 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle cc prototool |
| uname | Linux 2d7209801802 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / d6d7f8d |
| Default Java | 1.8.0_242 |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/branch-javadoc-hadoop-client-modules_hadoop-client-check-invariants.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/branch-javadoc-hadoop-client-modules_hadoop-client-check-test-invariants.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/branch-findbugs-hadoop-client-modules_hadoop-client-check-invariants.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/branch-findbugs-hadoop-client-modules_hadoop-client-check-test-invariants.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-compile-root.txt |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-compile-root.txt |
| javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-compile-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out//home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1846@2/out/maven-patch-checkstyle-root.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-common.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-findbugs-hadoop-common-project_hadoop-common.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/testReport/ |
| Max. process+thread count | 453 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-client-modules/hadoop-client-check-invariants hadoop-client-modules/hadoop-client-check-test-invariants hadoop-client-modules/hadoop-client-runtime hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/4/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1846: HADOOP-15566.
Support Opentracing
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-585669612
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 10s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 18m 56s | trunk passed |
| +1 :green_heart: | compile | 17m 3s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 11s | trunk passed |
| +1 :green_heart: | mvnsite | 6m 21s | trunk passed |
| +1 :green_heart: | shadedclient | 25m 2s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 5m 6s | trunk passed |
| +0 :ok: | spotbugs | 0m 36s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +0 :ok: | findbugs | 0m 34s | branch/hadoop-project no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 35s | branch/hadoop-client-modules/hadoop-client-runtime no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 35s | branch/hadoop-client-modules/hadoop-client-check-invariants no findbugs output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 35s | branch/hadoop-client-modules/hadoop-client-check-test-invariants no findbugs output file (findbugsXml.xml) |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 48s | hadoop-hdfs-client in the patch failed. |
| -1 :x: | mvninstall | 0m 41s | hadoop-hdfs in the patch failed. |
| -1 :x: | mvninstall | 2m 33s | hadoop-client-runtime in the patch failed. |
| +1 :green_heart: | compile | 16m 23s | the patch passed |
| -1 :x: | cc | 16m 23s | root generated 3 new + 23 unchanged - 3 fixed = 26 total (was 26) |
| +1 :green_heart: | javac | 16m 23s | the patch passed |
| -0 :warning: | checkstyle | 3m 12s | root: The patch generated 35 new + 1322 unchanged - 8 fixed = 1357 total (was 1330) |
| -1 :x: | mvnsite | 0m 59s | hadoop-hdfs in the patch failed. |
| +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 8s | The patch has no ill-formed XML file. |
| -1 :x: | shadedclient | 4m 22s | patch has errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 5m 0s | the patch passed |
| +0 :ok: | findbugs | 0m 33s | hadoop-project has no data from findbugs |
| -1 :x: | findbugs | 0m 59s | hadoop-hdfs in the patch failed. |
| +0 :ok: | findbugs | 0m 34s | hadoop-client-modules/hadoop-client-runtime has no data from findbugs |
| +0 :ok: | findbugs | 0m 34s | hadoop-client-modules/hadoop-client-check-invariants has no data from findbugs |
| +0 :ok: | findbugs | 0m 35s | hadoop-client-modules/hadoop-client-check-test-invariants has no data from findbugs |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 34s | hadoop-project in the patch passed. |
| -1 :x: | unit | 9m 22s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 23s | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 1m 0s | hadoop-hdfs in the patch failed. |
| +1 :green_heart: | unit | 0m 34s | hadoop-client-runtime in the patch passed. |
| +1 :green_heart: | unit | 0m 33s | hadoop-client-check-invariants in the patch passed. |
| +1 :green_heart: | unit | 0m 34s | hadoop-client-check-test-invariants in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | The patch does not generate ASF License warnings. |
| | | 148m 7s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.fs.TestFsShell |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1846 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle cc prototool |
| uname | Linux 1a9a81a27930 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / fe7d67a |
| Default Java | 1.8.0_242 |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt |
| mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/patch-mvninstall-hadoop-client-modules_hadoop-client-runtime.txt |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/diff-compile-cc-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/diff-checkstyle-root.txt |
| mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/testReport/ |
| Max. process+thread count | 1399 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-client-modules/hadoop-client-runtime hadoop-client-modules/hadoop-client-check-invariants hadoop-client-modules/hadoop-client-check-test-invariants U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1846/1/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r381033615
##########
File path: hadoop-project/pom.xml
##########
@@ -1196,6 +1196,16 @@
<artifactId>jsch</artifactId>
<version>0.1.54</version>
</dependency>
+ <dependency>
+ <groupId>io.opentracing</groupId>
+ <artifactId>opentracing-util</artifactId>
+ <version>0.31.0</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.hadoop.thirdparty</groupId>
+ <artifactId>hadoop-shaded-jaeger</artifactId>
+ <version>1.0.0-SNAPSHOT</version>
Review comment:
Done in 22cf60451b2910f6f00d9274e10a68c7042d3e8d
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r387383409
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/TraceUtils.java
##########
@@ -72,4 +97,71 @@ private String getInternal(String key) {
}
};
}
+
+ public static Tracer createAndRegisterTracer(String name) {
+ if (!GlobalTracer.isRegistered()) {
+ io.jaegertracing.Configuration config =
+ io.jaegertracing.Configuration.fromEnv(name);
+ Tracer tracer = config.getTracerBuilder().build();
+ GlobalTracer.register(tracer);
+ }
+
+ return GlobalTracer.get();
+ }
+
+ public static SpanContext byteStringToSpanContext(ByteString byteString) {
+ if (byteString == null || byteString.isEmpty()) {
+ LOG.debug("The provided serialized context was null or empty");
+ return null;
+ }
+
+ SpanContext context = null;
+ ByteArrayInputStream stream =
+ new ByteArrayInputStream(byteString.toByteArray());
+
+ try {
+ ObjectInputStream objStream = new ObjectInputStream(stream);
+ Map<String, String> carrier =
+ (Map<String, String>) objStream.readObject();
+
+ context = GlobalTracer.get().extract(Format.Builtin.TEXT_MAP,
+ new TextMapExtractAdapter(carrier));
+ } catch (Exception e) {
+ LOG.warn("Could not deserialize context {}", e);
+ }
+
+ return context;
+ }
+
+ public static ByteString spanContextToByteString(SpanContext context) {
+ if (context == null) {
+ LOG.debug("No SpanContext was provided");
+ return null;
+ }
+
+ Map<String, String> carrier = new HashMap<String, String>();
+ GlobalTracer.get().inject(context, Format.Builtin.TEXT_MAP,
+ new TextMapInjectAdapter(carrier));
+ if (carrier.isEmpty()) {
+ LOG.warn("SpanContext was not properly injected by the Tracer.");
+ return null;
+ }
+
+ ByteString byteString = null;
+ ByteArrayOutputStream stream = new ByteArrayOutputStream();
+
+ try {
+ ObjectOutputStream objStream = new ObjectOutputStream(stream);
Review comment:
This is dangerous
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/ObjectInputStream.html
"Warning: Deserialization of untrusted data is inherently dangerous and should be avoided."
We need an alternative.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] adamantal commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
adamantal commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r383182796
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/Tracer.java
##########
@@ -0,0 +1,112 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.tracing;
+
+import io.opentracing.Scope;
+import io.opentracing.SpanContext;
+import io.opentracing.util.GlobalTracer;
+
+public class Tracer {
Review comment:
Could you please add javadoc here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] adamantal commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
adamantal commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r383182524
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/TraceScope.java
##########
@@ -0,0 +1,66 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.tracing;
+
+import java.io.Closeable;
+
+public class TraceScope implements Closeable {
Review comment:
Could you please add javadoc here as well?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on issue #1846: HADOOP-15566. Support
Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on issue #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#issuecomment-596784886
Rebased onto latest trunk (44afe1154dd8ce937470c04a126310989f3dc2cb): Resolved a minor import conflict in NameNode class due to the introduction of `org.apache.hadoop.util.GcTimeMonitor`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on a change in pull request #1846:
HADOOP-15566. Support Opentracing
Posted by GitBox <gi...@apache.org>.
smengcl commented on a change in pull request #1846: HADOOP-15566. Support Opentracing
URL: https://github.com/apache/hadoop/pull/1846#discussion_r389960788
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/TraceUtils.java
##########
@@ -72,4 +97,71 @@ private String getInternal(String key) {
}
};
}
+
+ public static Tracer createAndRegisterTracer(String name) {
+ if (!GlobalTracer.isRegistered()) {
+ io.jaegertracing.Configuration config =
+ io.jaegertracing.Configuration.fromEnv(name);
+ Tracer tracer = config.getTracerBuilder().build();
+ GlobalTracer.register(tracer);
+ }
+
+ return GlobalTracer.get();
+ }
+
+ public static SpanContext byteStringToSpanContext(ByteString byteString) {
+ if (byteString == null || byteString.isEmpty()) {
+ LOG.debug("The provided serialized context was null or empty");
+ return null;
+ }
+
+ SpanContext context = null;
+ ByteArrayInputStream stream =
+ new ByteArrayInputStream(byteString.toByteArray());
+
+ try {
+ ObjectInputStream objStream = new ObjectInputStream(stream);
+ Map<String, String> carrier =
+ (Map<String, String>) objStream.readObject();
+
+ context = GlobalTracer.get().extract(Format.Builtin.TEXT_MAP,
+ new TextMapExtractAdapter(carrier));
+ } catch (Exception e) {
+ LOG.warn("Could not deserialize context {}", e);
+ }
+
+ return context;
+ }
+
+ public static ByteString spanContextToByteString(SpanContext context) {
+ if (context == null) {
+ LOG.debug("No SpanContext was provided");
+ return null;
+ }
+
+ Map<String, String> carrier = new HashMap<String, String>();
+ GlobalTracer.get().inject(context, Format.Builtin.TEXT_MAP,
+ new TextMapInjectAdapter(carrier));
+ if (carrier.isEmpty()) {
+ LOG.warn("SpanContext was not properly injected by the Tracer.");
+ return null;
+ }
+
+ ByteString byteString = null;
+ ByteArrayOutputStream stream = new ByteArrayOutputStream();
+
+ try {
+ ObjectOutputStream objStream = new ObjectOutputStream(stream);
Review comment:
Good find @jojochuang . I agree we should definitely use a more optimized solution for serialization/deserialization if possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org