You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Matt McCline (Jira)" <ji...@apache.org> on 2021/05/10 16:28:00 UTC
[jira] [Comment Edited] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17341993#comment-17341993 ]
Matt McCline edited comment on HIVE-25069 at 5/10/21, 4:27 PM:
---------------------------------------------------------------
[~abstractdog] Thank you for your comments. I am actively working on the design document and preparing a first patch.
I have Thrift propagation almost done. Other areas in progress.
One general issue I have run into is version conflicts with libthrift (e.g. 0.9.3 vs 0.13.0). I am looking at doing *shade-renames* of dependent JAR used for tracing. Otherwise, I am very concerned about JAR conflicts and execution errors including Throwable MethodNotFoundException. Without *shade-renames* I think it will be harder to use the tracing in Tez, HWC, and other components which I want to address next after the first part is checked in. I too think Tez state-changes would be great to trace. LLAP, too. Note that Tez needs tracing propagation with its protobuf msgs (it is not Thrift). It has protobuf dependencies version problems (2 vs. 3), also.
Thrift Propagation:
!image-2021-05-10-09-20-54-688.png!
High level Tez related Span in Hive Server 2 so far:
!screenshot-1.png!
was (Author: mattmccline):
[~abstractdog] Thank you for your comments. I am actively working on the design document and preparing a first patch.
I have Thrift propagation almost done. Other areas in progress.
One general issue I have run into is version conflicts with libthrift (e.g. 0.9.3 vs 0.13.0). I am looking at doing *shade-renames* of dependent JAR used for tracing. Otherwise, I am very concerned about JAR conflicts and execution errors including Throwable MethodNotFoundException. Without *shade-renames* I think it will be harder to use the tracing in Tez, HWC, and other components which I want to address next after the first part is checked in. I too think Tez state-changes would be great to trace. LLAP, too. Note that Tez needs tracing propagation with its protobuf msgs (it is not Thrift). It has protobuf dependencies, also.
Thrift Propagation:
!image-2021-05-10-09-20-54-688.png!
High level Tez related Span in Hive Server 2 so far:
!screenshot-1.png!
> Hive Distributed Tracing
> ------------------------
>
> Key: HIVE-25069
> URL: https://issues.apache.org/jira/browse/HIVE-25069
> Project: Hive
> Issue Type: New Feature
> Reporter: Matt McCline
> Assignee: Matt McCline
> Priority: Major
> Attachments: image-2021-05-10-09-01-28-346.png, image-2021-05-10-09-10-42-811.png, image-2021-05-10-09-20-54-688.png, screenshot-1.png
>
>
> Instrument Hive code to gather distributed traces and export trace data to a configurable collector.
> Distributed tracing is a revolutionary tool for debugging issues.
> We will use the new OpenTelemetry open-source standard that our industry has aligned on. OpenTelemetry is the merger of two earlier distributed tracing projects OpenTracing and OpenCensus.
> Next step: Add design document that goes into more detail on the benefits of distributed tracing and describes how Hive will enhanced.
> Also see:
> HBASE-22120 Replace HTrace with OpenTelemetry
--
This message was sent by Atlassian Jira
(v8.3.4#803005)