[DISCUSS][RIP-18]RocketMQ OpenTracing Support

[liqipeng <wl...@163.com>]

Dear Apache RocketMQ Community:
 
We would like to propose a RIP-18 about RocketMQ OpenTracing Support.
 
Opentracing is a lightweight distributed tracking API standard. At present, distributed tracking systems such as Jaeger and Zipkin support the open tracking specification. Usually, the sending and consuming of messages is only part of the whole process of the application. There may also be RPC calls and database operations in the whole process. After supporting the open tracking specification, rocektmq users can better use distributed systems such as Jaeger to track and monitor the entire process including message processing.
 
Therefore, we want to provide OpenTracking support for Apache RocketMQ through RIP-17, so that the distributed tracking system can better monitor the message processing of RocketMQ .
 
​The following is the relevant information about this RIP and we are listed directly below for your convenience, hope everyone can give more suggestions. 
 
Thanks,
The Apache RocketMQ Team
 


Status
Current State: Proposed
Authors: wlliqipeng
Shepherds: wlliqipeng
Mailing List discussion: dev, users
Pull Request: none
Released: 4.6.0
Background & Motivation
Opentracing is a lightweight distributed tracking API standard. At present, distributed tracking systems such as Jaeger and Zipkin support the open tracking specification. Usually, the sending and consuming of messages is only part of the whole process of the application. There may also be RPC calls and database operations in the whole process. After supporting the open tracking specification, rocektmq users can better use distributed systems such as Jaeger to track and monitor the entire process including message processing.
Goals
What problem is this proposal designed to solve?
Distributed tracking system has become an important means of monitoring distributed systems, but if users use RocketMQ, the current distributed tracking system such as Jeager can not monitor the production and consumption of RocketMQ messages. At present, the RIP is to solve the compatibility problem of RocketMQ with OpenTracing, so that the distributed tracking system can monitor the message processing process of RocketMQ just like rpc.
 
To what degree should we solve the problem?
Through this RIP, OpenTracing-enabled distributed systems such as Jaeger can obtain the time-consuming of sending and consuming each message in RocketMQ, as well as related message information such as topic name, Offset, etc.
Non-Goals
What problem is this proposal NOT designed to solve?
This project will not implement the API interface of OpenTracing
Are there any limits to this proposal?
This rip only involves the modification of the Java client, not the client of other languages such as cpp, python, etc.
Changes
Architecture
Before or after the RocketMQ message is sent or consumed, RocketMQ Client implements the creation and information collection of Span by calling the OpenTracing interface. The following figure takes Jaeger as an example to illustrate the overall architecture of RocketMQ supporting OpenTracing.
  
 
 

 
 
Interface Design/Change
Plan to add Method as follow:
public DefaultMQProducer(final String producerGroup, Tracer tracer, Span span)；
public DefaultMQPushConsumer(final String consumerGroup, Tracer tracer, Span spa)；
 
Plan to add Class as follow:
public class SendMessageOpenTracingHookImpl implements SendMessageHook
public class ConsumeMessageOpenTracingHookImpl implements ConsumeMessageHook
 
 Compatibility, Deprecation, and Migration Plan
No issue
Implementation Outline
We will implement the proposed changes by 2 phase. 
Phase 1
Implement the Class of ConsumeMessageOpenTracingHookImpl and ConsumeMessageOpenTracingHookImpl.
 
Phase 2 
Implementing RocketMQ Tracking in Message Sending and Consumption