[GitHub] [rocketmq] dugenkui03 commented on pull request #4099: [ISSUE #4098]Optimized the algorithm of trace message send.

[GitBox <gi...@apache.org>]

dugenkui03 commented on PR #4099:
URL: https://github.com/apache/rocketmq/pull/4099#issuecomment-1099351837

   > @dugenkui03 I found another issue. In the AsyncAppenderRequest, the max message size is 128000. So If the byte size of the accumulated TraceContext is more than 128000, it cannot reduce the RPC to improve the performance, but cause delay.
   > 
   > So the algorithm may need to consider the byte size too. If the byte size is enough, it should be scheduled immediately.
   > 
   > BTW, the data topic generated by "sendTraceData" is not used in "sendTraceDataByMQ".
   > 
   > Maybe we need to clarify the design of topic mechanism for trace.
   
   @dongeforever Thanks for you suggestion. I trace the code and find we need do more to speed up  performance( such as reduce the total number of send trace message).  And the only benefit of this pr is that make the timing of submiting AsyncAppenderRequest more clearly, and it's is hard to say it's really improve the perfromance.
   
   # Describe in Chinese
   感谢你的建议和提醒，按照你的提示我跟踪代码后发现提交的任务`AsyncAppenderRequest`并非一次发送给服务端、还会按照topic+regionId分组且每组数据按照maxMsgSize分批发送。
   
   ##  当前逻辑
   
   <img width="1156" alt="image" src="https://user-images.githubusercontent.com/18216266/163427423-99836b88-8ada-46a1-8981-bd90fbe14d2e.png">
   
   1. 业务线程调用`AsyncTraceDispatcher#append`向任务元数据队列`traceContextQueue`添加`TraceContext`；
   2.  `worker`线程循环从`traceContextQueue`中获取`TraceContext`、并包装为任务`AsyncAppenderRequest`、提交给`#traceExecutor`；
   3. `AsyncAppenderRequest#sendTraceData`中将`TraceContext`转换为可计量大小的`TraceTransferBean`，并按照按照**topic+regionId**对`TraceTransferBean`进行分组、获取结构为`Map<String, List<TraceTransferBean>>`的数据；
   4. 将`TraceTransferBean`按照`maxMsgSize`进行分批次的发送，目标队列根据`AsyncTraceDispatcher#accessChannel`决定发送到`rmq_sys_TRACE_DATA_${regionId}`或者轨迹消息队列(默认为`RMQ_SYS_TRACE_TOPIC`) 。
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org