[I] [enhancement] Optimization of Metric Calculation in trace-etl-server​ [ozhera]

["sadadw1 (via GitHub)" <gi...@apache.org>]

sadadw1 opened a new issue, #586:
URL: https://github.com/apache/ozhera/issues/586

   Recently, we have further optimized the metric calculation in ​​trace-etl-server​​. The background for this optimization stems from our observation that in today's microservices architecture, there are often applications with extremely high call volumes and numerous upstream/downstream dependencies—we refer to these as **​​"head applications"​​**.
   
   Due to their high traffic, instances of these head applications typically rank among the ​​top consumers​​ in the entire cluster. Additionally, because of their extensive dependencies, the ​​dispersion of metric labels​​ (e.g., Http or Dubbo methodName) can be ​​orders of magnitude higher​​ compared to other applications.
   
   These two factors lead to ​​trace-etl-server generating an enormous volume of metrics​​ when processing head applications. This excessive metric load can cause:
   
   ​​**Memory exhaustion**​​ in trace-etl-server itself.
   ​​**Significant delays**​​ when Prometheus scrapes these metrics.
   **​​High pressure**​​ on Prometheus during metric aggregation.
   
   To address these issues, we have implemented the following improvements:
   
   ​​1. Configurable metric merging per application.​​
   2. ​​A recommendation to isolate head applications​​ by consuming their traces in dedicated ​​trace-etl-server instances​​ in production environments.
   
   This optimization ensures better stability and performance for both ​​trace-etl-server​​ and ​​Prometheus​​ when handling high-traffic services.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@ozhera.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozhera.apache.org
For additional commands, e-mail: commits-help@ozhera.apache.org

Re: [I] [enhancement] Optimization of Metric Calculation in trace-etl-server​ [ozhera]

["wodiwudi (via GitHub)" <gi...@apache.org>]

wodiwudi commented on issue #586:
URL: https://github.com/apache/ozhera/issues/586#issuecomment-2987363506

   This optimization is very good. I hope to provide more specific indicators to measure the improvement. For example, how much time series data will be reduced in reducing the pressure on Prometheus?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@ozhera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozhera.apache.org
For additional commands, e-mail: commits-help@ozhera.apache.org