[GitHub] [skywalking] hanahmily opened a new issue, #9930: [BanyanDB] TopNAggregation Verification Testing

[GitBox <gi...@apache.org>]

hanahmily opened a new issue, #9930:
URL: https://github.com/apache/skywalking/issues/9930

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   In the testing, we should prove that the TopNAggregation meets the requirements of our design. 
   
   ## Architecture
   
   [data-generate](https://github.com/apache/skywalking/pull/8725) -> OAP -> BanyanDB
   
   We should add a new [MetricGenerator](https://github.com/apache/skywalking/blob/master/oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/generator) to generate a series of hierarchies of "service-> endpoints" to stimulate a high volume of endpoints.
   
   ## Test Cases
   
   1. Top 10 query by `top` function
   1. Bottom 10 query by `top` function
   1. Top 10 query by `TopNAggregation` with `counters_number` 1000
   1. Bottom 10 query by `TopNAggregation`  with `counters_number` 1000
   
   The number of endpoints varies in `100`, `1K`, `10k`, and `1M`.
   
   Below metrics should be collected during the testing:
   
   * Latency of querying
   * CPU usage
   * Memory increment
   * Disk usage
   
   ## Integration with CI
   
   Run the test with `10k` endpoints daily through a GHA workflow to check the writing/querying of TopNAggregation.
   
   ## Location to hold the test
   
   Place relevant codes and scripts in the `test/aggregation` folder.
   
   
   ### Use case
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] lujiajing1126 closed issue #9930: [BanyanDB] TopNAggregation Verification Testing

["lujiajing1126 (via GitHub)" <gi...@apache.org>]

lujiajing1126 closed issue #9930: [BanyanDB] TopNAggregation Verification Testing 
URL: https://github.com/apache/skywalking/issues/9930


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] lujiajing1126 commented on issue #9930: [BanyanDB] TopNAggregation Verification Testing

["lujiajing1126 (via GitHub)" <gi...@apache.org>]

lujiajing1126 commented on issue #9930:
URL: https://github.com/apache/skywalking/issues/9930#issuecomment-1596133410

   In order to verify TopN Aggregation, we need
   - scripts to write data
   - scripts to query TopN in two approaches, i.e. full-scan and preaggregation 
   
   the infra can be found https://github.com/lujiajing1126/banyandb-billy. The detailed setup can be found there.
   
   The metric we are writing is basically, `temperature{sensor_id="<seq_id>"} <temperature>`. It means the measure schema only has a single tag, i.e. `sensor_id` which is a number within a sequence while the temperature (int) is randomly generated.
   
   The sequence range is used to control the actual cardinality of the data points. And we have the following benchmark report,
   
   | Cardinality | FullScan | PreAggregation(1k counter) | 
   | ----------- | -------- | --------------------------- |
   | 5K | 2.5 s | 1.0 s |
   | 10K | 3.6 | 1.0 |
   | 25K | 10 | 1.0 |
   | 50K | 19.9 | 1.0 |
   | 100K | 40 | 1.0 |
   
   Lower is better. (Tested on Mac Studio M1 Ultra with 64 GB memory)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org