You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2022/10/25 09:54:04 UTC

[GitHub] [skywalking] wu-sheng opened a new issue, #9844: [Feature] [BanyanDB storage] Support PreOrderedIndexing annotation

wu-sheng opened a new issue, #9844:
URL: https://github.com/apache/skywalking/issues/9844

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   Today, all TopN queries in BanyanDB are using on-demand aggregation and sorting. This is good from a function perspective, but it doesn't have good performance when we face a big data set. Such as query TopN for `endpoint_cpm`(Calls per minute for every endpoint/API), we would face 10k+ endpoint candidates in some services, which leads to slow queries.
   
   ### Use case
   
   With this new BanyanDB-specific annotation, the storage could build some specific pre-calculating indices with given conditions. Then in the query stage, we could run a 2nd-round query for the sorted result, which should have a smaller size and the cost would be predictable.
   
   We should only support this kind when the size of candidates is bigger than 1k for a time point.(For now, endpoint only).
   
   This change should include one annotation as well as OAL/MAL relative mechanisms. Because we only know the candidates of the metric owners(endpoints) would be huge when the metric is declared as an endpoint metric. This can't be done through an annotation on the function level.
   
   @lujiajing1126 Would you like to do this? Is there anything uncertain to discuss?
   
   FYI @hanahmily 
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #9844: [Feature] [BanyanDB storage] Support PreOrderedIndexing annotation

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #9844:
URL: https://github.com/apache/skywalking/issues/9844#issuecomment-1532682086

   @lujiajing1126 had added this through https://github.com/apache/skywalking/pull/10448


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng closed issue #9844: [Feature] [BanyanDB storage] Support PreOrderedIndexing annotation

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #9844: [Feature] [BanyanDB storage] Support PreOrderedIndexing annotation
URL: https://github.com/apache/skywalking/issues/9844


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org