[GitHub] [incubator-doris] EmmyMiao87 commented on issue #5509: Doris 聚合模型bitmap聚合类型性能差的问题

[GitBox <gi...@apache.org>]

EmmyMiao87 commented on issue #5509:
URL: https://github.com/apache/incubator-doris/issues/5509#issuecomment-797417753


   The bitmap algorithm itself does not have very good computational performance when the cardinality is large and the distribution is loose. Performance tuning needs to be combined with modeling and real data distribution.
   1. Through the global dictionary, the value distribution of bitmap is changed from sparse to compact
   2. Observe your own data distribution to see if you can use the udaf orthogonal bitmap function
   How to use: http://doris.apache.org/master/zh-CN/extending-doris/udf/contrib/udaf-orthogonal-bitmap-manual.html
   
   By the way, I would like to recommend an article to you: A performance tuning case for small programs using bitmap to achieve precision marketing.
   https://blog.csdn.net/weixin_47452131/article/details/113393764
   
   bitmap算法本身在基数大且分布松散的情况下，计算性能不是很好。需要结合建模，真实数据分布进行性能调优。
   1. 通过全局字典的方式，使得bitmap的数值分布从稀疏改为紧凑
   2. 观察自己的数据分布，看是否能利用上udaf 正交bitmap函数
   使用方式： http://doris.apache.org/master/zh-CN/extending-doris/udf/contrib/udaf-orthogonal-bitmap-manual.html
   
   顺便给你推荐个文章：小程序使用 bitmap 实现精准营销的性能调优案例。
   https://blog.csdn.net/weixin_47452131/article/details/113393764


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org