You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/12/30 11:24:00 UTC
[GitHub] [incubator-doris] 924060929 opened a new issue #7546: [Enhancement] Support merge IN predicate & Adaptive runtime filter
924060929 opened a new issue #7546:
URL: https://github.com/apache/incubator-doris/issues/7546
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Description
## 问题
1. 当前的runtime filter在shuffle join时不支持合并IN predicate。
2. IN predicate存在一个比较奇怪的逻辑：如果遇到remote target则会隐式地把IN改成Bloom filter。这和用户设定的`runtime_filter_type='IN'`不一致，会导致用户迷惑。
3. IN predicate和其他filter通常不会一起用，因为IN predicate本身就是一个精确的过滤条件，已经足以把数据提前过滤好，如果加上其他filter，那么其他filter可能会做无用功。但现在没法在运行时自适应的决定使用runtime filter还是其他filter。
# 目标
1. 支持IN predicate的合并。
2. 去掉IN predicate隐式转变成Bloom filter的逻辑。
3. 支持一种自适应filter，在运行时选择应该使用哪种filter。
注：由于不容易判断什么情况适合使用MIN MAX filter，因此自适应filter仅在运行时判断使用IN predicate还是Bloom filter，判断依据是in过滤值数量少于runtime_filter_max_in_num时，尽量用IN predicate而不是Bloom filter，否则使用Bloom filter。
### Solution
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 closed issue #7546: [Enhancement] Support merge IN predicate & Adaptive runtime filter
Posted by GitBox <gi...@apache.org>.
EmmyMiao87 closed issue #7546:
URL: https://github.com/apache/incubator-doris/issues/7546
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org