You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/08/12 03:54:55 UTC
[GitHub] [druid] rohangarg opened a new pull request, #12894: Fix hasBuiltInFilters for joins converted to filters
rohangarg opened a new pull request, #12894:
URL: https://github.com/apache/druid/pull/12894
As a part of the fix in #11517, the case where if a join is fully converted into filter is missing.
If the join fully converts to a non-null filter, we should convey that the storage adapter might have built-in filters.
It might be the case that the outer query doesn't have any filters, so to inform the topN query engine that the scan is not on the full segment, we need to convey it via the join's storage adapter. This will help the topN query engine in taking decision whether to short circuit the scan or not.
This was uncovered when the existing test for `CalciteQueryTest#testExactTopNOnInnerJoinWithLimit` started failing as a part of #12868
This PR has:
- [x] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [x] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on a diff in pull request #12894: Fix hasBuiltInFilters for joins converted to filters
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on code in PR #12894:
URL: https://github.com/apache/druid/pull/12894#discussion_r944203041
##########
processing/src/main/java/org/apache/druid/segment/join/HashJoinSegmentStorageAdapter.java:
##########
@@ -212,8 +212,11 @@ public Metadata getMetadata()
@Override
public boolean hasBuiltInFilters()
{
- return clauses.stream()
- .anyMatch(clause -> clause.getJoinType() == JoinType.INNER && !clause.getCondition().isAlwaysTrue());
+ // empty clause means that the join has been converted to a filter (captured in baseFilter). So, if the baseFilter
+ // non-null, then the adapter can potentially filter rows.
+ return (clauses.isEmpty() && baseFilter != null) || clauses.stream().anyMatch(
+ clause -> clause.getJoinType() == JoinType.INNER && !clause.getCondition().isAlwaysTrue()
Review Comment:
if the baseFilter is not null, then rows from underlying storage can be filtered, no matter what is in the clauses.
##########
processing/src/main/java/org/apache/druid/segment/join/HashJoinSegmentStorageAdapter.java:
##########
@@ -212,8 +212,11 @@ public Metadata getMetadata()
@Override
public boolean hasBuiltInFilters()
{
- return clauses.stream()
- .anyMatch(clause -> clause.getJoinType() == JoinType.INNER && !clause.getCondition().isAlwaysTrue());
+ // empty clause means that the join has been converted to a filter (captured in baseFilter). So, if the baseFilter
+ // non-null, then the adapter can potentially filter rows.
+ return (clauses.isEmpty() && baseFilter != null) || clauses.stream().anyMatch(
+ clause -> clause.getJoinType() == JoinType.INNER && !clause.getCondition().isAlwaysTrue()
Review Comment:
shouldn't this be
```suggestion
// empty clause means that the join has been converted to a filter (captured in baseFilter). So, if the baseFilter
// non-null, then the adapter can potentially filter rows.
return (baseFilter != null) || clauses.stream().anyMatch(
clause -> clause.getJoinType() == JoinType.INNER && !clause.getCondition().isAlwaysTrue()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] rohangarg merged pull request #12894: Fix hasBuiltInFilters for joins
Posted by GitBox <gi...@apache.org>.
rohangarg merged PR #12894:
URL: https://github.com/apache/druid/pull/12894
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] rohangarg commented on a diff in pull request #12894: Fix hasBuiltInFilters for joins converted to filters
Posted by GitBox <gi...@apache.org>.
rohangarg commented on code in PR #12894:
URL: https://github.com/apache/druid/pull/12894#discussion_r944289931
##########
processing/src/main/java/org/apache/druid/segment/join/HashJoinSegmentStorageAdapter.java:
##########
@@ -212,8 +212,11 @@ public Metadata getMetadata()
@Override
public boolean hasBuiltInFilters()
{
- return clauses.stream()
- .anyMatch(clause -> clause.getJoinType() == JoinType.INNER && !clause.getCondition().isAlwaysTrue());
+ // empty clause means that the join has been converted to a filter (captured in baseFilter). So, if the baseFilter
+ // non-null, then the adapter can potentially filter rows.
+ return (clauses.isEmpty() && baseFilter != null) || clauses.stream().anyMatch(
+ clause -> clause.getJoinType() == JoinType.INNER && !clause.getCondition().isAlwaysTrue()
Review Comment:
good catch! your suggestion will also accomodate cases where the baseFilter is originating from other places too apart from join to filter conversion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org