You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/03/06 20:43:29 UTC
[GitHub] [druid] jon-wei opened a new pull request #9472: Add option to
control join filter rewrites
jon-wei opened a new pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472
This PR adds a new query context option for joins `enableJoinFilterRewrite` that controls whether filter clauses that reference non-base table columns will be rewritten into filters on base table columns.
The default is false, as the rewrite can be costly and not worth it.
This PR has:
- [x] been self-reviewed.
- [ ] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/licenses.yaml)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths.
- [ ] added integration tests.
- [x] been tested in a test Druid cluster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jon-wei commented on a change in pull request #9472: Add
option to control join filter rewrites
Posted by GitBox <gi...@apache.org>.
jon-wei commented on a change in pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472#discussion_r389932426
##########
File path: processing/src/main/java/org/apache/druid/segment/join/filter/JoinFilterAnalyzer.java
##########
@@ -88,7 +88,8 @@ public static JoinFilterSplit splitFilter(
HashJoinSegmentStorageAdapter hashJoinSegmentStorageAdapter,
Set<String> baseColumnNames,
@Nullable Filter originalFilter,
- boolean enableFilterPushDown
+ boolean enableFilterPushDown,
+ boolean enableFilterRewrite
Review comment:
Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ccaominh commented on a change in pull request #9472: Add
option to control join filter rewrites
Posted by GitBox <gi...@apache.org>.
ccaominh commented on a change in pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472#discussion_r389176394
##########
File path: processing/src/main/java/org/apache/druid/segment/join/filter/JoinFilterAnalyzer.java
##########
@@ -88,7 +88,8 @@ public static JoinFilterSplit splitFilter(
HashJoinSegmentStorageAdapter hashJoinSegmentStorageAdapter,
Set<String> baseColumnNames,
@Nullable Filter originalFilter,
- boolean enableFilterPushDown
+ boolean enableFilterPushDown,
+ boolean enableFilterRewrite
Review comment:
Javadoc reference to this method on line 54 needs to be updated (flagged by intellij inspection job)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jon-wei commented on a change in pull request #9472: Add
option to control join filter rewrites
Posted by GitBox <gi...@apache.org>.
jon-wei commented on a change in pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472#discussion_r389932223
##########
File path: processing/src/main/java/org/apache/druid/query/QueryContexts.java
##########
@@ -46,6 +46,7 @@
public static final String VECTORIZE_KEY = "vectorize";
public static final String VECTOR_SIZE_KEY = "vectorSize";
public static final String JOIN_FILTER_PUSH_DOWN_KEY = "enableJoinFilterPushDown";
+ public static final String JOIN_FILTER_REWRITE_KEY = "enableJoinFilterRewrite";
Review comment:
It should, I'll do a follow on PR that adds docs for `enableJoinFilterPushDown` as well
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ccaominh commented on a change in pull request #9472: Add
option to control join filter rewrites
Posted by GitBox <gi...@apache.org>.
ccaominh commented on a change in pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472#discussion_r389175177
##########
File path: processing/src/main/java/org/apache/druid/query/QueryContexts.java
##########
@@ -225,6 +227,11 @@ public String toString()
return parseBoolean(query, JOIN_FILTER_PUSH_DOWN_KEY, DEFAULT_ENABLE_JOIN_FILTER_PUSH_DOWN);
}
+ public static <T> boolean getEnableJoinFilterRewrite(Query<T> query)
+ {
+ return parseBoolean(query, JOIN_FILTER_PUSH_DOWN_KEY, DEFAULT_ENABLE_JOIN_FILTER_REWRITE);
Review comment:
Wrong key name (unused new rewrite key name getting flagged by intellij inspection job)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ccaominh commented on a change in pull request #9472: Add
option to control join filter rewrites
Posted by GitBox <gi...@apache.org>.
ccaominh commented on a change in pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472#discussion_r389176394
##########
File path: processing/src/main/java/org/apache/druid/segment/join/filter/JoinFilterAnalyzer.java
##########
@@ -88,7 +88,8 @@ public static JoinFilterSplit splitFilter(
HashJoinSegmentStorageAdapter hashJoinSegmentStorageAdapter,
Set<String> baseColumnNames,
@Nullable Filter originalFilter,
- boolean enableFilterPushDown
+ boolean enableFilterPushDown,
+ boolean enableFilterRewrite
Review comment:
Javadoc reference to this method on line 54 needs to be updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ccaominh commented on a change in pull request #9472: Add
option to control join filter rewrites
Posted by GitBox <gi...@apache.org>.
ccaominh commented on a change in pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472#discussion_r389172836
##########
File path: processing/src/main/java/org/apache/druid/query/QueryContexts.java
##########
@@ -46,6 +46,7 @@
public static final String VECTORIZE_KEY = "vectorize";
public static final String VECTOR_SIZE_KEY = "vectorSize";
public static final String JOIN_FILTER_PUSH_DOWN_KEY = "enableJoinFilterPushDown";
+ public static final String JOIN_FILTER_REWRITE_KEY = "enableJoinFilterRewrite";
Review comment:
Does this need to be added to the docs?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jon-wei commented on a change in pull request #9472: Add
option to control join filter rewrites
Posted by GitBox <gi...@apache.org>.
jon-wei commented on a change in pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472#discussion_r389932379
##########
File path: processing/src/main/java/org/apache/druid/query/QueryContexts.java
##########
@@ -225,6 +227,11 @@ public String toString()
return parseBoolean(query, JOIN_FILTER_PUSH_DOWN_KEY, DEFAULT_ENABLE_JOIN_FILTER_PUSH_DOWN);
}
+ public static <T> boolean getEnableJoinFilterRewrite(Query<T> query)
+ {
+ return parseBoolean(query, JOIN_FILTER_PUSH_DOWN_KEY, DEFAULT_ENABLE_JOIN_FILTER_REWRITE);
Review comment:
Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jon-wei merged pull request #9472: Add option to control
join filter rewrites
Posted by GitBox <gi...@apache.org>.
jon-wei merged pull request #9472: Add option to control join filter rewrites
URL: https://github.com/apache/druid/pull/9472
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org