You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "pranavbhole (via GitHub)" <gi...@apache.org> on 2024/04/05 06:30:56 UTC
[PR] Moving object contains to Bound for string/object matchers (druid)
pranavbhole opened a new pull request, #16241:
URL: https://github.com/apache/druid/pull/16241
Followup of https://github.com/apache/druid/pull/16029/files + Addressing @abhishekagarwal87 's comments from previous PR.
### Description
Creating a generic object contains interface in Spatial Bound filters for creating string/object matchers on top of the spatial column. SpatialFilter creates string matchers in case if we dont have indexes( Happens this in creating Incremental index), moving it to bound makes it extensible to have custom object matchers for given bound filter.
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "LakshSingla (via GitHub)" <gi...@apache.org>.
LakshSingla commented on code in PR #16241:
URL: https://github.com/apache/druid/pull/16241#discussion_r1591906052
##########
processing/src/main/java/org/apache/druid/segment/filter/SpatialFilter.java:
##########
@@ -174,8 +173,18 @@ public DruidObjectPredicate<String> makeStringPredicate()
if (input == null) {
return DruidPredicateMatch.UNKNOWN;
}
- final float[] coordinate = SpatialDimensionRowTransformer.decode(input);
- return DruidPredicateMatch.of(bound.contains(coordinate));
+ return DruidPredicateMatch.of(bound.containsObj(input));
+ };
+ }
+
+ @Override
+ public DruidObjectPredicate<Object> makeObjectPredicate()
+ {
+ return input -> {
+ if (input == null) {
+ return DruidPredicateMatch.UNKNOWN;
+ }
+ return DruidPredicateMatch.of(bound.containsObj(input));
Review Comment:
Since we know that it is always a string, why do we need a new method called `containsObj` that handles strings? Since the knowledge that it will always be a string is present here, I think it would be better if we cast the input to float[] and pass it to the filter as `bound.contains(decodedInput)`.
##########
processing/src/main/java/org/apache/druid/collections/spatial/search/Bound.java:
##########
@@ -43,6 +45,8 @@ public interface Bound<TCoordinateArray, TPoint extends ImmutableNode<TCoordinat
boolean contains(TCoordinateArray coords);
+ boolean containsObj(@Nullable Object input);
Review Comment:
Can you please add a Javadoc for this method explaining how it differs from `contains`, what arguments can the implementing class expect, and what happens if `null` is passed as the argument?
##########
processing/src/main/java/org/apache/druid/collections/spatial/search/RectangularBound.java:
##########
@@ -118,6 +121,19 @@ public boolean contains(float[] coords)
return true;
}
+ @Override
+ public boolean containsObj(@Nullable Object input)
Review Comment:
Method seems to be too "one dimensional" to be called `containsObj`. It doesn't handle the float[] types.
It is decoding and calling contains on the string input. Better names would be along the lines of `containsSerializedString` or `decodeAndCheckContains`, and the input type should be a String.
A better solution in my opinion would be to handle this conversion in the callers, and pass the decoded dimension to the filter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "kgyrtkirk (via GitHub)" <gi...@apache.org>.
kgyrtkirk merged PR #16241:
URL: https://github.com/apache/druid/pull/16241
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "kgyrtkirk (via GitHub)" <gi...@apache.org>.
kgyrtkirk commented on code in PR #16241:
URL: https://github.com/apache/druid/pull/16241#discussion_r1556113895
##########
processing/src/main/java/org/apache/druid/segment/filter/SpatialFilter.java:
##########
@@ -174,8 +173,18 @@ public DruidObjectPredicate<String> makeStringPredicate()
if (input == null) {
return DruidPredicateMatch.UNKNOWN;
}
- final float[] coordinate = SpatialDimensionRowTransformer.decode(input);
- return DruidPredicateMatch.of(bound.contains(coordinate));
+ return DruidPredicateMatch.of(bound.containsObj(input));
+ };
+ }
+
+ @Override
+ public DruidObjectPredicate<Object> makeObjectPredicate()
+ {
+ return input -> {
+ if (input == null) {
+ return DruidPredicateMatch.UNKNOWN;
+ }
+ return DruidPredicateMatch.of(bound.containsObj(input));
Review Comment:
I wonder if `input` could be a `float[]` or not at this point - could it be?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "pranavbhole (via GitHub)" <gi...@apache.org>.
pranavbhole commented on code in PR #16241:
URL: https://github.com/apache/druid/pull/16241#discussion_r1597914376
##########
processing/src/main/java/org/apache/druid/collections/spatial/search/Bound.java:
##########
@@ -43,6 +45,8 @@ public interface Bound<TCoordinateArray, TPoint extends ImmutableNode<TCoordinat
boolean contains(TCoordinateArray coords);
+ boolean containsObj(@Nullable Object input);
Review Comment:
added javadoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "pranavbhole (via GitHub)" <gi...@apache.org>.
pranavbhole commented on code in PR #16241:
URL: https://github.com/apache/druid/pull/16241#discussion_r1562702088
##########
processing/src/main/java/org/apache/druid/segment/filter/SpatialFilter.java:
##########
@@ -174,8 +173,18 @@ public DruidObjectPredicate<String> makeStringPredicate()
if (input == null) {
return DruidPredicateMatch.UNKNOWN;
}
- final float[] coordinate = SpatialDimensionRowTransformer.decode(input);
- return DruidPredicateMatch.of(bound.contains(coordinate));
+ return DruidPredicateMatch.of(bound.containsObj(input));
+ };
+ }
+
+ @Override
+ public DruidObjectPredicate<Object> makeObjectPredicate()
+ {
+ return input -> {
+ if (input == null) {
+ return DruidPredicateMatch.UNKNOWN;
+ }
+ return DruidPredicateMatch.of(bound.containsObj(input));
Review Comment:
Spatial column is stored as String and input can be only String in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "pranavbhole (via GitHub)" <gi...@apache.org>.
pranavbhole commented on code in PR #16241:
URL: https://github.com/apache/druid/pull/16241#discussion_r1597914376
##########
processing/src/main/java/org/apache/druid/collections/spatial/search/Bound.java:
##########
@@ -43,6 +45,8 @@ public interface Bound<TCoordinateArray, TPoint extends ImmutableNode<TCoordinat
boolean contains(TCoordinateArray coords);
+ boolean containsObj(@Nullable Object input);
Review Comment:
Yes, I will add java doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "pranavbhole (via GitHub)" <gi...@apache.org>.
pranavbhole commented on code in PR #16241:
URL: https://github.com/apache/druid/pull/16241#discussion_r1597912954
##########
processing/src/main/java/org/apache/druid/segment/filter/SpatialFilter.java:
##########
@@ -174,8 +173,18 @@ public DruidObjectPredicate<String> makeStringPredicate()
if (input == null) {
return DruidPredicateMatch.UNKNOWN;
}
- final float[] coordinate = SpatialDimensionRowTransformer.decode(input);
- return DruidPredicateMatch.of(bound.contains(coordinate));
+ return DruidPredicateMatch.of(bound.containsObj(input));
+ };
+ }
+
+ @Override
+ public DruidObjectPredicate<Object> makeObjectPredicate()
+ {
+ return input -> {
+ if (input == null) {
+ return DruidPredicateMatch.UNKNOWN;
+ }
+ return DruidPredicateMatch.of(bound.containsObj(input));
Review Comment:
containsObj method is mainly added for extensibility perspective, it makes life easier to create object matchers and we can utilize the existing spatial filters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "pranavbhole (via GitHub)" <gi...@apache.org>.
pranavbhole commented on code in PR #16241:
URL: https://github.com/apache/druid/pull/16241#discussion_r1597913858
##########
processing/src/main/java/org/apache/druid/collections/spatial/search/RectangularBound.java:
##########
@@ -118,6 +121,19 @@ public boolean contains(float[] coords)
return true;
}
+ @Override
+ public boolean containsObj(@Nullable Object input)
Review Comment:
containsObj method is mainly added for extensibility perspective to deal with bound filtering on object matchers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Moving object contains to Bound for string/object matchers (druid)
Posted by "kgyrtkirk (via GitHub)" <gi...@apache.org>.
kgyrtkirk commented on code in PR #16241:
URL: https://github.com/apache/druid/pull/16241#discussion_r1604609689
##########
processing/src/main/java/org/apache/druid/collections/spatial/search/Bound.java:
##########
@@ -43,6 +45,8 @@ public interface Bound<TCoordinateArray, TPoint extends ImmutableNode<TCoordinat
boolean contains(TCoordinateArray coords);
+ boolean containsObj(@Nullable Object input);
Review Comment:
I wonder if changing the original `contains` method to have a `TCoordinateArray` could help?
if that's not enough I still don't think that supporting different types like `float[],int[],double[]` should flow down entirely to the shapes themselves....instead of `Object` it could be `GenericPoint` or something like that - which could then handle comparision differently...
it seems to me that this class and its methods are a little bit skewed ; adding `containsObj` here - and implementing it only in `RectangularBound` makes it also avaialable for `Polygon` - which is wrong...
a way to fix this could be to change the method's name to reflect its intention - do you want to extend the system to enable a quick `AABB` based check - in that case you could name this method as `isInsideAABB` or something which reflects its desired use - but then there must be a followup check which will be done to ensure the real contains relation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org