You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "ankitsultana (via GitHub)" <gi...@apache.org> on 2023/04/03 02:03:58 UTC
[GitHub] [pinot] ankitsultana opened a new issue, #10523: [multistage] Handle Comparisons Against Extremal Values
ankitsultana opened a new issue, #10523:
URL: https://github.com/apache/pinot/issues/10523
In V1 engine, we do not allow comparisons such as `longCol < Long.MIN_VALUE`. [Example](https://github.com/apache/pinot/blob/master/pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/RangePredicateEvaluatorFactory.java#L381)
This works fine for V1 queries, but in V2 engine Calcite can re-write filters. For example: a query with filters like `colName != Long.MIN_VALUE or colName < 10` gets translated to `(colName < Long.MIN_VALUE) OR (colName > Long.MIN_VALUE AND colName < 10`.
This can lead to failures with errors such as follows:
```
Caused by: java.lang.IllegalArgumentException: Invalid range: colName < '-9223372036854775808'
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:210)
at org.apache.pinot.core.operator.filter.predicate.RangePredicateEvaluatorFactory$LongRawValueBasedRangePredicateEvaluator.<init>(RangePredicateEvaluatorFactory.java:386)
at org.apache.pinot.core.operator.filter.predicate.RangePredicateEvaluatorFactory.newRawValueBasedEvaluator(RangePredicateEvaluatorFactory.java:85)
at org.apache.pinot.core.operator.filter.predicate.RangePredicateEvaluatorFactory$UnsortedDictionaryBasedRangePredicateEvaluator.<init>(RangePredicateEvaluatorFactory.java:273)
```
Some possible solutions:
1. Add support for these filters in V1 engine itself.
2. Add a rule in v2 optimizers to detect tautologies. The rule could be enhanced in the future to add more cases.
cc: @walterddr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] ankitsultana commented on issue #10523: [multistage] Handle Comparisons Against Extremal Values
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on issue #10523:
URL: https://github.com/apache/pinot/issues/10523#issuecomment-1504720847
Hi @abhioncbr , thanks for showing interest.
At this point we are not sure what's the right approach for this. Both of the approaches have pros and cons. Would be best to wait for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on issue #10523: [multistage] Handle Comparisons Against Extremal Values
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on issue #10523:
URL: https://github.com/apache/pinot/issues/10523#issuecomment-1502553125
@ankitsultana / @walterddr I am planning to start work on this. Just checking, do you guys think it's good to start work and not be dependent on any other work? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on issue #10523: [multistage] Handle Comparisons Against Extremal Values
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on issue #10523:
URL: https://github.com/apache/pinot/issues/10523#issuecomment-1494527660
I want to give it a try if it's not urgent. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org