You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/09/20 05:47:25 UTC
[GitHub] [pinot] ankitsultana opened a new pull request, #9433: [Draft] Add Support for Range Predicates
ankitsultana opened a new pull request, #9433:
URL: https://github.com/apache/pinot/pull/9433
This has a few bugs and room for perf improvements. Will update soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9433: [Draft] Add Support for Range Predicates
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9433:
URL: https://github.com/apache/pinot/pull/9433#discussion_r976608078
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/parser/CalciteRexExpressionParser.java:
##########
@@ -168,6 +168,7 @@ private static Expression compileFunctionExpression(RexExpression.FunctionCall r
return compileAndExpression(rexCall, pinotQuery);
case OR:
return compileOrExpression(rexCall, pinotQuery);
+ case SEARCH:
Review Comment:
why is this one needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9433: [Draft] Add Support for Range Predicates
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9433:
URL: https://github.com/apache/pinot/pull/9433#discussion_r976627663
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/RexExpressionUtils.java:
##########
@@ -61,31 +62,70 @@ static RexExpression handleCast(RexCall rexCall) {
operands);
}
- // TODO: Add support for range filter expressions (e.g. a > 0 and a < 30)
static RexExpression handleSearch(RexCall rexCall) {
List<RexNode> operands = rexCall.getOperands();
RexInputRef rexInputRef = (RexInputRef) operands.get(0);
RexLiteral rexLiteral = (RexLiteral) operands.get(1);
FieldSpec.DataType dataType = RexExpression.toDataType(rexLiteral.getType());
Sarg sarg = rexLiteral.getValueAs(Sarg.class);
- if (sarg.isPoints()) {
- return new RexExpression.FunctionCall(SqlKind.IN, dataType, SqlKind.IN.name(), toFunctionOperands(rexInputRef,
- sarg.rangeSet.asRanges(), dataType));
- } else if (sarg.isComplementedPoints()) {
- return new RexExpression.FunctionCall(SqlKind.NOT_IN, dataType, SqlKind.NOT_IN.name(),
- toFunctionOperands(rexInputRef, sarg.rangeSet.complement().asRanges(), dataType));
+ if (sarg.isPoints() || sarg.isComplementedPoints()) {
+ SqlKind kind = sarg.isPoints() ? SqlKind.IN : SqlKind.NOT_IN;
+ Set<Range> rangeSet = kind.equals(SqlKind.IN) ? sarg.rangeSet.asRanges() : sarg.rangeSet.complement().asRanges();
+ List<RexExpression> functionOperands = new ArrayList<>(1 + rangeSet.size());
+ functionOperands.add(RexExpression.toRexExpression(rexInputRef));
+ functionOperands.addAll(rangeSet.stream().map(
+ range -> new RexExpression.Literal(dataType, RexExpression.toRexValue(dataType, range.lowerEndpoint()))
+ ).collect(Collectors.toList()));
+ return new RexExpression.FunctionCall(kind, dataType, kind.name(), functionOperands);
Review Comment:
i wasn't sure if this would be the best way to handle. but let's do it this way. since all range predicates for a single input reference will be lump into one search. let's do
1. all points gets to be in a `IN` search; (only do complemented points if user explicitly said NOT_IN)
2. all continuous ranges be compiled into `OR` search chain.
link 1 and 2 together with an OR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] ankitsultana commented on pull request #9433: [Draft] Add Support for Range Predicates
Posted by GitBox <gi...@apache.org>.
ankitsultana commented on PR #9433:
URL: https://github.com/apache/pinot/pull/9433#issuecomment-1254477439
Closing since Rong has raised a PR for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] ankitsultana closed pull request #9433: [Draft] Add Support for Range Predicates
Posted by GitBox <gi...@apache.org>.
ankitsultana closed pull request #9433: [Draft] Add Support for Range Predicates
URL: https://github.com/apache/pinot/pull/9433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] ankitsultana commented on a diff in pull request #9433: [Draft] Add Support for Range Predicates
Posted by GitBox <gi...@apache.org>.
ankitsultana commented on code in PR #9433:
URL: https://github.com/apache/pinot/pull/9433#discussion_r976875234
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/parser/CalciteRexExpressionParser.java:
##########
@@ -168,6 +168,7 @@ private static Expression compileFunctionExpression(RexExpression.FunctionCall r
return compileAndExpression(rexCall, pinotQuery);
case OR:
return compileOrExpression(rexCall, pinotQuery);
+ case SEARCH:
Review Comment:
Please ignore. This is not needed now. It was needed in the approach in the first-commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #9433: [Draft] Add Support for Range Predicates
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9433:
URL: https://github.com/apache/pinot/pull/9433#issuecomment-1251898777
# [Codecov](https://codecov.io/gh/apache/pinot/pull/9433?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#9433](https://codecov.io/gh/apache/pinot/pull/9433?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (473e53a) into [master](https://codecov.io/gh/apache/pinot/commit/2d6665b8e5fa0842ef67b3d9896c5e04ecad78e9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2d6665b) will **decrease** coverage by `43.73%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #9433 +/- ##
=============================================
- Coverage 69.73% 26.00% -43.74%
+ Complexity 4787 44 -4743
=============================================
Files 1890 1877 -13
Lines 100756 100526 -230
Branches 15350 15298 -52
=============================================
- Hits 70266 26142 -44124
- Misses 25507 71736 +46229
+ Partials 4983 2648 -2335
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `26.00% <0.00%> (-0.02%)` | :arrow_down: |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/9433?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...pinot/query/parser/CalciteRexExpressionParser.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGFyc2VyL0NhbGNpdGVSZXhFeHByZXNzaW9uUGFyc2VyLmphdmE=) | `0.00% <ø> (-51.07%)` | :arrow_down: |
| [...inot/query/planner/logical/RexExpressionUtils.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvcGxhbm5lci9sb2dpY2FsL1JleEV4cHJlc3Npb25VdGlscy5qYXZh) | `0.00% <0.00%> (-96.43%)` | :arrow_down: |
| [...in/java/org/apache/pinot/spi/utils/BytesUtils.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdXRpbHMvQnl0ZXNVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...java/org/apache/pinot/spi/trace/BaseRecording.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvQmFzZVJlY29yZGluZy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...java/org/apache/pinot/spi/trace/NoOpRecording.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvTm9PcFJlY29yZGluZy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/spi/config/table/FSTType.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvY29uZmlnL3RhYmxlL0ZTVFR5cGUuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/spi/config/user/RoleType.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvY29uZmlnL3VzZXIvUm9sZVR5cGUuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/spi/data/MetricFieldSpec.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvZGF0YS9NZXRyaWNGaWVsZFNwZWMuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...java/org/apache/pinot/common/tier/TierFactory.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/pinot/spi/config/table/TableType.java](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvY29uZmlnL3RhYmxlL1RhYmxlVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [1375 more](https://codecov.io/gh/apache/pinot/pull/9433/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org