You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "saurabhd336 (via GitHub)" <gi...@apache.org> on 2023/08/29 06:03:57 UTC
[GitHub] [pinot] saurabhd336 opened a new pull request, #11455: Add array functions to v2 fn registry
saurabhd336 opened a new pull request, #11455:
URL: https://github.com/apache/pinot/pull/11455
```select arrayAverage(intMv) as col from transformsAndAggregationsTest```
fails with
```
[{"errorCode":150,"message":"SQLParsingError:\nError composing query plan for 'SET \"useMultistageEngine\"=true;\nselect col from (select arrayLength(\"t1\".\"floatMv\") as col, \"t1\".\"$docId\" as docId from transformsAndAggregationsTest t1 join transformsAndAggregationsTest t2 on \"t1\".\"$docId\" = \"t2\".\"$docId\") order by docId limit 100': From line 2, column 25 to line 2, column 51: No match found for function signature arrayLength(<REAL ARRAY>)'\norg.apache.pinot.query.QueryEnvironment.planQuery(QueryEnvironment.java:180)\norg.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.handleRequest(MultiStageBrokerRequestHandler.java:137)\norg.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:263)\norg.apache.pinot.broker.requesthandler.BrokerRequestHandler.handleRequest(BrokerRequestHandler.java:48)\nFrom line 2, column 25 to line 2, column 51: No match found for function signature arrayLength(<REAL ARRAY>)\nj
dk.internal.reflect.GeneratedConstructorAccessor116.newInstance(Unknown Source)\njava.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\njava.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)\norg.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)\nNo match found for function signature arrayLength(<REAL ARRAY>)\njdk.internal.reflect.GeneratedConstructorAccessor115.newInstance(Unknown Source)\njava.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\njava.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)\norg.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)\n"}]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #11455: Add array functions to v2 fn registry
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #11455:
URL: https://github.com/apache/pinot/pull/11455#issuecomment-1696859928
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11455?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#11455](https://app.codecov.io/gh/apache/pinot/pull/11455?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (e1c1830) into [master](https://app.codecov.io/gh/apache/pinot/commit/94d3b7dfe96092fd892274445bd9c4ed34f27849?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (94d3b7d) will **decrease** coverage by `48.54%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #11455 +/- ##
=============================================
- Coverage 63.00% 14.47% -48.54%
+ Complexity 1094 201 -893
=============================================
Files 2302 2302
Lines 124025 124030 +5
Branches 18901 18901
=============================================
- Hits 78137 17948 -60189
- Misses 40338 104564 +64226
+ Partials 5550 1518 -4032
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [integration](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-17](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-20](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.47% <0.00%> (-48.38%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.47% <0.00%> (-48.54%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.47% <0.00%> (-48.53%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/11455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.47% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files Changed](https://app.codecov.io/gh/apache/pinot/pull/11455?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...apache/pinot/common/function/FunctionRegistry.java](https://app.codecov.io/gh/apache/pinot/pull/11455?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25SZWdpc3RyeS5qYXZh) | `0.00% <0.00%> (-71.70%)` | :arrow_down: |
| [...e/pinot/common/function/TransformFunctionType.java](https://app.codecov.io/gh/apache/pinot/pull/11455?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `0.00% <0.00%> (-88.39%)` | :arrow_down: |
... and [1470 files with indirect coverage changes](https://app.codecov.io/gh/apache/pinot/pull/11455/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on a diff in pull request #11455: Add array functions to v2 fn registry
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 commented on code in PR #11455:
URL: https://github.com/apache/pinot/pull/11455#discussion_r1308435246
##########
pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java:
##########
@@ -58,7 +58,7 @@ public enum TransformFunctionType {
FLOOR("floor"),
LOG("log", "ln"),
LOG2("log2"),
- LOG10("log10"),
+ LOG10("log10", "log1"),
Review Comment:
No actually. `select log10(col)` fails because calcite rewrites it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11455: Add array functions to v2 fn registry
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on code in PR #11455:
URL: https://github.com/apache/pinot/pull/11455#discussion_r1308426688
##########
pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java:
##########
@@ -58,7 +58,7 @@ public enum TransformFunctionType {
FLOOR("floor"),
LOG("log", "ln"),
LOG2("log2"),
- LOG10("log10"),
+ LOG10("log10", "log1"),
Review Comment:
typo?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11455: Add array functions to v2 fn registry
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on code in PR #11455:
URL: https://github.com/apache/pinot/pull/11455#discussion_r1308983963
##########
pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java:
##########
@@ -58,7 +58,7 @@ public enum TransformFunctionType {
FLOOR("floor"),
LOG("log", "ln"),
LOG2("log2"),
- LOG10("log10"),
+ LOG10("log10", "log1"),
Review Comment:
ack, we should fix the logic inside: `canonicalizeFunctionNameInternal`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 commented on pull request #11455: Add array functions to v2 fn registry
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on PR #11455:
URL: https://github.com/apache/pinot/pull/11455#issuecomment-1698535211
Thanks for fixing this @saurabhd336 !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on a diff in pull request #11455: Add array functions to v2 fn registry
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 commented on code in PR #11455:
URL: https://github.com/apache/pinot/pull/11455#discussion_r1331528911
##########
pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java:
##########
@@ -217,5 +217,30 @@ public static Object clpDecode(String logtypeFieldName, String dictVarsFieldName
public static String arrayToMV(Object multiValue) {
throw new UnsupportedOperationException("Placeholder scalar function, should not reach here");
}
+
+ @ScalarFunction(names = "arrayMin", isPlaceholder = true)
Review Comment:
Sorry I missed this comment. We added unit tests in a different pr here https://github.com/apache/pinot/pull/11498/files#diff-9724fd2cc0075a3c9c9a29cc7b98bb921cfe2690c2cc0b6e0ad47dec0589e6abR630
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on a diff in pull request #11455: Add array functions to v2 fn registry
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 commented on code in PR #11455:
URL: https://github.com/apache/pinot/pull/11455#discussion_r1308435246
##########
pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java:
##########
@@ -58,7 +58,7 @@ public enum TransformFunctionType {
FLOOR("floor"),
LOG("log", "ln"),
LOG2("log2"),
- LOG10("log10"),
+ LOG10("log10", "log1"),
Review Comment:
No actually. `select log10(col)` fails because calcite rewrites it to `log1(col)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on pull request #11455: Add array functions to v2 fn registry
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 commented on PR #11455:
URL: https://github.com/apache/pinot/pull/11455#issuecomment-1698468930
@xiangfu0 removed the log10 change. Will raise seperate PR for fixing that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 commented on pull request #11455: Add array functions to v2 fn registry
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on PR #11455:
URL: https://github.com/apache/pinot/pull/11455#issuecomment-1698535435
> @xiangfu0 removed the log10 change. Will raise seperate PR for fixing that
I gave a try here: https://github.com/apache/pinot/pull/11460
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on a diff in pull request #11455: Add array functions to v2 fn registry
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 commented on code in PR #11455:
URL: https://github.com/apache/pinot/pull/11455#discussion_r1308477507
##########
pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java:
##########
@@ -58,7 +58,7 @@ public enum TransformFunctionType {
FLOOR("floor"),
LOG("log", "ln"),
LOG2("log2"),
- LOG10("log10"),
+ LOG10("log10", "log1"),
Review Comment:
Actually, looks like the issue is with the canonicalization, https://github.com/apache/pinot/blob/master/pinot-query-planner/src/main/java/org/apache/pinot/query/parser/CalciteRexExpressionParser.java#L297
So maybe we should fix that instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 merged pull request #11455: Add array functions to v2 fn registry
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 merged PR #11455:
URL: https://github.com/apache/pinot/pull/11455
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #11455: Add array functions to v2 fn registry
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #11455:
URL: https://github.com/apache/pinot/pull/11455#discussion_r1333164859
##########
pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java:
##########
@@ -217,5 +217,30 @@ public static Object clpDecode(String logtypeFieldName, String dictVarsFieldName
public static String arrayToMV(Object multiValue) {
throw new UnsupportedOperationException("Placeholder scalar function, should not reach here");
}
+
+ @ScalarFunction(names = "arrayMin", isPlaceholder = true)
Review Comment:
no worries. was my retro review after the merge. resolving
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #11455: Add array functions to v2 fn registry
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #11455:
URL: https://github.com/apache/pinot/pull/11455#discussion_r1326660277
##########
pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java:
##########
@@ -217,5 +217,30 @@ public static Object clpDecode(String logtypeFieldName, String dictVarsFieldName
public static String arrayToMV(Object multiValue) {
throw new UnsupportedOperationException("Placeholder scalar function, should not reach here");
}
+
+ @ScalarFunction(names = "arrayMin", isPlaceholder = true)
Review Comment:
did we add any tests for this? unit or integraiton tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org