You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "abhioncbr (via GitHub)" <gi...@apache.org> on 2023/08/25 20:00:58 UTC
[GitHub] [pinot] abhioncbr opened a new pull request, #11444: Skip computation incase of Null Literal for case/when transformation
abhioncbr opened a new pull request, #11444:
URL: https://github.com/apache/pinot/pull/11444
As per the discussion of the [PR](https://github.com/apache/pinot/pull/11395)
- Skipping computation of value block in case of Null Literal for case/when transformation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] shenyu0127 commented on pull request #11444: Skip computation incase of Null Literal for case/when transformation
Posted by "shenyu0127 (via GitHub)" <gi...@apache.org>.
shenyu0127 commented on PR #11444:
URL: https://github.com/apache/pinot/pull/11444#issuecomment-1696679544
> If would be good to add a test to `CaseTransformFunctionTest` which uses a query "SELECT CASE WHEN xx THEN xx ELSE NULL" and assert the query results to exercise the modified code paths.
Did you add a test? The rest looks good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #11444: Skip computation incase of Null Literal for case/when transformation
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #11444:
URL: https://github.com/apache/pinot/pull/11444#discussion_r1306585847
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -128,16 +129,22 @@ private void constructStatementListLegacy(List<TransformFunction> arguments) {
for (int i = numWhenStatements; i < numWhenStatements * 2; i++) {
_thenStatements.add(arguments.get(i));
}
- if (arguments.size() % 2 != 0) {
+ if (arguments.size() % 2 != 0 && isNotNullLiteralTransformation(arguments.get(arguments.size() - 1))) {
_elseStatement = arguments.get(arguments.size() - 1);
}
}
private TransformResultMetadata calculateResultMetadata() {
- TransformResultMetadata elseStatementResultMetadata = _elseStatement.getResultMetadata();
- DataType dataType = elseStatementResultMetadata.getDataType();
- Preconditions.checkState(elseStatementResultMetadata.isSingleValue(),
- "Unsupported multi-value expression in the ELSE clause");
+ DataType dataType;
Review Comment:
This change is required for supporting the NULL `_elseStatement`. Setting it up as `DataType.UNKNOWN`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] shenyu0127 commented on a diff in pull request #11444: Skip computation incase of Null Literal for case/when transformation
Posted by "shenyu0127 (via GitHub)" <gi...@apache.org>.
shenyu0127 commented on code in PR #11444:
URL: https://github.com/apache/pinot/pull/11444#discussion_r1306693880
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -257,6 +262,14 @@ public TransformResultMetadata getResultMetadata() {
return _resultMetadata;
}
+ private boolean isNotNullLiteralTransformation(TransformFunction function) {
Review Comment:
Prefer to name the function `isNullLiteralTransformation` to avoid potential double negatives.
https://blog.devgenius.io/code-smell-51-double-negatives-993b6160f3e1
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LiteralTransformFunction.java:
##########
@@ -269,4 +269,9 @@ public RoaringBitmap getNullBitmap(ValueBlock valueBlock) {
bitmap.add(0L, length);
return bitmap;
}
+
+ // checks whether the literal is null or not.
Review Comment:
nit: The comment is redundant, it simply repeats what the function name says.
https://blog.boot.dev/clean-code/code-comments/
Section "AVOID REDUNDANT COMMENTS. STRIVE FOR A SINGLE SOURCE OF TRUTH"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang merged pull request #11444: Skip computation incase of Null Literal for case/when transformation
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang merged PR #11444:
URL: https://github.com/apache/pinot/pull/11444
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #11444: Skip computation incase of Null Literal for case/when transformation
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #11444:
URL: https://github.com/apache/pinot/pull/11444#issuecomment-1693905119
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#11444](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (0fc0891) into [master](https://app.codecov.io/gh/apache/pinot/commit/2b4036292e68f97b0720583e04f3941644d4ec89?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (2b40362) will **decrease** coverage by `48.53%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #11444 +/- ##
=============================================
- Coverage 62.99% 14.46% -48.53%
+ Complexity 1107 201 -906
=============================================
Files 2303 2303
Lines 124041 124044 +3
Branches 18895 18896 +1
=============================================
- Hits 78137 17949 -60188
- Misses 40352 104577 +64225
+ Partials 5552 1518 -4034
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [integration](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-17](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.46% <0.00%> (-48.38%)` | :arrow_down: |
| [java-20](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.46% <0.00%> (-48.53%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.46% <0.00%> (-48.52%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.46% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files Changed](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...ator/transform/function/CaseTransformFunction.java](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vQ2FzZVRyYW5zZm9ybUZ1bmN0aW9uLmphdmE=) | `0.00% <0.00%> (-55.58%)` | :arrow_down: |
| [...r/transform/function/LiteralTransformFunction.java](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vTGl0ZXJhbFRyYW5zZm9ybUZ1bmN0aW9uLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
... and [1472 files with indirect coverage changes](https://app.codecov.io/gh/apache/pinot/pull/11444/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11444: Skip computation incase of Null Literal for case/when transformation
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #11444:
URL: https://github.com/apache/pinot/pull/11444#discussion_r1306589151
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -128,16 +129,22 @@ private void constructStatementListLegacy(List<TransformFunction> arguments) {
for (int i = numWhenStatements; i < numWhenStatements * 2; i++) {
_thenStatements.add(arguments.get(i));
}
- if (arguments.size() % 2 != 0) {
+ if (arguments.size() % 2 != 0 && isNotNullLiteralTransformation(arguments.get(arguments.size() - 1))) {
_elseStatement = arguments.get(arguments.size() - 1);
}
}
private TransformResultMetadata calculateResultMetadata() {
- TransformResultMetadata elseStatementResultMetadata = _elseStatement.getResultMetadata();
- DataType dataType = elseStatementResultMetadata.getDataType();
- Preconditions.checkState(elseStatementResultMetadata.isSingleValue(),
- "Unsupported multi-value expression in the ELSE clause");
+ DataType dataType;
Review Comment:
Good catch
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -128,16 +129,22 @@ private void constructStatementListLegacy(List<TransformFunction> arguments) {
for (int i = numWhenStatements; i < numWhenStatements * 2; i++) {
_thenStatements.add(arguments.get(i));
}
- if (arguments.size() % 2 != 0) {
+ if (arguments.size() % 2 != 0 && isNotNullLiteralTransformation(arguments.get(arguments.size() - 1))) {
_elseStatement = arguments.get(arguments.size() - 1);
}
}
private TransformResultMetadata calculateResultMetadata() {
- TransformResultMetadata elseStatementResultMetadata = _elseStatement.getResultMetadata();
- DataType dataType = elseStatementResultMetadata.getDataType();
- Preconditions.checkState(elseStatementResultMetadata.isSingleValue(),
- "Unsupported multi-value expression in the ELSE clause");
+ DataType dataType;
+ if (_elseStatement == null) {
+ // only possible in case of Null Literal transformation.
Review Comment:
This comment is not accurate. I believe the existing code has a bug that can throw `NPE` when there is no else statement. Suggest just removing this comment
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -257,6 +264,14 @@ public TransformResultMetadata getResultMetadata() {
return _resultMetadata;
}
+ private boolean isNotNullLiteralTransformation(TransformFunction function) {
+ if (Objects.equals(function.getName(), LiteralTransformFunction.FUNCTION_NAME)) {
Review Comment:
```suggestion
if (function instanceof LiteralTransformFunction) {
```
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -257,6 +264,14 @@ public TransformResultMetadata getResultMetadata() {
return _resultMetadata;
}
+ private boolean isNotNullLiteralTransformation(TransformFunction function) {
+ if (Objects.equals(function.getName(), LiteralTransformFunction.FUNCTION_NAME)) {
+ LiteralTransformFunction literalFUnction = (LiteralTransformFunction) function;
Review Comment:
(typo)
```suggestion
LiteralTransformFunction literalFunction = (LiteralTransformFunction) function;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org