You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "abhioncbr (via GitHub)" <gi...@apache.org> on 2023/08/25 20:00:58 UTC

[GitHub] [pinot] abhioncbr opened a new pull request, #11444: Skip computation incase of Null Literal for case/when transformation

abhioncbr opened a new pull request, #11444:
URL: https://github.com/apache/pinot/pull/11444

   As per the discussion of the [PR](https://github.com/apache/pinot/pull/11395) 
   
   - Skipping computation of value block in case of Null Literal for case/when transformation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] shenyu0127 commented on pull request #11444: Skip computation incase of Null Literal for case/when transformation

Posted by "shenyu0127 (via GitHub)" <gi...@apache.org>.
shenyu0127 commented on PR #11444:
URL: https://github.com/apache/pinot/pull/11444#issuecomment-1696679544

   > If would be good to add a test to `CaseTransformFunctionTest` which uses a query "SELECT CASE WHEN xx THEN xx ELSE NULL" and assert the query results to exercise the modified code paths.
   
   Did you add a test? The rest looks good to me.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] abhioncbr commented on a diff in pull request #11444: Skip computation incase of Null Literal for case/when transformation

Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #11444:
URL: https://github.com/apache/pinot/pull/11444#discussion_r1306585847


##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -128,16 +129,22 @@ private void constructStatementListLegacy(List<TransformFunction> arguments) {
     for (int i = numWhenStatements; i < numWhenStatements * 2; i++) {
       _thenStatements.add(arguments.get(i));
     }
-    if (arguments.size() % 2 != 0) {
+    if (arguments.size() % 2 != 0 && isNotNullLiteralTransformation(arguments.get(arguments.size() - 1))) {
       _elseStatement = arguments.get(arguments.size() - 1);
     }
   }
 
   private TransformResultMetadata calculateResultMetadata() {
-    TransformResultMetadata elseStatementResultMetadata = _elseStatement.getResultMetadata();
-    DataType dataType = elseStatementResultMetadata.getDataType();
-    Preconditions.checkState(elseStatementResultMetadata.isSingleValue(),
-        "Unsupported multi-value expression in the ELSE clause");
+    DataType dataType;

Review Comment:
   This change is required for supporting the NULL `_elseStatement`. Setting it up as `DataType.UNKNOWN`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] shenyu0127 commented on a diff in pull request #11444: Skip computation incase of Null Literal for case/when transformation

Posted by "shenyu0127 (via GitHub)" <gi...@apache.org>.
shenyu0127 commented on code in PR #11444:
URL: https://github.com/apache/pinot/pull/11444#discussion_r1306693880


##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -257,6 +262,14 @@ public TransformResultMetadata getResultMetadata() {
     return _resultMetadata;
   }
 
+  private boolean isNotNullLiteralTransformation(TransformFunction function) {

Review Comment:
   Prefer to name the function `isNullLiteralTransformation` to avoid potential double negatives.
   
   https://blog.devgenius.io/code-smell-51-double-negatives-993b6160f3e1



##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LiteralTransformFunction.java:
##########
@@ -269,4 +269,9 @@ public RoaringBitmap getNullBitmap(ValueBlock valueBlock) {
     bitmap.add(0L, length);
     return bitmap;
   }
+
+  // checks whether the literal is null or not.

Review Comment:
   nit: The comment is redundant, it simply repeats what the function name says.
   
   https://blog.boot.dev/clean-code/code-comments/
   Section "AVOID REDUNDANT COMMENTS. STRIVE FOR A SINGLE SOURCE OF TRUTH"



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang merged pull request #11444: Skip computation incase of Null Literal for case/when transformation

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang merged PR #11444:
URL: https://github.com/apache/pinot/pull/11444


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter commented on pull request #11444: Skip computation incase of Null Literal for case/when transformation

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #11444:
URL: https://github.com/apache/pinot/pull/11444#issuecomment-1693905119

   ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   > Merging [#11444](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (0fc0891) into [master](https://app.codecov.io/gh/apache/pinot/commit/2b4036292e68f97b0720583e04f3941644d4ec89?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (2b40362) will **decrease** coverage by `48.53%`.
   > The diff coverage is `0.00%`.
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #11444       +/-   ##
   =============================================
   - Coverage     62.99%   14.46%   -48.53%     
   + Complexity     1107      201      -906     
   =============================================
     Files          2303     2303               
     Lines        124041   124044        +3     
     Branches      18895    18896        +1     
   =============================================
   - Hits          78137    17949    -60188     
   - Misses        40352   104577    +64225     
   + Partials       5552     1518     -4034     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [integration](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration1](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration2](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-11](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-17](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.46% <0.00%> (-48.38%)` | :arrow_down: |
   | [java-20](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [temurin](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.46% <0.00%> (-48.53%)` | :arrow_down: |
   | [unittests](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.46% <0.00%> (-48.52%)` | :arrow_down: |
   | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/11444/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `14.46% <0.00%> (-0.01%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files Changed](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [...ator/transform/function/CaseTransformFunction.java](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vQ2FzZVRyYW5zZm9ybUZ1bmN0aW9uLmphdmE=) | `0.00% <0.00%> (-55.58%)` | :arrow_down: |
   | [...r/transform/function/LiteralTransformFunction.java](https://app.codecov.io/gh/apache/pinot/pull/11444?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vTGl0ZXJhbFRyYW5zZm9ybUZ1bmN0aW9uLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
   
   ... and [1472 files with indirect coverage changes](https://app.codecov.io/gh/apache/pinot/pull/11444/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11444: Skip computation incase of Null Literal for case/when transformation

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #11444:
URL: https://github.com/apache/pinot/pull/11444#discussion_r1306589151


##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -128,16 +129,22 @@ private void constructStatementListLegacy(List<TransformFunction> arguments) {
     for (int i = numWhenStatements; i < numWhenStatements * 2; i++) {
       _thenStatements.add(arguments.get(i));
     }
-    if (arguments.size() % 2 != 0) {
+    if (arguments.size() % 2 != 0 && isNotNullLiteralTransformation(arguments.get(arguments.size() - 1))) {
       _elseStatement = arguments.get(arguments.size() - 1);
     }
   }
 
   private TransformResultMetadata calculateResultMetadata() {
-    TransformResultMetadata elseStatementResultMetadata = _elseStatement.getResultMetadata();
-    DataType dataType = elseStatementResultMetadata.getDataType();
-    Preconditions.checkState(elseStatementResultMetadata.isSingleValue(),
-        "Unsupported multi-value expression in the ELSE clause");
+    DataType dataType;

Review Comment:
   Good catch



##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -128,16 +129,22 @@ private void constructStatementListLegacy(List<TransformFunction> arguments) {
     for (int i = numWhenStatements; i < numWhenStatements * 2; i++) {
       _thenStatements.add(arguments.get(i));
     }
-    if (arguments.size() % 2 != 0) {
+    if (arguments.size() % 2 != 0 && isNotNullLiteralTransformation(arguments.get(arguments.size() - 1))) {
       _elseStatement = arguments.get(arguments.size() - 1);
     }
   }
 
   private TransformResultMetadata calculateResultMetadata() {
-    TransformResultMetadata elseStatementResultMetadata = _elseStatement.getResultMetadata();
-    DataType dataType = elseStatementResultMetadata.getDataType();
-    Preconditions.checkState(elseStatementResultMetadata.isSingleValue(),
-        "Unsupported multi-value expression in the ELSE clause");
+    DataType dataType;
+    if (_elseStatement == null) {
+      // only possible in case of Null Literal transformation.

Review Comment:
   This comment is not accurate. I believe the existing code has a bug that can throw `NPE` when there is no else statement. Suggest just removing this comment



##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -257,6 +264,14 @@ public TransformResultMetadata getResultMetadata() {
     return _resultMetadata;
   }
 
+  private boolean isNotNullLiteralTransformation(TransformFunction function) {
+    if (Objects.equals(function.getName(), LiteralTransformFunction.FUNCTION_NAME)) {

Review Comment:
   ```suggestion
       if (function instanceof LiteralTransformFunction) {
   ```



##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunction.java:
##########
@@ -257,6 +264,14 @@ public TransformResultMetadata getResultMetadata() {
     return _resultMetadata;
   }
 
+  private boolean isNotNullLiteralTransformation(TransformFunction function) {
+    if (Objects.equals(function.getName(), LiteralTransformFunction.FUNCTION_NAME)) {
+      LiteralTransformFunction literalFUnction = (LiteralTransformFunction) function;

Review Comment:
   (typo)
   ```suggestion
         LiteralTransformFunction literalFunction = (LiteralTransformFunction) function;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org